Multi-temporal 3D model registration method and system based on adaptive multi-level matching

Abstract

The present invention provides a multi-temporal three-dimensional model registration method and system based on adaptive multi-level matching, the method comprising: rasterizing the three-dimensional model to obtain the original DSM and DOM; evenly dividing the DOM into blocks and generating a multi-level DOM pyramid by pooling; using the topmost DOM level with the lowest resolution to complete the rough registration; calculating the approximate model offset according to the result of the rough registration, applying the transformation matrix of the rough registration to the original three-dimensional model and aligning the three-dimensional model; adaptively selecting a suitable level for two-dimensional precise matching according to the point distribution density of the rough registration, locally screening and overall screening the two-dimensional precise matching results to obtain the final same-name point pairs; bringing the same-name point pairs into the solution equation to obtain the affine transformation model, and applying the affine change matrix to the three-dimensional model. The present invention is used to solve the problem that the current automatic registration algorithm based on point cloud is difficult to achieve a balance between accuracy and data scale, and improving the accuracy is bound to generate larger-scale intermediate data and occupy more computing resources.

Description

Technical Field

[0001] The present invention relates to the fields of digital photogrammetry, graphic image processing and computer vision technology, and in particular to a multi-phase three-dimensional model registration method and system based on adaptive multi-level matching. Background Art

[0002] With the development and rise of real-life 3D technology, the construction of digital cities and smart cities has been rapidly promoted. Technical fields such as cultural relics protection, smart cities, and natural resource change monitoring all require the support of high-precision 3D models. However, in actual operations, due to factors such as survey area restrictions and data collection conditions, the collection cycle of 3D model data is often unstable, ranging from a few days to several months or even more than a year. This unstable collection cycle will cause deviations and misalignments in the absolute orientation of 3D model data in multiple time phases, which will seriously affect subsequent data maintenance and change monitoring processes, and introduce significant errors. Therefore, in the current mainstream multi-temporal data processing process, the registration of 3D models has become a crucial link.

[0003] 3D model registration refers to the precise alignment and matching of two or more 3D models to ensure that they have consistent positions and orientations in the same coordinate system. This process usually involves advanced technologies in computer vision and computational geometry. In response to the challenges of multi-temporal data registration, the current mainstream processing methods can be divided into two categories: manual registration and point cloud-based automatic registration.

[0004] The manual registration method has become the most reliable means of multi-temporal model registration due to its stability and robustness. This method uses a third-party 3D model browsing software to manually select the same-name fixed feature points in the two 3D models to be registered as the basis for registration, then measures the 3D coordinates of these same-name points, and jointly establishes multiple same-name point pair equations to finally solve the affine transformation equation between the 3D models. Applying this transformation equation to the original model completes the registration work. However, the manual registration method is inefficient, consumes a lot of human resources, and is difficult to cope with the challenges of large-scale data processing. At the same time, in practical applications, factors such as the scarcity of fixed feature features and the lack of semantic information, as well as the proficiency of operators, may have a significant impact on the registration accuracy.

[0005] The automatic registration method based on point cloud effectively solves the efficiency problem of manual registration. This method converts the three-dimensional model into a discrete or regular point cloud, extracts the geometric feature points or outlier feature points of the point cloud as candidate same-name point sets through automatic algorithms, and then uses feature operators to screen and match them. After obtaining the same-name point pairs, subsequent solutions are performed. In addition, the point cloud-based registration method has derived a variety of algorithmic techniques, such as iterative nearest neighbor search algorithm, machine learning hyperplane feature matching algorithm, multi-source data hybrid filtering algorithm, etc. However, these methods still face many challenges in practical applications. Discrete point clouds are easily affected by noise points, especially when the quality and accuracy of the three-dimensional model cannot be guaranteed. At the same time, point cloud registration usually involves the processing of large-scale data, and the computational complexity is high. Therefore, efficient registration algorithms and optimization methods are crucial for real-time or near-real-time applications, but a balance needs to be struck between computational efficiency and accuracy.

[0006] In short, it is necessary to study a universal robust multi-phase 3D model registration method to ensure that the multi-phase 3D model data are in the same dimension, so that the 3D changes of the application scenarios can be discovered, which will help the development of industry applications such as engineering progress monitoring and surface 3D change monitoring. Summary of the invention

[0007] In view of the technical problems existing in the prior art, the present invention provides a multi-phase three-dimensional model registration method and system based on adaptive multi-level matching, which is used to solve the problem that the current point cloud-based automatic registration algorithm is difficult to achieve a balance between accuracy and data scale. Improving the accuracy will inevitably generate larger-scale intermediate data and occupy more computing resources. At the same time, the accuracy of solving discrete point clouds in the presence of more noise points will also fluctuate greatly.

[0008] According to a first aspect of the present invention, the present invention provides a multi-phase three-dimensional model registration method based on adaptive multi-level matching, comprising the following steps:

[0009] Input two 3D models of different phases, rasterize the 3D models, and obtain the original digital surface model and digital orthophoto model;

[0010] The digital orthophoto model is evenly divided into blocks, and pooled to generate a multi-level digital orthophoto model pyramid;

[0011] Use the Light Glue deep learning model to perform coarse registration on the top-level, lowest-resolution digital orthophoto model;

[0012] Calculate the transformation matrix of the rough registration result, and apply the transformation matrix of the rough registration result to the original 3D model to roughly align the 3D model;

[0013] The weight of each block is calculated according to the point distribution density obtained by coarse registration, and the appropriate level is adaptively selected for two-dimensional image matching to obtain a two-dimensional precise matching result;

[0014] The two-dimensional precise matching results are locally screened and globally screened to obtain the final pairs of points with the same name;

[0015] The image coordinates of the same-name points are converted into object coordinates and the same-name point equations are established simultaneously. The two-dimensional affine transformation matrix is ​​obtained by solving the equations. The two-dimensional affine transformation matrix and the simultaneous equations are applied to the three-dimensional model to complete the three-dimensional model registration.

[0016] Based on the above technical solution, the present invention can also make the following improvements.

[0017] Optionally, rasterizing the three-dimensional model to obtain the original digital surface model and the digital orthophoto model includes:

[0018] The three-dimensional models of both phases are rasterized and sampled, the digital surface model value of the three-dimensional model itself is selected, vertical sampling is performed within the bounding box of the three-dimensional model, the information of the highest point among the cumulative sampling points is selected as the raster information, the elevation information is assigned to the digital surface model data, and the color RGB information is assigned to the digital orthophoto model.

[0019] Optionally, the pooling to generate a multi-level digital orthophoto model pyramid includes:

[0020] Each block retains a multi-level image pyramid structure, where the bottom layer of the pyramid is the original block-by-block digital orthophoto model image, the second layer is the image of the first layer after proportional pooling, and so on, forming the local pyramid structure of the current block. After all blocks have generated all pyramid levels, the pyramidization of the entire scene is completed.

[0021] Optionally, the using the topmost digital orthophoto model level with the lowest resolution to complete the coarse registration includes:

[0022] The images at the top of the pyramid of all blocks of the two phases of data are merged to obtain a lower-resolution digital orthophoto model, and the low-resolution digital orthophoto model is input into the LightGlue model to obtain the roughly registered same-name point pairs.

[0023] Optionally, the calculating the weight of each block according to the point distribution density obtained by the rough registration, adaptively selecting a suitable level for two-dimensional image matching, and obtaining a two-dimensional precise matching result includes:

[0024] The adaptive hierarchical matching of the block digital ortho models is completed, and each pair of block digital ortho models with the same number of rows and columns in the two phases of digital ortho model data is used as the matching area;

[0025] After selecting the corresponding block level, the image of the corresponding level is used as the influence to extract and match the feature points.

[0026] Optionally, the matching of feature points includes:

[0027] The feature descriptor of each feature point is extracted, different weight values ​​are selected to be multiplied to the feature vector, and a dilution parameter is selected. After the dilution parameter is multiplied to the feature descriptor, the steps are repeated until the best matching point of the same name is found.

[0028] Optionally, the locally screening and overall screening of the two-dimensional precise matching results to obtain the final pairs of points with the same name include:

[0029] In each block area, the Ransac algorithm is used to filter out mismatched points, and the same-name point pairs after each block screening are merged into a large same-name point candidate set. Then the overall Ransac algorithm is performed to obtain the final same-name point set in the complete scene.

[0030] Optionally, the solving the equation to obtain a two-dimensional affine transformation matrix includes:

[0031] The two-dimensional affine transformation matrix is ​​obtained by the least squares method and converted into a three-dimensional point set for calculation by the following formula; the three-dimensional coordinates of each point with the same name are:

[0032]

[0033]

[0034]

[0035] in, are the object coordinates recorded in the digital orthophoto model file; Zs is the elevation value recorded at the same pixel position in the digital surface model; r represents the resolution of the rasterized digital orthophoto model. , The pixel coordinates of the points with the same name.

[0036] According to a second aspect of the present invention, a multi-phase 3D model registration system based on adaptive multi-level matching is provided, comprising:

[0037] The original data acquisition module is used to input the three-dimensional models of two different phases, rasterize the three-dimensional models, and obtain the original digital surface model and digital orthophoto model;

[0038] The coarse registration module is used to evenly divide the digital orthophoto model into blocks and generate a multi-level digital orthophoto model pyramid through pooling; the coarse registration is completed using the top-level digital orthophoto model level with the lowest resolution;

[0039] The fine registration module is used to solve the transformation matrix of the rough registration result, and apply the transformation matrix of the rough registration result to the original 3D model to roughly align the 3D model; calculate the weight of each block according to the point distribution density obtained by the rough registration, and adaptively select the appropriate level for 2D image matching to obtain the 2D fine matching result;

[0040] The three-dimensional model registration module is used to screen the two-dimensional precise matching results to obtain the final pairs of points with the same name; the pairs of points with the same name are substituted into the solution equation to obtain the two-dimensional affine transformation matrix, and the two-dimensional affine transformation matrix simultaneous equation group is applied to the three-dimensional model to complete the three-dimensional model registration.

[0041] According to a third aspect of the present invention, an electronic device is provided, comprising a memory and a processor, wherein the processor is configured to implement the steps of the multi-phase three-dimensional model registration method based on adaptive multi-level matching when executing a computer program stored in the memory.

[0042] Technical effects and advantages of the present invention:

[0043] The multi-phase three-dimensional model registration method and system based on adaptive multi-level matching provided by the present invention use block matching logic to solve the problems of uneven distribution of same-name points in the overall matching strategy when facing large-area terrain data, and difficulty in unifying the screening thresholds of different types of same-name points as a whole. At the same time, the method uses an adaptive multi-level matching method to solve the algorithm calculation redundancy caused by the traditional overall matching method performing the highest precision point matching on all local features. Because for a large area scene, different regional feature weights must be different, for low-weight areas, only the low-resolution data of the top layer of the pyramid is needed, which can greatly reduce the amount of calculation. According to the feature distribution, adaptive calculation of different matching and screening thresholds for different regions is also conducive to improving the proportion of high-quality feature points (such as fixed man-made objects, or hardened surfaces with distinct features, etc.), thereby improving the accuracy of registration. It is used to solve the problem that the current automatic registration algorithm based on point cloud is difficult to achieve a balance between accuracy and data scale, and improving accuracy is bound to generate larger-scale intermediate data and occupy more computing resources. At the same time, the accuracy of discrete point clouds in the case of more noise points will also fluctuate greatly. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of a multi-phase 3D model registration method based on adaptive multi-level matching provided by an embodiment of the present invention;

[0045] Figure 2 It is a local image pyramid map after segmentation provided by an embodiment of the present invention;

[0046] Figure 3 is a LightGlue rough registration result image provided by an embodiment of the present invention;

[0047] Figure 4 This is a local block precise matching result diagram provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] It is understandable that, based on the defects in the background technology, the embodiment of the present invention proposes a multi-phase 3D model registration method based on adaptive multi-level matching, as shown in the following example: Figure 1 As shown, the following steps are included:

[0050] Step S1, inputting two three-dimensional models of different phases, rasterizing the three-dimensional models, and obtaining original high-precision digital surface models (DSM) and digital orthophoto models (DOM);

[0051] It should be noted that the Digital Surface Model (hereinafter referred to as DSM) represents the earth's surface and includes all objects on it. It refers to a surface elevation model based on a raster data structure that includes the heights of surface buildings, bridges, and trees. It contains not only the elevation information of the terrain, but also the elevation information of the surface cover. In the raster DSM, the surface is divided into a series of regular grid cells (or pixels), each of which corresponds to an elevation value, which represents the average or highest elevation of all surface features in the grid cell.

[0052] Digital Orthographic Model (DOM) is a digital orthophoto dataset generated by using digital elevation model (DEM) to correct the projection difference of digitized aerial photography images pixel by pixel, mosaic, and cut according to the national basic scale topographic map range. It is a digital orthophoto dataset generated by digital differential correction and mosaicking of aerial (or aerospace) photos and cutting according to a certain map range. It is an image with both map geometric accuracy and image characteristics.

[0053] In this embodiment, the three-dimensional model is rasterized to obtain the original high-precision DSM and DOM including:

[0054] The input 3D model data of two different phases are rasterized and sampled, and the appropriate resolution is selected (here the GSD value of the 3D model itself is generally selected to avoid loss of accuracy in the rasterization process as much as possible). Vertical sampling is performed within the bounding box of the 3D model, and the information of the highest point among the cumulative sampling points is selected as the raster information. The elevation information is assigned to the DSM data, and the color RGB information is assigned to the DOM.

[0055] Step S2, divide the DOM into blocks evenly and generate a multi-level DOM pyramid by pooling;

[0056] In this embodiment, the DOM is evenly divided into blocks, including: the DOM of the two phases of data to be aligned is evenly divided into different blocks according to a fixed step size, generally 1024 or 2048 is selected as the step size of the block, and then the object space coordinate position of each block is recorded in the tfw file of each divided block, and the row and column numbers of the block are recorded to facilitate the subsequent block merging work.

[0057] The pooling method for generating a multi-level DOM pyramid includes: each block retains a multi-level image pyramid structure, as shown in the attached figure. Figure 2 As shown, the bottom layer of the pyramid is the original block DOM image, the second layer is the image of the first layer pooled at a ratio of 1 / 4, and so on. Each layer is pooled from the image of the previous layer, forming a local pyramid structure of the current block. Generally, 4 layers of the pyramid are selected (at least 3 layers, and at most 6 layers); after all blocks are generated into all pyramid levels, the pyramidization of the entire scene is completed.

[0058] Step S3: Use the Light Glue deep learning model to perform a rough registration of the top-level DOM with the lowest resolution;

[0059] Since the two-phase 3D model data has a large survey area, it is extremely slow to directly use the original resolution DOM for registration calculation, and it is difficult to complete the cross-validation screening by matching the feature points of large frame images at the original resolution, so block-by-block local registration is necessary. However, the spatial coordinates of the two-phase 3D model data are far apart, and there is no prior knowledge to clarify the corresponding matching relationship between the blocks in the two-phase data, so before registration, the two-phase models need to be roughly in similar positions, that is, the step of coarse registration needs to be completed.

[0060] Rough registration requires global matching. Traditional image matching cannot handle high-precision feature matching of large-frame images. After reducing the resolution, traditional matching operators cannot guarantee correctness. Therefore, we use the LightGlue model in deep learning here. This model can extract a large number of matching points with the same name based on corner features and feature distribution rules. Its advantages are that there are many feature points and it is not affected by the resolution of the image. It also performs well at lower resolutions. The disadvantage is that the accuracy of the feature points is not high enough (3-8 pixels). The lower accuracy is unacceptable for fine registration, but it can be well applied in the coarse registration process.

[0061] The use of the Light Glue deep learning model to perform coarse registration of the top-level DOM with the lowest resolution includes:

[0062] The images of the top layers of the pyramids of all blocks of the two phases of data are merged to obtain a lower resolution DOM. The low resolution DOM is then input into the LightGlue model to obtain the roughly registered same-name point pairs. The matching effect is shown in the attached figure. Figure 3 As shown, the average offset is calculated by the object coordinates of the same-name point pairs:

[0063]

[0064] in is the number of pairs of points with the same name, and P and Pc are the object coordinate values ​​of the points with the same name in their respective 3D models.

[0065] After completing the offset solution of the rough registration in this step, the number and distribution of the same-name point pairs involved in the calculation in each block (the average distance between the same-name points) are recorded at the same time. This information will provide a basis for adaptive grading in the subsequent fine registration step.

[0066] Step S4, calculating the transformation matrix of the result of the rough registration, and applying the transformation matrix of the result of the rough registration to the original three-dimensional model to roughly align the three-dimensional model;

[0067] The above rough registration result Applied to the original model, the two phases of data are roughly close. At this time, there is still some slight rotation offset in the two phases of 3D models, but they are already very close. Therefore, in subsequent calculations, the DOM blocks of the two phases of model data only need to correspond according to the row and column numbers (for example, the nth row and mth column in the DOM block data of the two phases of data are the corresponding areas with the same name).

[0068] Step S5: Calculate the weight of each block according to the point distribution density obtained by coarse registration, adaptively select the appropriate level for two-dimensional image matching, and obtain the two-dimensional precise matching result.

[0069] Complete adaptive hierarchical precise matching of block DOM;

[0070] This step will complete the adaptive hierarchical matching of the block DOM, and take each pair of block DOM with the same number of rows and columns in the two phases of DOM data as the matching area; each block area has a multi-level image pyramid. In the fine matching process, the DOM image at which level is used is determined according to the result of the rough registration. Because in the rough registration, the distribution of the same-name point pairs matched by the deep learning model reflects the feature richness of the corresponding area. The more the number of same-name point pairs, the denser the distribution of the same-name points (the smaller the average distance of the same-name points), and the richer the features of the block area, so higher-resolution DOM image data should be used, that is, the image at the bottom of the pyramid. The selection formula is as follows:

[0071]

[0072]

[0073]

[0074] in, is the number of roughly registered points of the same name in the current block as the total The proportion of is the average distance between pairs of points with the same name in the current block, and L is the number of pyramid levels finally selected.

[0075] After selecting the corresponding block level, the feature point extraction and matching process can be carried out based on the image of the corresponding level. The precision of matching the same-name points in the precise registration process is high, so the SIFT algorithm is selected as the feature extraction algorithm in this step. The accuracy of SIFT feature points can reach the sub-pixel level, and the operation efficiency is stable. The feature extraction results are also very robust, which is very suitable for image matching of small-scale images. At the same time, the descriptor of SIFT features is a 128-dimensional feature vector, which is very suitable for subsequent weighted matching. The feature extraction process of SIFT algorithm is very mature and classic, so it is not described here. This method improves the subsequent feature point matching process. After the above SIFT feature points are extracted, each feature point will have its own feature descriptor. The matching process corresponding to the pyramid images of different levels will select different weighted values ​​to multiply the feature vector. After the experiment, the four pyramid levels from 1 to 4 are taken as examples. The general weights are selected as 0.1, 0.4, 0.5, and 0.75, which can also be called dilution parameters. After multiplying the above weights to the feature descriptor, this method changes the Euclidean distance judgment standard used in traditional matching to the Mahalanobis distance to ensure the correctness of matching under different dilution parameters. After the above steps, the best matching points of the same name can be found.

[0076] Step S6, performing local and overall screening on the two-dimensional precise matching results to obtain the final pairs of points with the same name;

[0077] The above-mentioned precise registration results have been preliminarily obtained. Due to the different feature strengths and semantic information between blocks, there are deviations and inconsistencies between the precise registration results of each block. In this step, the local and overall Ransac methods are used for screening. That is, in each block area, the Ransac algorithm is used to filter out the mismatched points, and then the same-name point pairs after each block screening are merged into a large same-name point candidate set, and then the overall Ransac algorithm is performed again to obtain the final same-name point set in the complete scene. The same-name point pair extraction effect is shown in the attached figure. Figure 4 shown.

[0078] Step S7, convert the image coordinates of the same-name points into object coordinates and establish a group of equations for the same-name points, solve the equations to obtain a two-dimensional affine transformation matrix, and apply the two-dimensional affine transformation matrix and the group of equations to the three-dimensional model to complete the registration of the three-dimensional model.

[0079] This step mainly performs the simultaneous equations of the same-name point pairs, and obtains the unknown values ​​of the affine transformation matrix through the least squares method. The above steps only obtain the two-dimensional pixel coordinates in the DOM data. When solving the actual affine transformation matrix, a three-dimensional point set needs to be used for calculation. From the pixel consistency of the raster data DOM and DSM, it can be obtained that the three-dimensional coordinates of each same-name point are:

[0080]

[0081]

[0082]

[0083] in, are the object coordinates of the upper left corner recorded in the DOM file; Zs is the elevation value recorded at the same pixel position in the DSM, and r represents the resolution of the rasterized DOM. , The pixel coordinates of the points with the same name.

[0084] The above steps can calculate the three-dimensional coordinates of all points with the same name, and then we can solve the system of equations. An affine transformation matrix consists of three parts: scaling factor , rotation matrix R, offset matrix T, the three-dimensional coordinate relationship of the same-name point pairs is as follows:

[0085] .

[0086] Each pair of points with the same name can establish one of the above equations. When there are more than 15 pairs of points with the same name forming a system of equations, the least squares algorithm can be used to solve the above unknowns. , R matrix, T matrix. So far, the affine transformation matrix has been solved. Applying the affine transformation matrix to the two-phase model can complete the registration of the three-dimensional model.

[0087] The multi-temporal 3D model registration method based on adaptive multi-level matching of the present invention utilizes DOM (digital orthophoto model) and DSM (digital surface model) data generated by the 3D model, generates a multi-level image pyramid according to the original high-precision DSM and DOM, uses the top-level low-resolution DOM image for coarse registration, and then performs fine registration according to the result of the coarse registration and the distribution in the multi-level DOM image pyramid to obtain two-dimensional same-name point pairs, and then utilizes the pixel consistency of the raster data to directly convert the two-dimensional image coordinates into three-dimensional object space coordinates, and brings them into the same-name point pair solution equation to calculate the affine transformation model required for registration.

[0088] At the same time, by using rasterized data, it is easy to perform local block processing and pyramid multi-level processing. The independent areas after blocking can be easily solved in parallel. The combination of coarse and fine registration can effectively control the calculation scale of least squares calculation, improving the calculation efficiency and accuracy. Multi-level adaptive matching can use low-resolution data in weak texture areas and weak semantic areas and assign low confidence weights, and use high-resolution data in strong texture areas and strong semantic areas and assign high confidence weights. This can avoid the difficulty of balancing accuracy and data scale when the whole scene is consistent.

[0089] It should be noted that the present invention has been fully verified in actual projects and has been applied in multiple urban real-life 3D modeling projects, mine intelligent monitoring projects and geological survey projects. It can still maintain a fairly high registration accuracy when the multi-temporal data itself has uneven distribution of changing areas and the overall scene area is large. The following table shows the horizontal mean error and elevation mean error of the registration after testing multiple groups of test data. The results show that the horizontal error and elevation error of the registration are relatively small. At the same time, it is also very stable in applications on various types of terrains, and the method has good robustness.

[0090]

[0091] According to a second aspect of the present invention, a multi-phase 3D model registration system based on adaptive multi-level matching is provided, comprising:

[0092] The original data acquisition module is used to input the three-dimensional models of two different phases, rasterize the three-dimensional models, and obtain the original digital surface model and digital orthophoto model;

[0093] The coarse registration module is used to evenly divide the digital orthophoto model into blocks and generate a multi-level digital orthophoto model pyramid through pooling; the coarse registration is completed using the top-level digital orthophoto model level with the lowest resolution;

[0094] The fine registration module is used to solve the transformation matrix of the rough registration result, and apply the transformation matrix of the rough registration result to the original 3D model to roughly align the 3D model; calculate the weight of each block according to the point distribution density obtained by the rough registration, and adaptively select the appropriate level for 2D image matching to obtain the 2D fine matching result;

[0095] The three-dimensional model registration module is used to screen the two-dimensional precise matching results to obtain the final pairs of points with the same name; the pairs of points with the same name are substituted into the solution equation to obtain the two-dimensional affine transformation matrix, and the two-dimensional affine transformation matrix simultaneous equation group is applied to the three-dimensional model to complete the three-dimensional model registration.

[0096] It can be understood that the multi-phase three-dimensional model registration system based on adaptive multi-level matching provided by the present invention corresponds to the multi-phase three-dimensional model registration method based on adaptive multi-level matching provided in the aforementioned embodiments. The relevant technical features of the multi-phase three-dimensional model registration system based on adaptive multi-level matching can refer to the relevant technical features of the multi-phase three-dimensional model registration method based on adaptive multi-level matching, which will not be repeated here.

[0097] In addition, an embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the multi-phase three-dimensional model registration method based on adaptive multi-level matching are implemented.

[0098] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0099] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-phase 3D model registration method based on adaptive multi-level matching, characterized in that: The following steps are involved: Input two 3D models of different phases, rasterize the 3D models respectively, and obtain the original digital surface model and digital orthophoto model; The digital orthophoto model is evenly divided into blocks, and pooled to generate a multi-level digital orthophoto model pyramid; Use the Light Glue deep learning model to perform coarse registration on the top-level, lowest-resolution digital orthophoto model; Calculate the transformation matrix of the rough registration result, and apply the transformation matrix of the rough registration result to the original 3D model to roughly align the 3D model; The weight of each block is calculated according to the point distribution density obtained by the rough registration, and the appropriate level is adaptively selected for two-dimensional image matching to obtain a two-dimensional precise matching result; including: completing the adaptive hierarchical matching of the block digital ortho models, and taking each pair of block digital ortho models with the same number of rows and columns in the two phases of digital ortho model data as the matching area; Select the corresponding block level, use the image of the corresponding level as the feature image to extract and match the feature points; The two-dimensional precise matching results are locally screened and globally screened to obtain the final pairs of points with the same name; The image coordinates of the same-name points are converted into object coordinates and the same-name point equations are established simultaneously. The two-dimensional affine transformation matrix is ​​obtained by solving the equations. The two-dimensional affine transformation matrix and the simultaneous equations are applied to the three-dimensional model to complete the three-dimensional model registration.

2. The multi-phase 3D model registration method based on adaptive multi-level matching according to claim 1, characterized in that: The input of two three-dimensional models at different time phases, rasterizing the three-dimensional models, and obtaining the original digital surface model and digital orthophoto model comprises: The three-dimensional models of two different phases are input into raster samples, the digital surface model value of the three-dimensional model itself is selected, vertical sampling is performed within the bounding box of the three-dimensional model, the information of the highest point among the cumulative sampling points is selected as the raster information, the elevation information is assigned to the digital surface model data, and the color RGB information is assigned to the digital orthophoto model.

3. The multi-temporal 3D model registration method based on adaptive multi-level matching according to claim 1, characterized in that: The pooling to generate a multi-level digital orthophoto model pyramid includes: A multi-level image pyramid structure is retained for each block, where the bottom layer of the pyramid is the digital orthophoto model image of the original block, the second layer is the image of the first layer after proportional pooling, and so on, forming the local pyramid structure of the current block. After all the blocks have generated all the pyramid levels, the pyramidization of the entire scene is completed.

4. The multi-temporal 3D model registration method based on adaptive multi-level matching according to claim 1, characterized in that: The use of the Light Glue deep learning model to perform a rough registration of the top-level digital orthophoto model with the lowest resolution includes: The images of the top layers of the pyramids of all blocks in the digital orthophoto image data after rasterization of the model data of two different phases are merged to obtain a low-resolution digital orthophoto model. The low-resolution digital orthophoto model is input into the LightGlue model to obtain the roughly registered same-name point pairs.

5. The multi-temporal 3D model registration method based on adaptive multi-level matching according to claim 1, characterized in that: The matching of feature points comprises: Extract the feature descriptor of each feature point, select different weight values ​​to multiply the feature vector, select a dilution parameter, multiply the dilution parameter to the feature descriptor, and repeat this step until the best matching point of the same name is found.

6. The multi-temporal 3D model registration method based on adaptive multi-level matching according to claim 1, characterized in that: The method of performing local screening and overall screening on the two-dimensional precise matching results to obtain the final pairs of points with the same name includes: In each block area, the Ransac algorithm is used to filter out mismatched points, and the same-name point pairs after each block screening are merged into a large same-name point candidate set. Then the overall Ransac algorithm is performed to obtain the final same-name point set in the complete scene.

7. The multi-temporal 3D model registration method based on adaptive multi-level matching according to claim 1, characterized in that: The two-dimensional affine transformation matrix obtained by solving the equation includes: The two-dimensional affine transformation matrix is ​​obtained by the least squares method and converted into a three-dimensional point set for calculation using the following formula. The three-dimensional coordinates of each point with the same name are: in, are the object coordinates recorded in the digital orthophoto model file; Zs is the elevation value recorded at the same pixel position in the digital surface model; r represents the resolution of the rasterized digital orthophoto model. , The pixel coordinates of the points with the same name.

8. A multi-temporal 3D model registration system based on adaptive multi-level matching, characterized in that: include: The original data acquisition module is used to input the three-dimensional models of two different phases, rasterize the three-dimensional models, and obtain the original digital surface model and digital orthophoto model; The coarse registration module is used to evenly divide the digital orthophoto model into blocks and generate a multi-level digital orthophoto model pyramid through pooling; the coarse registration is completed using the top-level digital orthophoto model level with the lowest resolution; The fine registration module is used to solve the transformation matrix of the rough registration result, and apply the transformation matrix of the rough registration result to the original 3D model to roughly align the 3D model; The weight of each block is calculated according to the point distribution density obtained by the rough registration, and a suitable level is adaptively selected for two-dimensional image matching to obtain a two-dimensional precise matching result; including: completing the adaptive hierarchical matching of the block digital ortho models, and taking each pair of block digital ortho models with the same number of rows and columns in the two phases of digital ortho model data as the matching area; After selecting the corresponding block level, the image of the corresponding level is used as the feature image to extract and match the feature points; The three-dimensional model registration module is used to screen the two-dimensional precise matching results to obtain the final pairs of points with the same name; the pairs of points with the same name are substituted into the solution equation to obtain the two-dimensional affine transformation matrix, and the two-dimensional affine transformation matrix simultaneous equation group is applied to the three-dimensional model to complete the three-dimensional model registration.

9. An electronic device, characterized in that: It comprises a memory and a processor, wherein the processor is used to implement the multi-phase three-dimensional model registration method based on adaptive multi-level matching as described in any one of claims 1 to 7 when executing the computer program stored in the memory.

Images