A multi-period real scene three-dimensional model automatic registration method

Abstract

A multi-period real scene three-dimensional model automatic registration method, comprising the following steps: extracting the vertex, texture, vertex corresponding texture coordinate and triangular net index data of the bottom layer model data of the real scene three-dimensional model; extracting the SIFT feature of the texture information; calculating the spatial position of the SIFT feature according to the triangular net plane constraint; judging the similarity of the spatial constraint SIFT feature; eliminating the gross error of the same name point based on the three-dimensional similarity conversion model constraint, and solving the three-dimensional similarity conversion model parameter of the three-dimensional real scene model to be registered relative to the reference three-dimensional real scene model; updating the spatial position of all vertices of the registered three-dimensional real scene model according to the three-dimensional similarity conversion model parameter; and outputting the real scene three-dimensional model to be registered after the vertex is updated. The application can automatically and quickly obtain the conversion parameter of the multi-period real scene three-dimensional model, unify the spatial position of the three-dimensional real scene model to be registered to the spatial coordinate system of the reference three-dimensional real scene model, and can carry out change detection analysis with high geometric precision based on the multi-period real scene three-dimensional model.

Description

Technical Field

[0001] The present invention belongs to the field of unmanned aerial vehicle (UAV) photogrammetry, and in particular relates to a method for automatically registering multi-period real-scene three-dimensional models using UAV photogrammetry. Background Art

[0002] UAV low-altitude photogrammetry has the advantages of strong timeliness, high resolution, flexible acquisition methods, and diversified sensors. Digital products such as digital orthophotos of the survey area, digital surface models, three-dimensional dense point clouds, and three-dimensional real-scene models generated based on high-resolution images obtained by drones are widely used in basic surveying and mapping, emergency rescue, geological disaster monitoring, and cultural relics protection. Geometric positioning accuracy is one of the core indicators that determine the application scenarios of digital products, and obtaining multi-period three-dimensional real-scene models with the same spatial reference and unified geometric positioning accuracy is the basic data for realizing intelligent management of large-scale engineering surveys. However, the existing UAV photogrammetry data processing technology process is highly dependent on evenly distributed ground control points. For applications based on multi-period real-scene three-dimensional models for change detection, there are the following two problems:

[0003] 1. For large-scale construction sites, each drone data collection requires the deployment of control points, which is inflexible and time- and economically expensive.

[0004] 2. The spatial benchmarks of multiple phases of real-scene 3D models are not unified, making it difficult to directly use the spatial position of objects to reflect changes in the construction site. Summary of the Invention

[0005] In response to the above-mentioned defects in the existing technology, the present invention discloses a multi-period real-scene 3D model registration method, which can automatically and highly accurately obtain the three-dimensional similarity transformation parameters between multiple-period real-scene 3D models, so that the spatial position of the real-scene 3D model to be registered is unified to the reference real-scene 3D model coordinate system, and the link of laying out ground control points when acquiring the real-scene 3D model in each period can be omitted, which greatly improves the flexibility and economy of transformation monitoring based on drone-based 3D real-scene models.

[0006] The present invention discloses a multi-period real-scene three-dimensional model registration method, comprising the following steps:

[0007] S1, extracting the vertices, textures, texture coordinates corresponding to the vertices, and triangulated network index data of the bottom layer of the real 3D model;

[0008] S2, extract SIFT features of texture information;

[0009] S3, calculate the SIFT feature space position based on the triangulated mesh plane constraint;

[0010] S4, spatially constrained SIFT feature similarity determination;

[0011] S5, eliminating the gross errors of homonymous points based on the constraints of the 3D similarity transformation model, and solving the 3D similarity transformation model parameters of the 3D real scene model to be registered relative to the reference 3D real scene model;

[0012] S6, updating the spatial positions of all vertices of the 3D real scene model to be registered according to the 3D similarity transformation model parameters;

[0013] S7, outputting the real scene 3D model to be registered after the vertices are updated.

[0014] Preferably, in S3, the specific steps of calculating the SIFT feature space position based on the triangulated mesh plane constraint include:

[0015] a1, convert the extracted SIFT feature point coordinates into texture coordinates;

[0016] a2, search for the triangulated mesh contained in the current texture coordinates;

[0017] a3, interpolate the spatial coordinates of the current SIFT feature point based on the vertex spatial coordinates and vertex texture coordinates of the triangulated network.

[0018] Preferably, the method of interpolating the spatial coordinates of the current SIFT feature point based on the vertex spatial coordinates and vertex texture coordinates of the triangulated network is an interpolation method.

[0019] Preferably, the spatial coordinates of the current SIFT feature point are interpolated in two steps.

[0020] Preferably, the spatial coordinates of the current SIFT feature point are interpolated in two steps, namely: interpolating the plane coordinates of the SIFT feature point, and interpolating the elevation coordinates of the SIFT feature point.

[0021] Preferably, in S4, the specific method for determining the similarity of spatially constrained SIFT features includes:

[0022] b1, determine the spatial bounding box where the current SIFT feature point is located;

[0023] b2, search for the SIFT feature vector of the reference image within the current bounding box to form a set of feature vectors to be matched;

[0024] b3, calculate the similarity between the feature vector of the current SIFT feature point and the set of feature vectors to be matched, and determine the initial correspondence between the points with the same name.

[0025] Preferably, the determining of the spatial bounding box where the current SIFT feature point is located is performed by determining the spatial bounding box to be registered based on the geometric positioning accuracy of multiple-phase real-scene three-dimensional models.

[0026] Preferably, in S5, the specific steps of eliminating the gross errors of homonymous points based on the constraints of the three-dimensional similarity transformation model and calculating the three-dimensional similarity transformation model parameters of the three-dimensional real scene model to be registered relative to the reference three-dimensional real scene model include:

[0027] c1, randomly extract 3 pairs of points with the same name from the initial matching image pair;

[0028] c2, calculate the parameters of the 3D similarity transformation model based on three pairs of points of the same name;

[0029] c3, evaluate the number of inliers based on the three-dimensional similarity transformation model parameters and record the current number of inliers;

[0030] c4, repeat c1 to c3 until the upper limit of the cycle number;

[0031] c5, the three-dimensional similarity transformation model parameters with the largest number of inliers are taken as the optimal three-dimensional similarity transformation model parameters.

[0032] Preferably, the upper limit of the number of cycles is set to 100 times.

[0033] Preferably, in said S6, the specific step of updating the spatial positions of all vertices of the 3D real scene model to be registered according to the 3D similarity transformation model parameters includes:

[0034] d1, extract the coordinates of all vertices of the 3D model to be registered;

[0035] d2, performs coordinate transformation on the vertices extracted by d1 according to the optimal three-dimensional similarity transformation model parameters obtained by c5.

[0036] The beneficial effects of the present invention are: 1. The present invention automatically extracts SIFT feature points of the real-scene 3D model, and solves the 3D similarity transformation model parameters of the multi-period real-scene 3D model by eliminating the points of the same name after gross errors, thereby unifying the spatial reference of the multi-period real-scene 3D model and improving the relative positioning geometric accuracy of the multi-period real-scene 3D model; 2. The present invention maps the extracted SIFT feature point coordinates to the texture coordinate space, and solves the spatial point coordinates corresponding to the SIFT feature points based on the triangulation vertex coordinates and the texture coordinates, thereby solving the spatial positioning problem of the SIFT feature point coordinates; 3. The present invention can automatically and quickly obtain the transformation parameters of the real-scene 3D model to be aligned relative to the benchmark real-scene 3D model, and can be used for high-precision geometric alignment of multi-period drone real-scene 3D models. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flow chart of the present invention;

[0038] Figure 2 A schematic diagram of the composition of a three-dimensional real scene model with the highest resolution;

[0039] Figure 3Schematic diagram of the SIFT feature extraction results and triangulated network vertex texture coordinates of the real-scene 3D model texture map of the present invention;

[0040] Figure 4 Schematic diagram of the SIFT feature space position calculated by the present invention;

[0041] Figure 5 This is the residual distribution map of the points with the same name obtained by matching in the present invention;

[0042] Figure 6 This is a residual distribution diagram of the same-name points after the multi-period real-scene 3D model registration is completed based on the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention and the accompanying drawings.

[0044] SIFT, or Scale-invariant feature transform (SIFT), is a description used in the field of image processing.

[0045] The present invention discloses a multi-period real-scene three-dimensional model registration method, comprising the following steps:

[0046] S1, extracting the vertices, textures, texture coordinates corresponding to the vertices, and triangulated network index data of the bottom layer of the real 3D model;

[0047] Obtain a phase of real-life 3D model data, obtain the bottom layer of each block data in turn, that is, the model data with the highest resolution, extract and save the model vertices, texture maps, texture map coordinates corresponding to the vertices, and triangulated network index data connecting the vertices.

[0048] S2, extract SIFT features of texture information;

[0049] The SIFT algorithm implemented in parallel on GPU is used to extract the 128-dimensional SIFT feature vectors of all texture images, and the extracted SIFT feature vectors are saved.

[0050] S3, calculate the SIFT feature space position based on the triangulated mesh plane constraint;

[0051] The specific steps include:

[0052] a1, convert the extracted SIFT feature point coordinates into texture coordinates;

[0053] The origin of the SIFT feature point coordinate system is in the upper left corner of the texture image, where the x-axis coordinate is horizontal to the right and the y-axis coordinate is vertically downward, and the unit is pixel. The origin of the texture coordinate system is in the lower left corner of the texture image, where t x The axis coordinate system is horizontal to the right, t y The axis coordinate system is vertically upward, and the unit is 1. After obtaining the SIFT feature points, the image point coordinates are mapped to texture coordinates. The mapping method is shown in formula (1):

[0054]

[0055] In formula (1), wid is the width of the texture image, and hei is the height of the texture image.

[0056] a2, search for the triangulated mesh contained in the current texture coordinates;

[0057] The real-world 3D model is composed of triangulated meshes. The vertices of the triangulated mesh contain spatial coordinates in the world coordinate system and the corresponding texture coordinates in the texture map. Based on the texture coordinates of the SIFT feature points, the triangulated mesh containing the current texture coordinates is searched in sequence to obtain the spatial coordinates and texture coordinates of the triangulated mesh vertices.

[0058] a3, interpolate the spatial coordinates of the current SIFT feature point based on the vertex spatial coordinates and vertex texture coordinates of the triangulated network.

[0059] Interpolating the spatial coordinates of the current SIFT feature point is divided into two steps: interpolating the plane coordinates of the SIFT feature point and interpolating the elevation coordinates of the SIFT feature point.

[0060] First, the TIN texture coordinates and the TIN space coordinate plane position are defined as an affine transformation, as shown in equation (2):

[0061]

[0062] In formula (2), a, b, c, d, e, and f are the affine transformation model parameters from the triangulated mesh texture coordinates to the triangulated mesh space coordinate plane position, (X w ,Y w ) is the plane coordinate of the world coordinate system corresponding to the current SIFT feature point.

[0063] Secondly, the world coordinate system of the three vertices of the triangulated network is defined as a coplanar condition, and the world coordinate system of the three vertices of the triangulated network can be expressed as formula (3):

[0064]

[0065] In formula (3), Z w is the spatial point elevation coordinate corresponding to the current SIFT feature point, (X i ,Y i ,Zi ) i＝1,2,3 is the world coordinate of the three vertices of the triangulation network, then the elevation Z of the current SIFT feature point is w It can be calculated by formula (4):

[0066]

[0067] In formula (4), Y 21 =Y2-Y1, and so on for other variables.

[0068] S4, spatially constrained SIFT feature similarity determination;

[0069] Specific methods include:

[0070] b1, determine the spatial bounding box where the current SIFT feature point is located;

[0071] The benchmark real-world 3D model is divided into fixed intervals so that the number of feature points contained in a single grid does not exceed a threshold (the threshold is 2000-5000). A single grid is the search bounding box of the benchmark real-world 3D model, and all SIFT feature points within the current bounding box are used as a search unit.

[0072] b2, search for the SIFT feature vector of the reference image within the current bounding box to form a set of feature vectors to be matched;

[0073] With the search bounding box of the reference real-scene 3D model as the center, expand it by a fixed distance in all directions (the expansion distance is set based on the relative positioning error between the reference 3D model and the real-scene 3D model to be registered, generally taking a value of 20-50 meters). The expanded bounding box is the search bounding box of the real-scene 3D model to be registered, and the SIFT feature points contained in the bounding box are used as the units to be matched.

[0074] b3, calculate the similarity between the feature vector of the current SIFT feature point and the set of feature vectors to be matched, and determine the initial correspondence between the points with the same name.

[0075] Based on the SIFT feature points within the search unit of the reference real-world 3D model determined by b1, a similarity check is performed with the SIFT feature points within the search unit of the real-world 3D model to be registered determined by b2. This is based on the SIFT algorithm's similarity threshold (typically 0.7-0.9, with 0.8 recommended) and the threshold for the ratio of the distance between the nearest neighbor and the next nearest neighbor (typically 0.6-0.8, with 0.7 recommended). The similarity checks are then completed sequentially between the search bounding box of the reference real-world 3D model and the search bounding box of the real-world 3D model to be registered, obtaining an initial list of points of the same name between all reference real-world 3D models and the real-world 3D model to be registered.

[0076] S5, eliminating the gross errors of homonymous points based on the constraints of the 3D similarity transformation model, and solving the 3D similarity transformation model parameters of the 3D real scene model to be registered relative to the reference 3D real scene model;

[0077] The specific steps include:

[0078] c1, randomly extract 3 pairs of points with the same name from the initial matching image pair;

[0079] Randomly extract 3 pairs of points with the same name from the initial list of points with the same name obtained in b3.

[0080] c2, calculate the parameters of the 3D similarity transformation model based on three pairs of points of the same name;

[0081] The coordinate system transformation between the real-scene 3D model to be registered and the reference real-scene 3D model is defined as a 3D similarity transformation, and its transformation relationship is shown in formula (5):

[0082]

[0083] Among them, (X g ,Y g ,Z g ) is the spatial coordinate of the vertex of the real 3D model to be registered, (X b ,Y b ,Z b ) is the same as (X g ,Y g ,Z g ) corresponds to the reference real-scene 3D model vertex space coordinates. λ is the scale factor of the two-phase real-scene 3D model space coordinate system, (a1…b1…c3) is the relative rotation matrix element of the two-phase real-scene 3D model, (t x ,t y ,t z ) is the relative translation of the two real-scene 3D models. The definition of the rotation matrix elements is shown in equation (6):

[0084]

[0085] in, It is the rotation angle element of the relative rotation matrix of the two real-scene 3D models.

[0086] After obtaining the lists of three groups of points with the same name, based on the spatial coordinates of the SIFT feature points obtained by S3, the seven unknowns in formula (5) are solved according to the principle of least squares adjustment to obtain the three-dimensional similarity transformation model parameters of the three-dimensional real scene model to be registered relative to the reference three-dimensional real scene model.

[0087] c3, evaluate the number of inliers based on the three-dimensional similarity transformation model parameters and record the current number of inliers;

[0088] Based on the 3D similarity transformation model parameters obtained by solving C2, the spatial coordinate difference of the remaining initial list of same-name points in C1 is calculated according to formula (5). If the difference exceeds a certain threshold (usually 0.1-0.5 meters), it is considered an outlier; otherwise, it is an inlier. The number of inliers obtained by the current 3D similarity transformation model parameters is counted.

[0089] c4, repeat c1 to c3 until the upper limit of the cycle number;

[0090] The upper limit of the number of cycles is generally set to 100 times.

[0091] c5, the three-dimensional similarity transformation model parameters with the largest number of inliers are taken as the optimal three-dimensional similarity transformation model parameters.

[0092] According to the list of inliers obtained by the optimal 3D similarity transformation model parameters, an overall adjustment solution is performed according to C2 to obtain the optimal 3D similarity transformation model parameters of the real scene 3D model to be registered relative to the reference real scene 3D model.

[0093] S6, updating the spatial positions of all vertices of the 3D real scene model to be registered according to the 3D similarity transformation model parameters;

[0094] d1, extract the coordinates of all vertices of the 3D model to be registered;

[0095] d2, performs coordinate transformation on the vertices extracted in d1 according to the three-dimensional similarity transformation model parameters obtained in c5.

[0096] The coordinates of the vertices extracted by d1 are transformed according to the optimal three-dimensional similarity transformation model parameters obtained by c5. The transformation formula is formula (5).

[0097] S7, outputting the real scene 3D model to be registered after the vertices are updated;

[0098] Get the real scene 3D model to be registered after the vertex is updated, and based on the real scene 3D model after the vertex coordinates are updated, get the real scene 3D model with a unified spatial reference. The result is as follows Figure 4 shown.

[0099] The present invention is further verified and illustrated by the following examples:

[0100] ⑴Computer operating conditions

[0101] The executable program of the present invention was developed using Visual Studio 2017 C++ on a Windows 10 64-bit operating system to test the adaptability and accuracy of the method. The hardware platform was a Dell Precision 3630 workstation with a CPU i7-8700K 3.7 GHz, 64 GB DDR4 memory, a 512 GB SSD hard drive, and an NVIDIA Titan XP 12 GB graphics card.

[0102] ⑵Data source

[0103] The actual terrain in the data area is primarily mountainous and canyon-like. A full-frame DG4Pros drone camera was used to acquire 1,659 drone images with a ground spatial resolution of 2 cm. 3D automatic reconstruction was performed based on drone images acquired in 2020 to obtain a baseline 3D model of the real scene. 3D automatic reconstruction was performed based on drone images acquired in 2021 to obtain a 3D model of the real scene to be registered.

[0104] ⑶Experimental content

[0105] First, verify the process of calculating the spatial position of the SIFT feature by calculating the plane constraint of the triangulated network. Obtain a data file of a certain block of the benchmark real-scene 3D model, extract the texture map, vertex coordinates, texture coordinates corresponding to the vertex coordinates, and the triangulated network composed of the vertices of its high-resolution 3D model, and the results are as follows: Figure 2 Extract SIFT feature points from the texture map and map the texture coordinates corresponding to the triangulated mesh vertices to the texture map. The result is as follows: Figure 3 According to the present invention, the spatial position of SIFT feature points is calculated and superimposed with the three-dimensional model and triangulated network vertices. The result is shown as follows. Figure 4 shown.

[0106] Secondly, the calculation results of the three-dimensional similarity transformation model parameters are verified. Figure 5 shown.

[0107] Finally, the spatial position consistency of the 3D real scene model before and after registration is verified. The results are as follows: Figure 6 shown.

[0108] (4) Experimental results

[0109] from Figure 2 It can be seen that the highest resolution real-life 3D model is composed of vertices (solid points), triangulated networks (solid lines connecting vertices), and texture maps. Figure 2 The texture coordinates corresponding to each vertex in Figure 3 After extracting SIFT features from the texture map, the SIFT feature point coordinates are converted to texture coordinates, as shown in the square in the figure. Figure 3After calculating the spatial coordinates of each SIFT feature point according to the present invention, it is mapped to the real-scene 3D model, as shown in FIG. Figure 4 As shown in the crosshairs. Figure 4 As can be seen from the figure, the spatial plane positions of the SIFT feature points are accurately mapped to the triangulated mesh of the real-world 3D model according to the affine transformation model parameters. At the same time, their elevation coordinates closely follow the surface of the real-world 3D model, indicating that their elevation interpolation is also accurate. This shows that the spatial position calculation of SIFT feature points using triangulated mesh plane constraints is accurate and reliable.

[0110] After similarity determination based on the SIFT feature points extracted by the present invention, a large number of homonymous points can be obtained. However, the homonymous points still contain some gross error points. After the gross error elimination based on the present invention, homonymous points with uniform distribution can be obtained, and the homonymous points contain spatial coordinate information. Figure 5 As shown in the figure, a total of 3560 groups of homonymous points were obtained from the two experimental data. However, since the two groups of data do not have a unified spatial reference, their residuals are obviously systematic. After the three-dimensional similarity transformation solved by the present invention, the residual distribution of homonymous points is as follows: Figure 6 Analysis Figure 6 The residual error of the homonymous points after registration stabilizes near zero and is no longer systematic. The overall mean square error of the homonymous points before and after registration improves from 3.521 meters to 0.095 meters. Experimental results demonstrate that this method can effectively register multi-phase real-world 3D models with high registration accuracy.

Claims

1. A method for automatic registration of multi-period real-scene 3D models, characterized in that: The following steps are involved: S1, extracting the vertices, textures, texture coordinates corresponding to the vertices, and triangulated network index data of the bottom layer of the real 3D model; S2, extract SIFT features of texture information; S3, calculate the SIFT feature space position based on the triangulated mesh plane constraint. The specific steps include: a1, convert the extracted SIFT feature point coordinates into texture coordinates; a2, search for the triangulated mesh contained in the current texture coordinates; a3, interpolate the spatial coordinates of the current SIFT feature point based on the vertex spatial coordinates and vertex texture coordinates of the triangulated network; The method of interpolating the spatial coordinates of the current SIFT feature point based on the vertex spatial coordinates and vertex texture coordinates of the triangulated network is an interpolation method; Interpolate the spatial coordinates of the current SIFT feature point in two steps; Interpolate the spatial coordinates of the current SIFT feature point in two steps: interpolate the plane coordinates of the SIFT feature point and interpolate the elevation coordinates of the SIFT feature point; S4, spatially constrained SIFT feature similarity determination; S5, eliminating the gross errors of homonymous points based on the constraints of the 3D similarity transformation model, and solving the 3D similarity transformation model parameters of the 3D real scene model to be registered relative to the reference 3D real scene model; S6, updating the spatial positions of all vertices of the 3D real scene model to be registered according to the 3D similarity transformation model parameters; S7, outputting the real scene 3D model to be registered after the vertices are updated.

2. The method for automatic registration of multi-period real-scene 3D models according to claim 1, characterized in that: In S4, the specific method for determining the similarity of spatially constrained SIFT features includes: b1, determine the spatial bounding box where the current SIFT feature point is located; b2, search for the SIFT feature vector of the reference image within the current bounding box to form a set of feature vectors to be matched; b3, calculate the similarity between the feature vector of the current SIFT feature point and the set of feature vectors to be matched, and determine the initial correspondence between the points with the same name.

3. The method for automatic registration of multi-period real-scene 3D models according to claim 2, characterized in that: The spatial bounding box where the current SIFT feature point is located is determined, and the spatial bounding box to be registered is determined based on the geometric positioning accuracy of the multi-phase real-scene three-dimensional model.

4. The method for automatic registration of multi-period real-scene 3D models according to claim 1, characterized in that: In S5, the specific steps of eliminating the gross errors of homonymous points based on the constraints of the three-dimensional similarity transformation model and calculating the three-dimensional similarity transformation model parameters of the three-dimensional real scene model to be registered relative to the reference three-dimensional real scene model include: c1, randomly extract 3 pairs of points with the same name from the initial matching image pair; c2, calculate the parameters of the 3D similarity transformation model based on three pairs of points of the same name; c3, evaluate the number of inliers based on the three-dimensional similarity transformation model parameters and record the current number of inliers; c4, repeat c1 to c3 until the upper limit of the cycle number; c5, the three-dimensional similarity transformation model parameters with the largest number of inliers are taken as the optimal three-dimensional similarity transformation model parameters.

5. The method for automatic registration of multi-period real-scene 3D models according to claim 4, characterized in that: The upper limit of the number of cycles is set to 100 times.

6. The method for automatic registration of multi-period real-scene 3D models according to claim 4, characterized in that: In S6, the specific steps of updating the spatial positions of all vertices of the 3D real scene model to be registered according to the 3D similarity transformation model parameters include: d1, extract the coordinates of all vertices of the 3D model to be registered; d2, performs coordinate transformation on the vertices extracted by d1 according to the optimal three-dimensional similarity transformation model parameters obtained by c5.

Images