TIN network neighboring image registration method based on sparse feature points under high similarity image background

Abstract

The present invention relates to a TIN network neighboring image registration method based on sparse feature points under a high-similarity image background. The method uses feature points extracted from an image to select a few significant feature points to construct a TIN network, and then completes the registration and superposition of neighboring images through congruent analysis and judgment of the TIN network. The method has a small amount of calculation, high accuracy, and fast registration speed, and can greatly improve the registration accuracy and efficiency of the image, and can speed up the aerial triangulation and modeling speed of the later three-dimensional modeling, and improve the efficiency and model quality of three-dimensional real scene modeling at the construction site of the construction project.

Description

Technical Field

[0001] The invention relates to the field of engineering information technology, in particular to a TIN network neighboring image registration method based on sparse feature points under a high-similarity image background. Background Art

[0002] In the process of 3D real-scene modeling of construction sites, an important task is to find and obtain feature points from a large number of construction site images collected by small drones, and to perform registration and splicing of adjacent images based on feature points to obtain an overall image of the construction site, providing technical support for the final formation of a 3D model of the construction site. There are currently a variety of registration algorithms, the main methods of which are SIFT algorithm, grayscale-based matching algorithm, and image feature-based matching algorithm:

[0003] (1) SIFT algorithm

[0004] The SIFT (Scale Invariant Feature Transform) algorithm is also called the scale-invariant feature transform algorithm. This algorithm obtains feature points from the original image and the target image, then uses the feature information to generate highly distinguishable feature descriptors, and then performs image registration by measuring the similarity between feature descriptors. From the use effect, the algorithm has scale invariance, rotation invariance and strong anti-noise ability, but the algorithm analysis process is relatively complex, the calculation takes a long time, the feature points are unevenly distributed, and the matching accuracy is not high.

[0005] (2) Gray value-based matching algorithm

[0006] The image matching algorithm based on gray value takes the gray value distribution state of the image as the matching object, and the image matching effect mainly depends on the gray value similarity measurement criterion and its search strategy. Classic similarity measurement includes the absolute value of gray difference and the absolute square sum of gray difference. From the perspective of use effect, the matching algorithm based on gray value is more intuitive, does not require additional calculation and extraction of image features, and can avoid parameter estimation errors introduced by feature extraction. However, due to the instability of the small UAV flight platform, and the gray value of the image is usually affected by the lighting conditions during image acquisition, it has a great impact on the gray value calculation and precision analysis, especially in the registration of color images, it is difficult to achieve ideal results.

[0007] (3) Matching algorithm based on image features

[0008] The matching algorithm based on image features is a method of image registration that extracts significant features such as points, lines, edges, contours, colors, and textures from the image to be registered, then uses similarity metrics and some constraints to determine the geometric transformation, and finally applies the transformation to the image to be matched. Among the extracted features, points, lines, edges, contours and other features are used relatively more. From the application effect, the algorithm is sensitive to the relative position changes of image features, and image matching is closely related to the selected image features. Therefore, the image registration effect obtained by the image feature matching algorithm is highly unstable and the image quality fluctuates greatly.

[0009] Based on the above registration algorithm, the current registration process of many modeling software is to conduct a comprehensive analysis and comparison of all feature points after completing the extraction of image feature points. However, this process is computationally intensive, time-consuming, and inefficient. In particular, since feature points have significant discrete features in image space, and feature points of adjacent images have weak distinguishability in terms of color, brightness, texture, etc., they interfere with each other more, but have less constraints on each other. Therefore, the current image registration method has low operating efficiency, and the image registration effect and quality obtained are relatively poor. Summary of the invention

[0010] In view of the defects existing in the prior art, the purpose of the present invention is to provide a TIN network neighboring image registration method based on sparse feature points in a high-similarity image background. The method uses the feature points extracted from the neighboring images to select a few significant feature points to build a TIN network, and then completes the registration and superposition of the neighboring images through congruence analysis and judgment of the TIN network. Compared with the existing methods, this method can greatly improve the registration accuracy and efficiency of the images.

[0011] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0012] 1. A TIN network neighboring image registration method based on sparse feature points under a high-similarity image background, characterized by comprising the following steps:

[0013] Step 1: Select the neighboring image I with high similarity to the background that needs to be registered in the aerial image. h and I l ;

[0014] Step 2: Extract I in the overlapping area of ​​adjacent images h and I l Uniformly distributed feature points p in the image ij and p nm , and establish sparse feature point sets T(p ij (x,y)) and T′(p nm (x′,y′)):

[0015]

[0016] Where (i, j) and (n, m) represent the feature points in image I h and I l The row and column numbers in the , x and x′ represent I h Image and I l BGR value of sparse feature points in the image (c b ,c g ,c r ), y and y′ represent I h Image and I l LBP value of sparse feature points in the image

[0017] Step 3, carry out I h and I l Sparse feature point mapping, specifically, T(p ij (x,y)) and T′(p nm The corresponding relationship F between (x′, y′) is expressed as:

[0018] F:T(p ij (x,y))→T′(p nm (x′,y′))

[0019] F:T(p ij (BRG),p ij (LPB))→T′(p nm (BRG),p nm (LPB)

[0020]

[0021] Step 4: construct a triangle based on sparse feature points. Specifically, in T(p ij (x,y)) and T′(p nm (x′, y′)), take three feature point groups (p1, p2, p3) and (p′1, p′2, p′3), each of which contains three sparse feature points:

[0022] T(p1,p2,p3)=p1((x 11 ,y 11 ),(x 12 ,y 12 ),(x 13 ,y 13 ))Up2((x 21 ,y 21 ),(x 22 ,y 22),(x 23 ,y 23 ))Up3((x 31 ,y 31 ),(x 32 ,y 32 ),(x 33 ,y 33 ))T′(p′1,p′2,p′3)=p′1((x1′1,y1′1),(x1′2,y1′2),(x1′3,y1′3))Up′2((x2′1,y′ 21 ),(x′ 22 ,y2′2),(x2′3,y2′3))Up′3((x3′1,y3′1),(x3′2,y3′2),(x3′3,y3′3))

[0023] In the above formula, x 11 、x 12 、x 13 、x 21 、x 22 、x 23 、x 31 、x 32 、x 33 are the BGR values ​​of the sparse feature points in the feature point group (p1, p2, p3); 11 ,y 12 ,y 13 ,y 21 ,y 22 ,y 23 ,y 31 ,y 32 ,y 33 are the LBP values ​​of each sparse feature point in the feature point group (p1, p2, p3); x' 11 、x' 12 、x' 13 、x' 21 、x' 22 、x' 23 、x' 31 、x' 32 、x' 33 They are feature point groups (p′1, p′2, p′3)

[0024] BGR value of each sparse feature point in; y' 11 ,y' 12 ,y' 13 ,y' 21 ,y' 22 ,y' 23 ,y' 31 ,y' 32 ,y' 33are the LBP values ​​of each sparse feature point in the feature point group (p′1, p′2, p′3);

[0025] Connect the three sparse feature points of each feature point group into a triangle, that is, in I h and I l We get a set of three triangles in each;

[0026] Step 5, analyze the I obtained in step 4 h The three triangles and I l The congruence of the three corresponding triangles in ;

[0027] Step 6: Construct a TIN network, analyze and determine the congruence of the TIN network, and then determine the neighboring image I h and I l Can a complete match be achieved: If the TIN networks are congruent, then the neighboring images I h and I l An exact match can be achieved.

[0028] On the basis of the above scheme,

[0029] The criteria for determining triangle congruence described in step 5 are: the following must be satisfied:

[0030]

[0031] In the above formulas, (a, b, c) and (a′, b′, c′) are respectively h and I l The three corner points of the two corresponding triangles; x1, x2, x3 are the BGR values ​​of the corner points (a, b, c) respectively; y1, y2, y3 are the LBP values ​​of the corner points (a, b, c) respectively; x'1, x'2, x'3 are the BGR values ​​of the corner points (a′, b′, c′) respectively; y'1, y'2, y'3 are the LBP values ​​of the corner points (a′, b′, c′) respectively.

[0032] On the basis of the above scheme,

[0033] The specific method of constructing the TIN network and analyzing and determining the congruence of the TIN network described in step 6 is:

[0034] In determining I h The three triangles in I l If the three triangles in the corresponding positions are congruent, h and I lIn each triangle in , select the corner point closest to the other two triangles, and form a new triangle p4 and p′4 with the three selected corner points. If p4 and p′4 also meet the congruence conditions, then the TIN network composed of (p1, p2, p3, p4) and (p′1, p′2, p′3, p′4) also has congruence. Whether the above p4 and p′4 are congruent is also determined according to the triangle congruence determination criteria mentioned above.

[0035] The TIN network neighboring image registration method based on sparse feature points under high-similarity image background of the present invention has the following beneficial effects:

[0036] Compared with the current construction site image registration method, this method uses the feature points extracted from the image, selects a few significant feature points from them to build a TIN network, and then completes the registration and superposition of adjacent images through congruent analysis and judgment of the TIN network. This method has small calculation amount, high accuracy, and fast registration speed, which can greatly improve the image registration accuracy and efficiency, and can speed up the aerial triangulation and modeling speed in the later 3D modeling, and improve the efficiency and model quality of 3D real scene modeling at the construction site of construction projects.

[0037] This method is used to write a neighboring image registration program, which is then implanted into the current 3D modeling software for engineering construction sites. The TIN network can be constructed using the sparse feature points selected in the neighboring images, and then the image registration can be completed through the congruent mapping of the TIN network. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The present invention has the following accompanying drawings:

[0039] Figure 1 is a triangle composed of sparse feature points in the high-resolution background described in the embodiment of the present invention;

[0040] Figure 2 This is a congruence analysis of the TIN network described in the embodiment of the present invention;

[0041] Figure 3 It is a system diagram of the role played by the present invention in three-dimensional real-scene modeling of engineering construction sites and related work. DETAILED DESCRIPTION

[0042] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0043] In the construction project construction site 3D real scene modeling, after using a small drone to obtain the construction site pictures, these pictures can be input into the 3D modeling software, and the image feature point extraction can be completed using the existing many image feature point extraction methods. On this basis, it is necessary to perform BGR (Blue Green Red) + LBP (Local Binary Pattern) dual parameter description and feature point enhancement work on the extracted feature points. After completing these tasks, the image registration work can be carried out according to the following six steps proposed by the present invention.

[0044] (1) Step 1: Select the high-similarity background neighboring image to be registered

[0045] The construction site images taken by small drones are all small images, but the adjacent images have some overlapping images and have a high similarity, so they belong to high-similarity background images. Before registering two adjacent images, it is necessary to select adjacent images with high-similarity background and define them as I h and I l .

[0046] (2) Step 2: Construct a sparse feature point set

[0047] After the adjacent images that need to be registered are identified, the image I is extracted using the modeling software. h and I l Then from the neighboring image I h and I l Select a small number of feature points p from the many feature points ij and p nm The principle of selection is that the feature points are evenly distributed in the overlapping areas of adjacent images, and these selected points are used to establish a sparse feature point set. Among them, (i, j) and (n, m) represent the feature points in image I, respectively. h and I l The row and column numbers in .

[0048] Let I h Image and I l The sparse feature point sets of the image are T(p ij (x,y)) and T′(p nm (x′,y′)). Since each feature point p in the image ij The parameters are described by the BGR and LBP double parameters, x and x' represent I h Image and I l BGR value of sparse feature points in the image (c b ,c g ,c r ), y and y′ represent I hImage and I l LBP value of sparse feature points in the image Then we have:

[0049]

[0050] (3) Step 3: Conduct I h and I l Sparse feature point mapping

[0051] If T(p ij (x,y)) and T′(p nm (x′, y′)) has a corresponding relationship F, which can be expressed as:

[0052] F:T(p ij (x,y))→T′(p nm (x′,y′))

[0053] F:T(p ij (BRG),p ij (LPB))→T′(p nm (BRG),p nm (LPB)

[0054]

[0055] If Python is used to complete this task, the program code is:

[0056] def is_one_to_one_mapping(set_T,set_T′):

[0057] iflen(set_T)! =len(set_T′):

[0058] return False

[0059] dict_map={x:x′for x,x′in zip(set_T,set_T′)}

[0060] dict_map={y:y′for y,y′in zip(set_T,set_T′)}

[0061] return all(dict_map[x]==x′for x,x′in zip(set_T,set_T′))

[0062] return all(dict_map[y]==y′for y,y′in zip(set_T,set_T′))

[0063] (4) Step 4: Constructing a triangle based on sparse feature points

[0064] In T(p ij (x,y)) and T′(p nm (x′, y′)), take three feature point groups (p1, p2, p3) and (p′1, p′2, p′3), each of which contains three feature points, that is: T(p1, p2, p3) = p1((x′, y′) 11 ,y 11 ),(x 12 ,y 12 ),(x 13 ,y 13 ))Up2((x 21 ,y 21 ),(x 22 ,y 22 ),(x 23 ,y 23 ))Up3((x 31 ,y 31 ),(x 32 ,y 32 ),(x 33 ,y 33 ))T′(p′1,p′2,p′3)=p′1((x1′1,y1′1),(x1′2,y1′2),(x1′3,y1′3))Up′2((x2′1,y2′1),(x2′2,y2′2),(x′ 23 ,y′ 23 ))Up′3((x3′1,y3′1),(x3′2,y3′2),(x3′3,y3′3))

[0065] Then connect the three points in each group into a triangle, so that in image I h and I l The high-resembling-background area has three triangles each. Take the triangles p1(a1,b1,c1) and p′1(a1′,b1′,c1′) formed by p1 and p′1 as examples. (a1,b1,c1) and (a1′,b1′,c1′) are the three corner points of the triangles, such as Figure 1 shown.

[0066] (5) Step 5: Analyze the congruence of triangles composed of sparse feature points

[0067] If the triangle formed by p1 and p′1 satisfies the following requirements, it is proved that the two triangles are congruent.

[0068]

[0069] Similarly, the congruence verification and alignment of two pairs of triangles consisting of another two sets of feature points (p2, p′2) and (p3, p′3) can be completed.

[0070] (6) Step 6: Construct TIN network and analyze and determine the congruence of TIN network

[0071] Under the premise of ensuring that the three triangles (p1, p2, p3) and (p′1, p′2, p′3) are respectively congruent, select the corner point closest to the other two triangles from each triangle, and form a new triangle p4 and p′4 with the selected three corner points. If p4 and p′4 also meet the congruence conditions (refer to the congruence judgment conditions described in step 5), then the TIN network composed of (p1, p2, p3, p4) and (p′1, p′2, p′3, p′4) is also congruent. At this time, the neighboring image I h and I l A complete match can be achieved, such as Figure 2 shown.

[0072] The method of constructing a TIN network and analyzing and determining the congruence of the TIN network is:

[0073]

[0074]

[0075] Select three points from Dis(p1,p2), Dis(p1,p3) and Dis(p3,p2) to form a new triangle. Let the three corner points of the new triangle be (a4,b4,c4) and can be obtained by the following search algorithm:

[0076]

[0077] Similarly, we can complete the analysis of Dis(p′1,p′2), Dis(p′1,p′3) and Dis(p′3,p′2) and obtain (a4′,b4′,c4′). If (a4,b4,c4) and (a4′,b4′,c4′) still have the following relationship, then image I h and I l Accurate matching can be achieved.

[0078]

[0079] at last, Figure 3 This is a system diagram to describe the role of the present invention in three-dimensional real scene modeling of engineering construction sites and the related work. The content of the present invention is the blue part of the figure, which describes the TIN network neighboring image registration method based on sparse feature points under a high-similarity image background, and the rest of the part expresses the role of the present invention in three-dimensional modeling and its relationship with other work.

[0080] The contents not described in detail in this specification belong to the prior art and professional common sense known to professional and technical personnel in this field.

Claims

1. A TIN network neighboring image registration method based on sparse feature points under high-similarity image background, characterized in that: The steps include: Step 1: Select the neighboring image I with high similarity to the background that needs to be registered in the aerial image. h and I l ; Step 2: Extract I h and I l Uniformly distributed feature points p in the image ij and p nm , and establish sparse feature point sets T(p ij (x,y)) and T′(p nm (x′,y′)): Where (i, j) and (n, m) represent the feature points in image I h and I l The row and column numbers in the , x and x′ represent I h Image and I l BGR value of sparse feature points in the image (c b ,c g ,c r ), y and y′ represent I h Image and I l LBP value of sparse feature points in the image Step 3, carry out I h and I l Sparse feature point mapping, specifically, T(p ij (x,y)) and T′(p nm The corresponding relationship F between (x′, y′) is expressed as: F:T(p ij (x,y))→T′(p nm (x′,y′)) F:T(p ij (BRG),p ij (LPB))→T′(p nm (BRG),p nm (LPB)) F: Step 4: construct a triangle based on sparse feature points. Specifically, in T(p ij (x,y)) and T′(p nm (x′, y′)), take three feature point groups (p1, p2, p3) and (p′1, p′2, p′3), each of which contains three sparse feature points: T(p1,p2,p3) =p1((x 11 ,y 11 ),(x 12 ,y 12 ),(x 13 ,y 13 ))Up2((x 21 ,y 21 ),(x 22 ,y 22 ),(x 23 ,y 23 ))Up3((x 31 ,y 31 ),(x 32 ,y 32 ),(x 33 ,y 33 ))T′(p′1,p′2,p′3) =p′1((x′ 11 ,y′ 11 ),(x′ 12 ,y′1′),(x′ 13 ,y′ 13 ))Up′2((x′ 21 ,y′ 21 ),(x′ 22 ,y′ 22 ),(x′ 23 ,y′ 23 ))Up′3((x′ 31 ,y′ 31 ),(x′ 32 ,y′ 32 ),(x′ 33 ,y′ 33 )) In the above formula, x 11 、x 12 、x 13 、x 21 、x 22 、x 23 、x 31 、x 32 、x 33 are the BGR values ​​of the sparse feature points in the feature point group (p1, p2, p3); 11 ,y 12 ,y 13 ,y 21 ,y 22 ,y 23 ,y 31 ,y 32 ,y 33 are the LBP values ​​of each sparse feature point in the feature point group (p1, p2, p3); x' 11 、x' 12 、x' 13 、x' 21 、x' 22 、x' 23 、x' 31 、x' 32 、x' 33 are the BGR values ​​of the sparse feature points in the feature point group (p′1, p′2, p′3); y' 11 ,y' 12 ,y' 13 ,y' 21 ,y' 22 ,y' 23 ,y' 31 ,y' 32 ,y' 33 are the LBP values ​​of each sparse feature point in the feature point group (p′1, p′2, p′3); Connect the three sparse feature points of each feature point group into a triangle, that is, in I h and I l We get a set of three triangles in each; Step 5, analyze the I obtained in step 4 h The three triangles and I l The congruence of the three corresponding triangles in ; Step 6: Construct a TIN network, analyze and determine the congruence of the TIN network, and then determine the neighboring image I h and I l Can a complete match be achieved? 2. The TIN network neighboring image registration method based on sparse feature points under a high-similarity image background as claimed in claim 1, characterized in that: The criteria for determining triangle congruence described in step 5 are: the following must be satisfied: In the above formulas, (a, b, c) and (a′, b′, c′) are I h and I l The three corner points of the two corresponding triangles in the image; x1, x2, x3 are the BGR values ​​of the corner points (a, b, c); y1, y2, y3 are the LBP values ​​of the corner points (a, b, c); x'1, x'2, x'3 are the BGR values ​​of the corner points (a′, b′, c′); y'1, y'2, y'3 are the LBP values ​​of the corner points (a′, b′, c′).

3. The TIN network neighboring image registration method based on sparse feature points under a high-similarity image background as claimed in claim 1, characterized in that: The specific method for constructing the TIN network and analyzing and determining the TIN network congruence described in step 6 is: In determining I h The three triangles in I l If the three triangles in the corresponding positions are congruent, h and I l From each triangle in , select the corner point closest to the other two triangles, and combine the three selected corner points to form a new triangle p4 and p′4. If p4 and p′4 also meet the congruence conditions, then the TIN network composed of (p1, p2, p3, p4) and (p′1, p′2, p′3, p′4) is also congruent.

Images