Rough matching method of heterogeneous point clouds based on geometry and texture mapping

Abstract

The present invention discloses a heterogeneous point cloud rough matching method based on geometry and texture mapping, comprising the following steps: S1: generating a point cloud mapping image based on a source point cloud and a model point cloud; S2: extracting feature points using an RGB channel mapping image; S3: generating a feature descriptor that fuses point cloud geometry and RGB attributes based on a multi-channel image of point cloud RGB, normal vector Dip component, roughness and curvature attribute mapping; S4: matching feature points of point cloud mapping images; S5: obtaining corresponding three-dimensional matching points based on a 3D-2D mapping relationship to complete a rough matching of point clouds. The heterogeneous point cloud rough matching method in the present invention can overcome the problem that the existing point cloud matching method based on mapping images cannot complete point cloud matching when the texture similarity of heterogeneous point cloud mapping images is low by fusing feature descriptors of geometry and texture.

Description

Technical Field

[0001] The invention relates to the technical field of three-dimensional point cloud processing, and in particular to a heterogeneous point cloud rough matching method based on geometry and texture mapping. Background Art

[0002] As an important data source of geographic information, three-dimensional point clouds are widely used in fine modeling and spatial analysis, and play a very important role in cadastral survey, environmental protection, urban planning, emergency rescue and other fields. Since each type of point cloud has its own advantages and disadvantages, and the acquisition angle of a single point cloud is limited, it is easy to produce point cloud missing caused by ground objects, so the fusion of heterogeneous point clouds can better serve the subsequent fine modeling and analysis. The basis of point cloud fusion is point cloud matching, and the quality of matching is directly related to the application effect of the fused point cloud. The matching of heterogeneous point clouds has always been a research difficulty in related fields and a bottleneck problem affecting the fusion of heterogeneous point clouds.

[0003] The matching of three-dimensional point clouds is generally based on the local or overall features of three-dimensional point clouds. However, the matching directly based on three-dimensional points has high computational overhead, high memory usage, and low computational efficiency. Therefore, many scholars use mature image matching algorithms to complete the registration of the images corresponding to the point clouds, and finally complete the matching between point clouds based on the mapping relationship between the image and the point cloud. The image used in this method can be a mapping image of point cloud intensity, depth, RGB texture and other attributes, or an image of the image point corresponding to the spatial point in the forward intersection of photogrammetry. The point cloud matching algorithm based on the mapping image has high matching efficiency and can handle point cloud matching with similar transformation relationships. However, this method requires that the textures between point clouds are highly similar. Due to the different acquisition methods, acquisition angles, and acquisition times of heterogeneous point clouds, the resolution, angle / scale, and texture differences of the point cloud mapping images will inevitably be brought about, which reduces the similarity between the mapping images, thereby reducing the matching effect of such algorithms in heterogeneous point clouds. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention aims to provide a rough matching method of heterogeneous point clouds based on geometry and texture mapping, which overcomes the problem that the existing point cloud matching method based on mapping images cannot complete point cloud matching when the texture similarity of the heterogeneous point cloud mapping images is low by fusing the feature descriptors of geometry and texture.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] The heterogeneous point cloud rough matching method based on geometry and texture mapping is characterized by comprising the following steps:

[0007] S1: Generate point cloud mapping image based on source point cloud and model point cloud;

[0008] S2: Use RGB channel mapping image to extract feature points;

[0009] S3: Generate a feature descriptor that integrates point cloud geometry and RGB attributes based on multi-channel images of point cloud RGB, normal vector Dip component, roughness and curvature attribute mapping;

[0010] S4: point cloud mapping image feature point matching;

[0011] S5: Obtain corresponding 3D matching points according to the 3D-2D mapping relationship to complete the rough matching of the point cloud.

[0012] Furthermore, the specific operation of step S1 includes the following steps:

[0013] S101: Establish a directed bounding box of the point cloud;

[0014] S102: performing cube voxel segmentation on the directed bounding box;

[0015] S103: Perform point cloud mapping based on voxels to obtain a multi-channel image of the point cloud.

[0016] Furthermore, the specific operation of step S2 includes the following steps:

[0017] S201: obtaining a rough overlapping area of ​​images based on GPS assistance;

[0018] S202: extracting images of the texture area of ​​interest;

[0019] S203: generating a gradient enhanced image of the texture region of interest;

[0020] S204: Establish Harris and SIFT feature point sets based on the scale space.

[0021] Furthermore, in step S202, the Grab Cut algorithm and the image mask Mask are used to extract the texture region of interest.

[0022] Furthermore, the specific operation of step S203 includes the following steps:

[0023] S2031: Use the canny operator to obtain the gradient image of the texture area of ​​interest;

[0024] S2032: superimposing the gradient image with the original image to obtain a gradient enhanced image;

[0025] Assume that the pixel value of the gradient image is {H i}, the pixel value of the original image is {J i}, the pixel value of the gradient enhanced image is {P i},but In the formula, H max With H min are pixels {H i}; max{Pix i} and min{Pix i} respectively {Pix i} maximum and minimum values ​​in .

[0026] Furthermore, the specific operation of step S204 includes the following steps:

[0027] S2041: Use Gaussian convolution kernel G(x, y, σ i ) Establish a Gaussian scale space and calculate the M matrix at different scales. Among them, A(σ i ) = G(x, y, σ i )*(I x ) 2 ,B(σ i ) = G(x, y, σ i )*(I y ) 2 ,C(σ i ) = G(x, y, σ i (I x I y ) 2 , where I x with I y Represent the first-order partial derivatives of the image in the x and y axis directions, respectively, and * represents the convolution operation; on this basis, the corresponding R(σ i ), R(σ i )=det(M(σ i ))-τ·tr 2 (M(σ i )), where τ∈[0.04-0.06] is the weight coefficient;

[0028] S2042: At each scale, use non-maximum suppression to obtain the corresponding R(σ i ), when R(σ i ) is greater than the threshold and is the maximum value within a given neighborhood, the corresponding point is taken as a candidate Harris corner point;

[0029] S2043: filtering candidate Harris corner points along the scale direction from small to large in the scale space. If a candidate Harris corner point exists continuously in the scale space, this corner point is a Harris feature point. Accumulating Harris feature points in all scale spaces as the final Harris feature point set;

[0030] S2044: Extract SIFT feature points from the gradient enhanced image in the same scale space.

[0031] Furthermore, the specific operation of step S3 includes the following steps:

[0032] S301: using the same Gaussian convolution kernel function as that used in feature point extraction in step S2 to establish a scale space of a point cloud mapping multi-channel image, wherein the multi-channel image includes an RGB channel, a Dip channel, a curvature channel, and a roughness channel;

[0033] S302: Counting the gradient distribution of the neighborhood of the RGB channel feature points of the gradient enhanced image of the texture region of interest, establishing the reference direction of the feature points, and determining the area of ​​each channel image;

[0034] S303: Rotate the coordinate axis of each channel image area determined in step S302 according to the reference direction of the feature point. The new coordinates of the sampling points in the neighborhood after rotation are

[0035] S304: After the feature point neighborhood is rotated, the gradients of each pixel in the sub-region of the neighborhood are distributed to 8 statistical directions, and the weights are calculated to perform histogram statistics;

[0036] S305: Calculate a 128-dimensional feature descriptor that fuses point cloud geometry and RGB attributes.

[0037] Furthermore, the specific operation of step S302 includes the following steps:

[0038] S3021: Search for the gradient magnitude and direction of pixels within a 3σ neighborhood in the scale image corresponding to the feature point, where 3σ = 3×1.5×σ oct , where σ oct The Gaussian parameters used to generate adjacent scale images in each scale space;

[0039]

[0040] Where L is the scale space where the feature point is located; m(x, y) and θ(x, y) are the magnitude and direction of the gradient respectively;

[0041] S3022: Using a histogram to count pixel gradient distribution in the neighborhood, using the gradient direction of the pixel to determine the histogram interval, multiplying the amplitude by the weight and superimposing it to the corresponding columnar interval to complete the final statistics;

[0042] S3023: In the scale space corresponding to the feature point, determine the image regions of each channel required by the descriptor; divide the neighborhood near the feature point into 2×2 sub-regions, each sub-region is used as a seed point, and the gradient features in 8 directions are counted. Each sub-region contains 12×σ oct sub-pixels, that is, the side length of the sub-region is Finally, the area radius corresponding to each channel image is

[0043] Furthermore, the specific operation of step S304 includes the following steps:

[0044] S3041: Determine the subscript of the sub-region where the pixel point is located after rotation

[0045] S3042: The gradient of the pixels in the area is weighted according to the Gaussian kernel G' with σ=1 and radius 3σ, and the weighted gradient is ω=M(a+x,b+y)*G', where (x,y)∈3σ and (a,b) is the position of the feature point in the scale image.

[0046] Furthermore, the specific operation of step S305 includes the following steps:

[0047] S3051: According to the region subscript (x", y") of each pixel point, the 8-directional gradient of the seed point is interpolated and calculated. Each channel image will generate 2×2×8, i.e., 32 gradient information as the feature vector corresponding to the channel;

[0048] S3052: The 32-dimensional gradient information obtained from each channel is normalized and threshold-truncated, and the 32-dimensional descriptors of the four channels are connected in series to finally obtain a 2×2×8×4, i.e., 128-dimensional descriptor. In the algorithm, the feature weights between the four channels are all 1.

[0049] The beneficial effects of the present invention are:

[0050] The heterogeneous point cloud rough matching method in the present invention aims to solve the problem that the existing point cloud matching algorithm based on mapping images cannot complete point cloud matching when the texture correlation between mapping images is low, and proposes a point cloud matching algorithm based on geometry and texture mapping images (Geometry and Texture Image-based Point Cloud Registration, GTIR). The algorithm first uses GPS auxiliary information and graph cut (Grab Cut) to extract the texture area of ​​interest of the point cloud mapping image, and removes the mismatch caused by non-overlapping areas and feature calculations of monotonous texture areas. Thereafter, in order to solve the problem that the texture correlation between images obtained by mapping only the point cloud RGB attributes is low and the existing algorithm cannot match, a feature descriptor that integrates the point cloud geometry and RGB attributes is designed based on multi-channel images mapped with point cloud RGB texture attributes, normal vector slope component (Dip), roughness and curvature attributes to improve the correlation between feature points with the same name. In order to reduce the computational overhead of feature point extraction of multi-channel images in scale space, GTIR only uses RGB channel images for feature point extraction to ensure the overall efficiency of the algorithm. In order to reduce the impact of low correlation between RGB textures on feature point extraction, the algorithm uses multi-scale Harris and SIFT (Scale-invariant Feature transform) as feature point sets based on gradient enhancement of the RGB image in the texture area of ​​interest, thereby ensuring the number and quality of feature point extraction and improving the stability of the subsequent matching algorithm. After mapping image feature point extraction and matching, the three-dimensional matching points corresponding to the point cloud are obtained based on the retained 3D-2D mapping relationship to complete the matching between heterogeneous point clouds. The method of the present invention can effectively handle the coarse matching of heterogeneous point clouds. The algorithm not only has the general advantages of point cloud matching algorithms based on mapping images, but also has small computational complexity, high algorithm efficiency, and high heterogeneous point cloud matching accuracy. It can also overcome mismatches caused by differences in resolution, lighting environment, and acquisition angle. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flow chart of the heterogeneous point cloud rough matching method in the present invention.

[0052] Figure 2 It is the AABB bounding box and OBB bounding box of the point cloud in the present invention.

[0053] Figure 3 It is the voxelized segmentation map of the point cloud in the present invention.

[0054] Figure 4 It is a 6-channel mapping image of the close-range photogrammetry point cloud in the present invention.

[0055] Figure 5 It is a 6-channel mapping image of the drone photogrammetry point cloud in the present invention.

[0056] Figure 6 Schematic diagram of the rough overlapping area between point clouds in the present invention.

[0057] Figure 7 This is a flowchart for extracting the texture region of interest in the present invention.

[0058] Figure 8 This is the gradient enhancement effect of the texture region of interest in the present invention.

[0059] Fig. 9 This is the Harris feature point detection in the mesoscale space of the present invention.

[0060] Fig.10 It is the multi-scale Harris and SIFT feature point set in the present invention.

[0061] Fig.11 It is a three-channel image obtained by mapping the point cloud geometric attributes except the RGB channel in the present invention.

[0062] Fig.12 It is a statistical graph of the gradient histogram of the image in the present invention.

[0063] Fig.13 It is the coordinate axis rotation of the feature point neighborhood in the present invention.

[0064] Fig.14 It is a feature descriptor that integrates geometric and RGB texture attributes in the present invention.

[0065] Fig.15 This is the correspondence and matching effect of the final feature points in the present invention.

[0066] Fig.16 This is the final matching effect of the image obtained by point cloud mapping in the present invention.

[0067] Fig.17 This is a distribution diagram of feature points with the same name in the close-range photogrammetry point cloud in the present invention.

[0068] Fig.18 This is a distribution diagram of the feature points with the same name in the UAV photogrammetry point cloud of the present invention.

[0069] Fig.19 This is a diagram showing the final matching effect of the coarse matching method of the present invention on the heterogeneous point cloud. DETAILED DESCRIPTION

[0070] In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0071] As attached Figure 1As shown, the heterogeneous point cloud rough matching method based on geometry and texture mapping includes the following steps:

[0072] S1: Generate a point cloud mapping image based on the source point cloud and the model point cloud (in the field of point cloud matching, point cloud A is used as the coordinate reference, and a matching algorithm is used to complete the matching of B to A (moving the coordinates of B to A). A is called the source point cloud and B is called the model point cloud);

[0073] Specifically, S101: establishing a directed bounding box of the point cloud;

[0074] Since point clouds are not necessarily distributed along the coordinate axis, in order to reduce the amount of calculation and storage space when mapping point clouds, it is necessary to establish an oriented bounding box (OBB) of the point cloud. The eigenvector is obtained by performing PCA (Principal Component Analysis) decomposition on the covariance matrix of the point cloud, and this is used as the principal axis of the OBB. The outer volume of the point cloud is established based on the principal axis of the OBB. The calculation process is as follows:

[0075] 1) Establish the covariance matrix A of the point cloud,

[0076]

[0077] The main diagonal elements corresponding to cov() represent the variance on the left, and the other elements represent the covariance between the coordinates.

[0078] 2) Perform Jacobi iteration and Schmidt orthogonalization on the covariance matrix to solve its eigenvalue and corresponding eigenvector. The larger one corresponds to the coordinate axis of the OBB bounding box.

[0079] 3) In the new coordinate system, the distribution of the point cloud can be statistically analyzed to obtain the length, width, height and center point of the OBB.

[0080] The OBB bounding box is calculated using close-range photogrammetry point cloud and drone photogrammetry point cloud data. Figure 2 To convert the AABB bounding box based on the point cloud coordinate distribution into an OBB bounding box, Figure 2 In the figure, (a) and (c) are the AABB bounding boxes of the close-range photogrammetry point cloud and the UAV photogrammetry point cloud, respectively, and (b) and (d) are the OBB bounding boxes of the close-range photogrammetry point cloud and the UAV photogrammetry point cloud, respectively.

[0081] S102: performing cube voxel segmentation on the directed bounding box;

[0082] The size of voxels is directly related to the quality of subsequent images. If the voxels are too large, the resolution of the mapped image will be reduced, affecting the subsequent feature point extraction and matching. If the voxels are too small, the amount of mapped image data will be large. In particular, when the side length of the voxels is smaller than the point spacing, there will be a certain amount of null pixels in the image, which will greatly increase the amount of calculation. After experimental comparison, the voxel side length is finally set to 3 times the average spacing of the point cloud, and the area without data is not segmented. The final voxel segmentation effect is shown in the attached figure. Figure 3 As shown in the attached Figure 3 In the figure, (a) and (c) are the OBB bounding boxes of the close-range photogrammetry point cloud and the UAV photogrammetry point cloud, respectively, and (b) and (d) are the point cloud voxel segmentation maps based on the OBB bounding boxes of the close-range photogrammetry point cloud and the UAV photogrammetry point cloud, respectively.

[0083] S103: Perform point cloud mapping based on voxels to obtain a multi-channel image of the point cloud.

[0084] Voxels correspond to pixels of the mapped image. The attribute mean of the points contained in the voxel is used as the grayscale value of the corresponding pixel, and the point closest to the center of the voxel is used as the corresponding point of the pixel, so as to preserve the 3D-2D mapping relationship. The RGB, relative elevation, normal vector slope (Dip) and slope direction (Dip direction) components, Gaussian curvature and roughness attributes of the point cloud are mapped separately, as shown in the attached figure. Figure 4 With attached Figure 5 They are the multi-channel images after mapping the close-range photogrammetry point cloud and the UAV photogrammetry point cloud. Figure 4 6-channel mapping image of the close-range photogrammetry point cloud, attached Figure 5 The 6-channel mapping image of the UAV photogrammetry point cloud is attached. Figure 4 and attached Figure 5 In the figure, (a)-(f) are the RGB texture channels of the point cloud voxel mapping; the relative elevation channel displayed in 0-255; the normal vector Dip component and Dip direction component channels displayed in 0-255; the Gaussian curvature channel displayed in 0-255; and the roughness channel displayed in 0-255.

[0085] Finally, through the above processing of the algorithm, a multi-channel image of heterogeneous point cloud mapping is obtained, and the channel image used contains various geometric and texture features of the point cloud.

[0086] Further, step S2: extracting feature points using RGB channel mapping images;

[0087] Since the actual coverage area of ​​the image mapped by the unmanned photogrammetry point cloud is much larger than the coverage area of ​​the image mapped by the close-range photogrammetry point cloud, the images in the non-overlapping area will bring a large number of unmatched images. Therefore, it is necessary to extract the approximate overlapping area of ​​the heterogeneous point cloud mapping, that is, the texture area of ​​interest.

[0088] Specifically, S201: obtaining a rough overlapping area of ​​images based on GPS assistance;

[0089] The UAV photogrammetry point cloud has geospatial information and a horizontal positioning accuracy of 12cm. However, the GPS accuracy of the close-range image is low, with a horizontal positioning accuracy of about 2m, and cannot be used as control data for close-range photogrammetry. Therefore, the close-range photogrammetry point cloud has no geospatial scale. However, the GPS information of the close-range image can be used to roughly select the overlapping area of ​​the two mapped images. Figure 6 As shown, the box area is the image overlap area roughly selected based on GPS; although it is only a rough overlap area, this rough overlap area can effectively remove the mismatch caused by vegetation on both sides of the road and the matching calculation of most flat road surfaces and other texture monotonous areas.

[0090] S202: extracting an image of a texture area of ​​interest;

[0091] The texture of the flat road in the rough overlapping area is single and cannot effectively extract features, while the texture of the pothole area is richer and is the texture area of ​​interest for feature extraction. In order to further refine the rough overlapping area, Grab Cut and image mask Mask are used to extract the texture area of ​​interest. Based on the RGB image as data input, Grab Cut is first used to obtain the outline of the texture area of ​​interest, and a binary mask Mask is constructed based on the outline to extract the texture area of ​​interest. The extraction process is shown in the attached figure. Figure 7 As shown in the attached Figure 7 In the figure, (a) and (d) are images of roughly overlapping areas; (b) and (e) are foreground contours extracted by Grab Cut; (c) and (f) are images of texture areas of interest extracted using Mask; it should be noted that Grab Cut is a classic algorithm in the field of computer vision, and the specific algorithm content will not be repeated in this application.

[0092] S203: generating a gradient enhanced image of the texture region of interest;

[0093] More specifically, S2031: using a canny operator to obtain a gradient image of a texture region of interest;

[0094] After the texture region of interest is extracted, the influence of non-overlapping areas and weak feature areas on matching can be excluded, but due to differences in acquisition equipment and lighting environment, the texture similarity between images is still poor. In particular, the texture difference caused by the resolution between images directly leads to the failure of subsequent matching. In order to improve the texture similarity between images of the texture region of interest, the present invention pre-processes the images of the texture region of interest, uses an edge detection algorithm to obtain the corresponding gradient image, and enhances the gradient image. Although the texture difference of the images of the texture region of interest is large, the internal texture change trend is similar. The gradient image corresponding to the edge detection can reflect the change trend of the texture in the original image, and the quality of the pixel information is higher, which can reduce the influence of noise and resolution differences on texture similarity to a certain extent. Since the Canny operator has good noise resistance, can detect weak edges, has high positioning accuracy and effectively suppresses false edges, it is a relatively excellent edge extraction operator, so the Canny operator is used to complete the gradient calculation of the texture region of interest image (Canny is a classic algorithm in the field of computer vision, and the algorithm principle is not described in this application).

[0095] S2032: superimposing the gradient image with the original image to obtain a gradient enhanced image;

[0096] The pixels in the gradient image are obtained by differentiating adjacent pixels in the original image, so the brightness of the gradient image obtained by some algorithms is low, which affects the subsequent feature extraction. In the present invention, the gradient image is superimposed with the original image to enhance the features of the image.

[0097] Assume that the pixel value of the gradient image is {H i}, the pixel value of the original image is {J i}, the pixel value of the gradient enhanced image is {P i},but In the formula, H max With H min are pixels {H i}; max{Pix i} and min{Pix i} respectively {Pix i} maximum and minimum values ​​in .

[0098] Attached Figure 8 is the enhanced image. Figure 8 In the figure, (a) and (d) are the images of the texture region of interest of the heterogeneous point cloud mapping image; (b) and (e) are the gradient images of the texture region of interest; (c) and (f) are the gradient enhanced images of the texture region of interest.

[0099] S204: Establish Harris and SIFT feature point sets based on the scale space.

[0100] Image features are generally divided into edge features, corner point features and spot features. Corner points are generally located at the intersection of edges, and representative algorithms include Harris, etc.; spots are different from corner points and edges, and mainly describe the extreme values ​​of pixel grayscale changes, and representative algorithms include SIFT, etc. When extracting features from the same image, spots and corner points will have partial intersections, but they are not congruent. In order to solve the problem that a single feature point extraction algorithm in a gradient enhanced image cannot guarantee the number of feature points, the present invention combines Harris and SIFT in scale space to form a feature point set to increase the stability of subsequent matching.

[0101] Since the feature points extracted by the SIFT algorithm are scale-invariant, while the feature points extracted by the Harris algorithm are not scale-invariant, this section introduces the Gaussian scale space and extracts Harris corner points with scale information. Finally, the combination of Harris and SIFT feature points is used as the feature point set of the image for subsequent image matching. The specific operation includes the following steps:

[0102] S2041: Use Gaussian convolution kernel G(x, y, σ i ) Establish a Gaussian scale space and calculate the M matrix at different scales. Among them, A(σ i ) = G(x, y, σ i )*(I x ) 2 ,B(σ i ) = G(x, y, σ i )*(I y ) 2 ,C(σ i ) = G(x, y, σ i )*(I x I y ) 2 , where I x with I y Represent the first-order partial derivatives of the image in the x and y axis directions, respectively, and * represents the convolution operation; on this basis, the corresponding R(σ i ), R(σ i )=det(M(σ i ))-τ·tr 2 (M(σ i )), where τ∈[0.04-0.06] is the weight coefficient;

[0103] S2042: At each scale, use non-maximum suppression to obtain the corresponding R(σ i ), when R(σi ) is greater than the threshold and is the maximum value within a given neighborhood, the corresponding point is taken as a candidate Harris corner point;

[0104] S2043: Filter candidate Harris corner points along the scale direction from small to large in the scale space, as shown in the attached figure. Fig. 9 As shown in the figure; if a candidate Harris corner point exists continuously in the scale space, then this corner point is a Harris feature point, which can remove the influence of noise or isolated pixels; accumulate the Harris feature points of all scale spaces as the final Harris feature point set;

[0105] S2044: Extract SIFT feature points from the gradient enhanced image in the same scale space.

[0106] SIFT is a classic algorithm in the field of computer vision. The present invention does not elaborate on the principle of the algorithm. Fig.10 Shown are SIFT and Harris feature point sets, where yellow is Harris feature points and red is SIFT feature points.

[0107] Further, step S3: based on the multi-channel image of the point cloud RGB, the normal vector Dip component, and the roughness and curvature attribute mapping, a feature descriptor integrating the point cloud geometry and the RGB attributes is generated;

[0108] Aiming at the problem that the existing single-measure feature descriptor cannot effectively complete gradient image matching, the present invention designs a SIFT improved descriptor based on multi-channel images of point cloud mapping, which integrates the geometric and RGB texture information of point clouds, to improve the similarity of feature points with the same name. The algorithm operation includes the following steps:

[0109] S301: using the same Gaussian convolution kernel function as that used in feature point extraction in step S2 to establish a scale space of a point cloud mapping multi-channel image, wherein the multi-channel image includes an RGB channel, a Dip channel, a curvature channel, and a roughness channel;

[0110] In step S103, the mapping images of various geometric attributes of the point cloud are extracted. After excluding the dip direction component and normalized elevation of the normal vector attribute, the mapping images of four channels are obtained, among which the RGB channel image has been used for the previous extraction of the texture area of ​​interest and the feature point set. Fig.11 The three-channel image obtained by mapping the geometric attributes of the point cloud except the RGB channel is shown in the attached Fig.11In the figure, (a)-(c) are the normal vector Dip channel, curvature channel and roughness channel images of the close-range photogrammetry point cloud mapping, (d)-(f) are the normal vector Dip channel, curvature channel and roughness channel images of the drone photogrammetry point cloud mapping; it should be noted that the attached Fig.11 In order to facilitate display, the grayscale values ​​of the three channels are normalized to the interval [0-255]. However, in actual calculations, since the Dip component, curvature and roughness values ​​have corresponding physical properties and the point cloud coverage range is inconsistent, non-normalized pixel values ​​are used to construct the descriptor.

[0111] S302: Count the gradient distribution of the neighborhood of the RGB channel feature points of the gradient enhanced image of the texture region of interest, establish the reference direction of the feature points, and determine the area of ​​each channel image; specifically,

[0112] S3021: Search for the gradient magnitude and direction of pixels within a 3σ neighborhood in the scale image corresponding to the feature point, where 3σ = 3×1.5×σ oct , where σ oct The Gaussian parameters used to generate adjacent scale images in each scale space;

[0113]

[0114] Where L is the scale space where the feature point is located (the scale space is a set of images formed by an image after different degrees of Gaussian blur, similar to an image pyramid, that is, the scale space is composed of the same image at different scales); m(x, y) and θ(x, y) are the magnitude and direction of the gradient respectively;

[0115] S3022: Use the histogram to count the pixel gradient distribution in the neighborhood, use the pixel gradient direction to determine the columnar interval, multiply the amplitude by the weight and add it to the corresponding columnar interval to complete the final statistics; similar to the SIFT algorithm, the gradient histogram divides the gradient direction [0-2] into 36 bins, as shown in the attached figure. Fig.12 As shown, the peak of the histogram represents the main direction of the feature point (only eight directions are drawn to simplify the expression). In order to enhance the robustness of the matching, the direction greater than 70% of the peak of the main direction is retained as the auxiliary direction of the feature point, so multiple feature points will be created at the same position and scale. Since the number of feature points obtained in step S2 is small, adding auxiliary directions will not significantly increase the computational overhead, but will increase the stability of the subsequent matching algorithm. After that, the discrete gradient histogram is smoothed to remove noise interference, and interpolation fitting is performed to finally determine the precise angle direction.

[0116] S3023: In the scale space corresponding to the feature point, determine the image regions of each channel required by the descriptor; divide the neighborhood near the feature point into 2×2 sub-regions, each sub-region is used as a seed point, and the gradient features in 8 directions are counted. Each sub-region contains 12×σ oct sub-pixels, that is, the side length of the sub-region is Similar to SIFT, taking into account the linear interpolation and rotation factors, the radius of the area corresponding to each channel image is The calculation result of radius is rounded.

[0117] S303: Rotate the image regions of each channel determined in step S302 according to the reference direction of the feature points to make the descriptor rotationally invariant, as shown in the attached figure. Fig.13 As shown, the coordinate axis is rotated to the direction of the feature point, and the new coordinates of the sampling points in the neighborhood after rotation are exist Fig.13 In the figure, the circular area is the neighborhood range of the feature point, and the red arrow is the main direction of the neighborhood pixel gradient statistics.

[0118] S304: After the feature point neighborhood is rotated, the gradients of each pixel in the sub-region of the neighborhood are distributed to 8 statistical directions, and the weights are calculated to perform histogram statistics;

[0119] First, determine the subscript of the sub-region where the pixel point is located after rotation

[0120] Then, the gradient of the pixels in the area is weighted according to the Gaussian kernel G' with σ=1 and radius 3σ. The weighted gradient is ω=M(a+x,b+y)*G', where (x,y)∈3σ and (a,b) is the position of the feature point in the scale image.

[0121] S305: Calculate a 128-dimensional feature descriptor that fuses point cloud geometry and RGB attributes.

[0122] As attached Fig.14 As shown in the figure, according to the region subscript (x", y") of each pixel point, the 8-directional gradient of the seed point is interpolated and calculated. Because a 2×2 subregion is used, each channel image will generate 2×2×8, i.e. 32 gradient information as the corresponding feature vector of the channel.

[0123] S3052: To enhance the robustness of the descriptor to changes in ambient light and device saturation, the 32-dimensional gradient information obtained from each channel is normalized and thresholded, as shown in the attached figure. Fig.14As shown in the figure, since each feature point corresponds to a 4-channel image at a specific scale (RGB gradient enhancement channel image, normal vector Dip channel image, curvature channel image and roughness channel image), the 32-dimensional descriptors of the 4 channels are connected in series to finally obtain a 2×2×8×4 or 128-dimensional descriptor. The feature weights between the four channels in the algorithm are all 1.

[0124] This descriptor not only integrates the geometric and RGB texture information of the point cloud, increases the similarity of features with the same name, but also does not increase the calculation dimension of the descriptor, which guarantees the efficiency of the algorithm to a certain extent. Each channel image will generate 2×2×8, or 32 gradient information as the feature vector corresponding to the channel;

[0125] Further, step S4: matching of feature points of point cloud mapping images;

[0126] Similar to SIFT, feature point matching is achieved by calculating the Euclidean distance of the 128-dimensional feature vector between two sets of feature points. The distance ratio between the nearest point and the second nearest point is used to finally determine the correspondence between the features, and RANSAC is used to remove false matches. Fig.15 The corresponding relationship and matching effect of the final feature points, where (a) is the feature of the same name found in the enhanced image of the texture region of interest by the method of the present invention, and (b) is the matching effect of the enhanced image of the texture region of interest. Fig.15 The accuracy of visible matching has been greatly increased; Fig.16 This is the final matching effect of the image obtained by point cloud mapping. It can be seen that the feature point extraction and description method proposed in the present invention can effectively solve related matching problems.

[0127] Furthermore, step S5: obtaining corresponding three-dimensional matching points according to the 3D-2D mapping relationship to complete the rough matching of the point cloud.

[0128] In the voxel-based point cloud mapping image, the mapping relationship between the three-dimensional points and the image points is retained when the image is generated. The corresponding points between the source point cloud and the target point cloud are obtained based on the matching points of the image. The transformation matrix between the heterogeneous point clouds can be solved to finally complete the rough matching between the point clouds. Fig.17 With attached Fig.18 The position distribution of the three-dimensional points of the same name found by close-range projection and drone photography based on the 3D-2D mapping relationship in the heterogeneous point cloud is shown in the attached figure. Fig.17 and 18 In the figure, the left picture shows the feature points with the same name extracted from the point cloud mapping image, and the right picture shows the feature points with the same name in the 3D point cloud obtained based on the 3D-2D mapping relationship. Fig.19 This is the final effect of using the method of the present invention to perform rough matching of point clouds.

[0129] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. Rough matching method of heterogeneous point clouds based on geometry and texture mapping, It is characterized in that The following steps are included: S1: Generate point cloud mapping image based on source point cloud and model point cloud; S2: Use RGB channel mapping image to extract feature points; S3: Generate a feature descriptor that integrates point cloud geometry and RGB attributes based on multi-channel images of point cloud RGB, normal vector Dip component, roughness and curvature attribute mapping; S4: point cloud mapping image feature point matching; S5: Obtain corresponding 3D matching points according to the 3D-2D mapping relationship to complete the rough matching of the point cloud.

2. According to the method for rough matching of heterogeneous point clouds based on geometry and texture mapping in claim 1, It is characterized in that The specific operation of step S1 includes the following steps: S101: Establish a directed bounding box of the point cloud; S102: performing cube voxel segmentation on the directed bounding box; S103: Perform point cloud mapping based on voxels to obtain a multi-channel image of the point cloud.

3. The method for coarse matching of heterogeneous point clouds based on geometry and texture mapping according to claim 2, It is characterized in that The specific operation of step S2 includes the following steps: S201: obtaining a rough overlapping area of ​​images based on GPS assistance; S202: extracting images of the texture area of ​​interest; S203: generating a gradient enhanced image of the texture region of interest; S204: Establish Harris and SIFT feature point sets based on the scale space.

4. The method for coarse matching of heterogeneous point clouds based on geometry and texture mapping according to claim 3, It is characterized in that In step S202, the Grab Cut algorithm and the image mask are used to extract the texture region of interest.

5. The method for coarse matching of heterogeneous point clouds based on geometry and texture mapping according to claim 4, It is characterized in that The specific operation of step S203 includes the following steps: S2031: Use the canny operator to obtain the gradient image of the texture area of ​​interest; S2032: superimposing the gradient image with the original image to obtain a gradient enhanced image; Assume that the pixel value of the gradient image is {H i }, the pixel value of the original image is {J i }, the pixel value of the gradient enhanced image is {P i },but In the formula, H max With H min are pixels {H i }; max{Pix i } and min{Pix i } respectively {Pix i } maximum and minimum values ​​in .

6. The method for coarse matching of heterogeneous point clouds based on geometry and texture mapping according to claim 5, It is characterized in that The specific operation of step S204 includes the following steps: S2041: Use Gaussian convolution kernel G(x, y, σ i ) Establish a Gaussian scale space and calculate the M matrix at different scales. Among them, A(σ i ) = G(x, y, σ i )*(I x ) 2 ,B(σ i ) = G(x, y, σ i )*(I y ) 2 ,C(σ i ) = G(x, y, σ i )*(I x I y ) 2 , where I x with I y Represent the first-order partial derivatives of the image in the x and y axis directions, respectively, and * represents the convolution operation; on this basis, the corresponding R(σ i ), R(σ i )=det(M(σ i ))-τ·tr 2 (M(σ i )), where τ∈[0.04-0.06] is the weight coefficient; S2042: At each scale, use non-maximum suppression to obtain the corresponding R(σ i ), when R(σ i ) is greater than the threshold and is the maximum value within a given neighborhood, the corresponding point is taken as a candidate Harris corner point; S2043: filtering candidate Harris corner points along the scale direction from small to large in the scale space. If a candidate Harris corner point exists continuously in the scale space, this corner point is a Harris feature point. Accumulating Harris feature points in all scale spaces as the final Harris feature point set; S2044: Extract SIFT feature points from the gradient enhanced image in the same scale space.

7. The method for coarse matching of heterogeneous point clouds based on geometry and texture mapping according to claim 6, It is characterized in that The specific operation of step S3 includes the following steps: S301: using the same Gaussian convolution kernel function as that used in feature point extraction in step S2 to establish a scale space of a point cloud mapping multi-channel image, wherein the multi-channel image includes an RGB channel, a Dip channel, a curvature channel, and a roughness channel; S302: Counting the gradient distribution of the neighborhood of the RGB channel feature points of the gradient enhanced image of the texture region of interest, establishing the reference direction of the feature points, and determining the area of ​​each channel image; S303: Rotate the coordinate axis of each channel image area determined in step S302 according to the reference direction of the feature point. The new coordinates of the sampling points in the neighborhood after rotation are S304: After the feature point neighborhood is rotated, the gradients of each pixel in the sub-region of the neighborhood are distributed to 8 statistical directions, and the weights are calculated to perform histogram statistics; S305: Calculate a 128-dimensional feature descriptor that fuses point cloud geometry and RGB attributes.

8. The method for coarse matching of heterogeneous point clouds based on geometry and texture mapping according to claim 7, It is characterized in that The specific operation of step S302 includes the following steps: S3021: Search for the gradient magnitude and direction of pixels within a 3σ neighborhood in the scale image corresponding to the feature point, where 3σ = 3×1.5×σ oct , where σ oct The Gaussian parameters used to generate adjacent scale images in each scale space; In the formula, L is the scale space where the feature points are located; m(x, y) and θ(x, y) are the magnitude and direction of the gradient respectively; S3022: Using a histogram to count pixel gradient distribution in the neighborhood, using the gradient direction of the pixel to determine the histogram interval, multiplying the amplitude by the weight and superimposing it to the corresponding columnar interval to complete the final statistics; S3023: In the scale space corresponding to the feature point, determine the image regions of each channel required by the descriptor; divide the neighborhood near the feature point into 2×2 sub-regions, each sub-region is used as a seed point, and the gradient features in 8 directions are counted. Each sub-region contains 12×σ oct sub-pixels, that is, the side length of the sub-region is Finally, the area radius corresponding to each channel image is 9. The method for coarse matching of heterogeneous point clouds based on geometry and texture mapping according to claim 8, It is characterized in that The specific operation of step S304 includes the following steps: S3041: Determine the subscript of the sub-region where the pixel point is located after rotation S3042: The gradient of the pixels in the area is weighted according to the Gaussian kernel G' with σ=1 and radius 3σ, and the weighted gradient is ω=M(a+x, b+y)*G', where (x, y)∈3σ and (a, b) is the position of the feature point in the scale image.

10. The method for coarse matching of heterogeneous point clouds based on geometry and texture mapping according to claim 9, It is characterized in that The specific operation of step S305 includes the following steps: S3051: According to the region subscript (x", y") of each pixel point, the 8-directional gradient of the seed point is interpolated and calculated. Each channel image will generate 2×2×8, i.e., 32 gradient information as the feature vector corresponding to the channel; S3052: The 32-dimensional gradient information obtained from each channel is normalized and threshold-truncated, and the 32-dimensional descriptors of the four channels are connected in series to finally obtain a 2×2×8×4, i.e., 128-dimensional descriptor. In the algorithm, the feature weights between the four channels are all 1.

Images