Rock slope weak texture image registration method fusing optical flow method, Point-CNN network and ORB algorithm

Abstract

The invention discloses a rock slope weak texture image registration method fusing an optical flow method, a Point-CNN network and an ORB algorithm. The method comprises the following steps: S1, extracting advanced features from a multi-source fusion remote sensing image of a weak texture rock slope surface by using the Point-CNN network; s2, generating a local feature descriptor and a global descriptor based on the convolutional neural network; s3, feature detection and feature description are carried out by using an ORB algorithm; s4, fusing the Point-CNN high-level features and the BRIEF low-level features based on a normalization method; step S5, obtaining an accurate feature point position by using a Lucas-Kanade optical flow method; and S6, introducing an RANSAC (Random Sample Consensus) algorithm to eliminate unmatched feature point pairs. According to the method, enough feature points can be efficiently obtained, and the problems that existing weak texture images are low in registration precision, small in feature point extraction number, low in efficiency and the like are solved.

Description

Technical Field

[0001] The invention relates to a weak texture image registration method, and in particular to a rock slope weak texture image registration method integrating an optical flow method, a Point-CNN network and an ORB algorithm. Background Art

[0002] The use of UAV remote sensing and mapping technology to obtain high-resolution three-dimensional models of geological disasters, and then to measure and analyze them, obtain the structural surface characteristics of the geological body surface, and provide important decision-making support for geological disaster rescue. Rock slopes with weak texture characteristics are widely distributed in Chongqing. The existing remote sensing technology for shooting weak texture features is often restricted by factors such as light intensity, shooting angle, and observation distance. At the same time, weak texture remote sensing images have the characteristics of low reflectivity, high pixel similarity, and unclear differences between regions, which makes feature extraction and description difficult. Therefore, it is urgent to study the mapping technology for Chongqing's weak texture rock slopes, extract and match the weak texture features of the slope body, and then analyze the control effect of weak texture on the stability of the slope body.

[0003] In the field of computer vision, feature extraction is an important task that can be used for applications such as image matching, target recognition, and image stitching. SIFT, SURF, ORB, and FAST are widely used feature detection algorithms. Among them, the running time of the ORB feature extraction algorithm is much better than that of SIFT and SURF, and it can be used for real-time feature detection. The ORB feature extraction algorithm is a feature point detection and description technology based on the FAST corner point. It has scale and rotation invariance, and is also invariant to perspective affine and environmental noise. The powerful feature monitoring and description performance makes the application scenarios of the ORB feature detection algorithm very wide, but the ORB feature extraction algorithm is more sensitive to lighting changes. In addition, in a weak texture environment, the traditional ORB algorithm is difficult to extract and match features, resulting in large inter-frame pose resolution errors. At the same time, the calculation and matching of feature point descriptors may be very time-consuming and computationally inefficient.

[0004] In recent years, with the development of deep learning methods, convolutional neural networks have achieved great success in various fields such as image retrieval, image classification, and object recognition. The latest performance of convolutional neural networks in these fields shows that convolutional neural networks have powerful feature representation and discrimination capabilities in various visual tasks. Based on convolutional neural networks, the problem of changing illumination patterns can be modeled as a supervised deep learning problem, and its changes can be directly predicted. The Point-CNN fully convolutional neural network proposed in recent years adapts to irregular and different density point cloud data through local and global feature learning, showing strong robustness and generalization ability. Therefore, combining the advantages of Point-CNN convolutional neural network and feature detection algorithm can provide better performance in image registration accuracy and the number of feature point extraction. Summary of the invention

[0005] The present invention provides a weak texture image registration method for rock slopes that integrates the optical flow method, the Point-CNN network and the ORB algorithm. The method can efficiently obtain enough feature points, solve the problems of low precision, small number of feature point extraction, and low efficiency in the existing weak texture image registration, enhance the application of the ORB feature extraction algorithm in complex environments, and further provide more accurate and effective information for three-dimensional real-scene modeling of rock slopes in weak texture environments.

[0006] The objective of the present invention is achieved through the following technical solutions:

[0007] A rock slope weak texture image registration method integrating optical flow method, Point-CNN network and ORB algorithm includes the following steps:

[0008] Step S1: Using Point-CNN convolutional neural network to extract high-level features from multi-source fused remote sensing images of weakly textured rock slope surfaces;

[0009] Step S2: generating local feature descriptors and global descriptors based on convolutional neural networks;

[0010] Step S3: Use ORB algorithm to perform feature detection and feature description;

[0011] Step S4: Fusion of Point-CNN high-level features and BRIEF low-level features based on normalization method;

[0012] Step S5: using the Lucas-Kanade optical flow method to obtain accurate feature point positions;

[0013] Step S6: Introduce the RANSAC algorithm to eliminate unmatched feature point pairs.

[0014] Compared with the prior art, the present invention has the following advantages:

[0015] The present invention integrates a Point-CNN convolutional neural network, a Lucas-Kanade optical flow method and an ORB feature extraction algorithm, performs feature extraction on a weak-texture remote sensing image of a rock slope through the Point-CNN convolutional neural network, generates local and global descriptors based on a convolutional neural network based on bilinear interpolation and L2-norm operations, calculates descriptors through FAST feature point extraction and a BRIEF algorithm based on the ORB feature extraction algorithm, and then normalizes the Point-CNN descriptors and the BRIEF descriptors. Finally, the Lucas-Kanade optical flow method is used to obtain accurate feature point positions, and the RANSAC algorithm is introduced to eliminate unmatched feature point pairs, thereby effectively improving the efficiency and accuracy of weak-texture feature extraction and image registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flow chart of the rock slope weak texture image registration method that integrates the optical flow method, Point-CNN network and ORB algorithm. DETAILED DESCRIPTION

[0017] The technical solution of the present invention is further described below in conjunction with the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be included in the protection scope of the present invention.

[0018] The present invention provides a rock slope weak texture image registration method integrating optical flow method, Point-CNN network and ORB algorithm. Figure 1 As shown, the method comprises the following steps:

[0019] Step S1: Use Point-CNN convolutional neural network to extract high-level features from multi-source fusion remote sensing images of weak texture rock slope surface, including feature extraction and X matrix training. The specific steps are as follows:

[0020] Step S11: Feature extraction

[0021] First, coordinate transformation is performed to establish a local coordinate system centered on the mapping point m. All points in the input image point set M are transformed to obtain a new point set M'. Then, high-dimensional space mapping is performed. A multi-layer perceptron (MLP) is used to map each point in the point set M' to the high-dimensional space γ to obtain the feature matrix K. γ , and concatenate the feature matrix with the feature point set K of the input point to obtain a new feature matrix K*;

[0022] Step S12: X matrix training

[0023] Combined with the new point set M' obtained in step S11, a transformation matrix X of size n×n is obtained through the multilayer perceptron training data, and the feature matrix K* obtained in step S11 is transformed to obtain a new feature matrix K X Finally, the convolution kernel and the feature matrix K X Perform convolution to generate feature maps.

[0024] Step S2: Generate local feature descriptors and global descriptors based on convolutional neural networks.

[0025] The output features obtained in step S1 are subjected to bilinear interpolation (quartic linear interpolation) and L2-norm operations to generate local feature descriptors, and the feature maps are fed to the NetVLAD layer, and then a dimension reduction operation is performed to obtain a global descriptor of size 4096, and the global feature generator is used to obtain the global descriptor of the image.

[0026] VLAD (Vector of Local Aggregated Descriptors) is an image feature extraction method. NetVLAD is based on it by designing a symbolic function a z (x i ) to obtain a derivable weight function and obtain the optimized feature extraction method, and the parameter c is obtained through training z , VLAD is considered to be a classification problem, and assuming that there are Z categories, the global feature of the image can be calculated by the difference distribution of local features in Z categories.

[0027] The symbolic function a is designed z (x i )for:

[0028]

[0029] In the formula, β is a parameter greater than 0.

[0030] Step S3: Use the ORB algorithm to perform feature detection and feature description.

[0031] The ORB feature detection algorithm combines the BRIEF feature descriptor with the FAST feature point detection method. The algorithm is divided into two parts: FAST feature point extraction and BRIEF feature description. The specific steps are as follows:

[0032] Step S31: FAST feature point extraction

[0033] Based on the ID3 algorithm, the feature with the largest information gain is calculated and selected as the node partition feature, and a sub-node is established based on it. Then, a decision tree is constructed by recursive operation on the sub-nodes. In the decision tree, 16 pixel points on the circumference centered on the feature point are input to screen out the optimal FAST feature point. The non-maximum suppression algorithm is used to remove multiple locally dense feature points. The absolute value and deviation of the feature point P and the 16 feature points on its circumference are calculated. The response size is calculated for each feature point, and the feature points with larger response values ​​are compared and retained, and other feature points are deleted. With the feature point as the center of the circle and r as the radius, the center of mass of the feature points within the range is calculated by the moment method, and the vector formed from the feature point coordinates to the center of mass is used as the direction of the feature point, thereby obtaining the direction of the FAST feature point.

[0034] Step S32: BRIEF feature description

[0035] Based on the BRIEF algorithm, feature descriptors are obtained. n pairs of point sets are selected in the neighborhood of the feature point with a size of 31*31. The point set data is Gaussian smoothed and then rotated at an angle θ to obtain new point pairs. A 5*5 sub-neighborhood of each point in the neighborhood is selected, and its grayscale average is calculated as the value of the point pair. Finally, a binary BRIEF feature descriptor is formed by comparing the grayscale values ​​of each point pair.

[0036] Step S4: Fusion of Point-CNN high-level features and BRIEF low-level features based on normalization method.

[0037] Since the BRIEF feature descriptor f r With the Point-CNN descriptor f p The difference is large, and both are calculated directly, so it is necessary to normalize the two feature descriptors based on the same candidate feature point. Here, the Euclidean distance is used to calculate the similarity between the reference image key point M1 and the sensed image candidate key point M2 corresponding to the candidate feature point, which is recorded as:

[0038]

[0039] Where a1 and a2 are two weight parameters, a1, a2∈[0, 1], and a1+a2=1, N1 and N2 are the dimensions of Point-CNN descriptor and BRIEF feature descriptor respectively.

[0040] Step S5: Use the Lucas-Kanade optical flow method to obtain accurate feature point positions.

[0041] The Lucas-Kanade optical flow method is used instead of the traditional feature point matching method to calculate the change in pixel values ​​between adjacent frames, determine the location of the feature points, and minimize the optical flow error through iterative optimization to obtain the accurate location of the feature points. The calculation process of the Lucas-Kanade optical flow method is as follows:

[0042] The Lucas-Kanade optical flow method assumes that the pixel values ​​of image feature points between adjacent frames do not change with frame movement and change slowly over time, thus obtaining the constraint equation:

[0043] H(z,y,t)=H(z+δz,y+δy,t+δt) (3)

[0044] Where H is the brightness value of the feature point at z, y at time t.

[0045] Taylor expansion is performed on the right side of formula (3), and formula (3) is simplified to become:

[0046]

[0047] In the formula, are the gradients of the image in the z and y directions respectively, is the speed at which the image grayscale changes over time.

[0048] In addition, the Lucas-Kanade optical flow method also assumes that the motion of feature points in a certain space is consistent. Therefore, the optical flow corresponding to the Lucas-Kanade optical flow method is obtained by combining formula (4):

[0049]

[0050] In the formula, and are the speeds of the pixel in the z and y directions respectively, B=[H z ,H y ] j .

[0051] Step S6: Introduce the RANSAC algorithm to eliminate unmatched feature point pairs.

[0052] First, the feature points of incorrect tracking are determined according to the defined threshold, and then feature point pairs are randomly extracted. The mathematical model of the extracted samples is established based on the least squares method. The errors between the model calculation results and other feature point data are compared, and the data that are not "inliers" (do not meet the error requirements) are marked as outliers. Through iterative calculation, the model with the largest number of "inliers" is found or the maximum number of iterations is reached, which is recorded as the optimal model. Finally, all feature point pairs whose distance to the optimal model is greater than the threshold are marked as mismatched feature point pairs.

[0053] Here, the maximum number of iterations can be calculated by the maximum number of iterations m as follows:

[0054]

[0055] Where N is the number of points selected for each fitting model, P is the probability of obtaining the correct model through RANSAC, and p is the probability that a randomly selected feature point is an "inlier", which is usually a priori value. However, if this value cannot be determined in advance, the adaptive number of iterations can be used. Initially, an infinite number of iterations is given, and then each time the model is updated, the current proportion of the number of "inliers" is used as an approximate value of p.

Claims

1. A rock slope weak texture image registration method integrating optical flow method, Point-CNN network and ORB algorithm, characterized by The method comprises the following steps: Step S1: Using Point-CNN convolutional neural network to extract high-level features from multi-source fused remote sensing images of weakly textured rock slope surfaces; Step S2: generating local feature descriptors and global descriptors based on convolutional neural networks; Step S3: Use ORB algorithm to perform feature detection and feature description; Step S4: Fusion of Point-CNN high-level features and BRIEF low-level features based on normalization method; Step S5: using the Lucas-Kanade optical flow method to obtain accurate feature point positions; Step S6: Introduce the RANSAC algorithm to eliminate unmatched feature point pairs.

2. The rock slope weak texture image registration method integrating optical flow method, Point-CNN network and ORB algorithm according to claim 1 is characterized in that The specific steps of step S1 are as follows: Step S11: Feature extraction First, coordinate transformation is performed to establish a local coordinate system centered on the mapping point m. All points in the input image point set M are transformed to obtain a new point set M'. Then, high-dimensional space mapping is performed. A multi-layer perceptron is used to map each point in the point set M' to the high-dimensional space γ to obtain the feature matrix K. γ , and concatenate the feature matrix with the feature point set K of the input point to obtain a new feature matrix K*; Step S12: X matrix training Combined with the new point set M' obtained in step S11, a transformation matrix X of size n×n is obtained through the multilayer perceptron training data, and the feature matrix K* obtained in step S11 is transformed to obtain a new feature matrix K X Finally, the convolution kernel and the feature matrix K X Perform convolution to generate feature maps.

3. The rock slope weak texture image registration method integrating optical flow method, Point-CNN network and ORB algorithm according to claim 1 is characterized in that In the step S2, bilinear interpolation and L2-norm operation are performed on the output features obtained in step S1 to generate local feature descriptors, the feature maps are fed to the NetVLAD layer, and then a dimensionality reduction operation is performed to obtain a global descriptor, and the global descriptor of the image is obtained using a global feature generator.

4. The rock slope weak texture image registration method integrating optical flow method, Point-CNN network and ORB algorithm according to claim 1 is characterized in that The specific steps of step S3 are as follows: Step S31: FAST feature point extraction Based on the ID3 algorithm, the feature with the largest information gain is calculated and selected as the node partition feature, and a sub-node is established based on it. Then, a decision tree is constructed by recursive operation on the sub-nodes. In the decision tree, 16 pixel points on the circumference centered on the feature point are input to screen out the optimal FAST feature point. The non-maximum suppression algorithm is used to remove multiple locally dense feature points. The absolute value and deviation of the feature point P and the 16 feature points on its circumference are calculated. The response size is calculated for each feature point. The feature points with larger response values ​​are compared and retained, and other feature points are deleted. With the feature point as the center of the circle and r as the radius, the center of mass of the feature points within the range is calculated by the moment method. The vector formed from the feature point coordinates to the center of mass is used as the direction of the feature point, and the direction of the FAST feature point is obtained. Step S32: BRIEF feature description Based on the BRIEF algorithm, feature descriptors are obtained. n pairs of point sets are selected in the neighborhood of the feature point with a size of 31*31. The point set data is Gaussian smoothed and then rotated at an angle θ to obtain new point pairs. A 5*5 sub-neighborhood of each point in the neighborhood is selected, and its grayscale average is calculated as the value of the point pair. Finally, a binary BRIEF feature descriptor is formed by comparing the grayscale values ​​of each point pair.

5. The rock slope weak texture image registration method integrating optical flow method, Point-CNN network and ORB algorithm according to claim 1 is characterized in that In step S4, the Euclidean distance is used to calculate the similarity between the reference image key point M1 and the sensed image candidate key point M2 corresponding to the candidate feature point, which is expressed as: Where a1 and a2 are two weight parameters, a1, a2∈[0, 1], and a1+a2=1, N1 and N2 are the dimensions of Point-CNN descriptor and BRIEF feature descriptor respectively, and f r is the BRIEF feature descriptor, f p is the Point-CNN descriptor.

6. The rock slope weak texture image registration method integrating optical flow method, Point-CNN network and ORB algorithm according to claim 1 is characterized in that In step S5, the Lucas-Kanade optical flow method is used to calculate the change in pixel values ​​between adjacent frames, determine the position of the feature point, and minimize the optical flow error through iterative optimization to obtain the accurate position of the feature point.

7. The rock slope weak texture image registration method integrating optical flow method, Point-CNN network and ORB algorithm according to claim 1 is characterized in that In step S6, the feature points of incorrect tracking are determined according to the defined threshold, and then feature point pairs are randomly extracted. A mathematical model of the extracted samples is established based on the least squares method, and the errors between the model calculation results and other feature point data are compared. The data that are not "inliers" are marked as outliers. Through iterative calculation, the model with the largest number of "inliers" is found or the maximum number of iterations is reached, which is recorded as the optimal model. Finally, all feature point pairs whose distance from the optimal model is greater than the threshold are marked as unmatched feature point pairs.

Images