Unmanned vehicle ORB visual multi-feature fusion matching method based on radial orthogonal mean value

Abstract

The invention provides an unmanned vehicle ORB visual multi-feature fusion matching method based on a radial orthogonal mean (ROM), and aims to extract uniformly distributed feature points through a self-adaptive ROM-FAST algorithm, and improve the robustness by adopting a multi-feature fusion descriptor. The method comprises the following steps: firstly, converting a color image into a grey-scale image and constructing a multi-scale Gaussian pyramid image; secondly, extracting feature points on the multi-scale image by using an adaptive ROM-FAST algorithm; thirdly, converting the grayscale image into an HSV color space for quantization, counting pixel intensity distribution in circular neighborhoods of feature points at different channels and different radiuses, taking the pixel intensity distribution as a local color histogram descriptor, and extracting an rBRIEF descriptor at the same time; then, weighted difference calculation is carried out to establish an FAST-SCORE map, and coordinate correction is carried out on the upper-layer feature points according to the FAST-SCORE map; and finally, splicing the local color histogram descriptor and the rBRIEF descriptor for visual image matching. Through the method, the detection number and the matching number of the feature points are increased, and mismatching points are remarkably eliminated.

Description

Technical Field

[0001] The present invention belongs to the field of unmanned systems and robot intelligent perception and control equipment, in particular to the field of vision-based feature point matching of unmanned vehicles, and more particularly to an unmanned vehicle ORB vision multi-feature fusion matching method based on radial orthogonal mean. Background Art

[0002] Simulation Location And Mapping (SLAM) is widely used in the field of intelligent perception and control equipment represented by unmanned systems and robots, such as autonomous intelligent unmanned vehicles, surface unmanned ships, underwater salvage equipment, intelligent unmanned aerial vehicles, industrial robots, special operation robots, service consumption robots, etc. Image matching technology, as the basis of SLAM, plays a decisive role in the mapping effect. Image matching algorithms can be divided into: transform domain-based methods, grayscale-based methods, and feature-based methods. Feature-based image registration has good robustness, small calculation amount, and fast speed, and has become the mainstream direction of research. The feature points extracted from the visual image need to have certain robustness to the image's illumination transformation, scale transformation, rotation transformation, etc., in order to effectively ensure the accurate matching of the feature points. The existing feature point extraction and matching algorithms still have shortcomings in matching accuracy. With the development of image processing technology, higher requirements are put forward for the accuracy and real-time performance of feature point extraction and matching algorithms.

[0003] In visual SLAM, feature point extraction and accurate feature matching play an important role in the robot's pose inference. At present, the ORB (Oriented FAST and Rotated BRIEF) algorithm is widely used in visual SLAM due to its fast feature point extraction speed and good stability. However, the ORB algorithm extracts features on grayscale images, without considering the color information and texture information of the original color image. At the same time, the traditional ORB algorithm uses the FAST algorithm to extract feature points. The extracted feature points are not scale-invariant, and the robustness of image registration with scale changes is low, and the feature effect is not ideal in dark light.

[0004] To address the above problems, the paper "ORB Feature Matching Algorithm Based on Multi-Scale Feature Description Fusion and Feature Point Mapping Error" proposed a multi-scale descriptor generation method using an image pyramid structure and performed coordinate correction on the mapping positions of the feature points in the upper layer of the pyramid. However, the following problems still exist: the paper still uses the FAST algorithm to extract feature points, and the extracted feature points are concentrated in areas with obvious textures such as edges. At the same time, since the descriptor lacks information such as color and texture of the image, the matching effect is poor in dim light.

[0005] The paper "Radial Orthogonal Mean Local Binary Pattern: An IlluminationInvariant Feature Extraction Method" proposes an illumination invariant feature extraction method based on the radial orthogonal mean. However, the paper only applies the radial orthogonal mean to the generation of descriptors and does not consider its application in feature point extraction. At the same time, the selection of the threshold size will affect the number of feature points extracted in the end. In this method, the feature point threshold is artificially set to a fixed value, which will be affected by changes in ambient light and contrast.

[0006] The paper "A Novel Color-Texture Descriptor Based on Local Histograms for Image Segmentation" proposes a color-texture image segmentation method based on local histograms. This method first quantifies the color, and for each pixel, by counting the number of pixels representing each color in its circular neighborhood, obtains a local color histogram to describe the texture and color features of the image. However, this method performs feature extraction in the RGB color space and cannot fully extract the color and texture information of the image. Summary of the invention

[0007] In view of the problems existing in the prior art, the present invention proposes an unmanned vehicle ORB visual multi-feature fusion matching method based on radial orthogonal mean. A scale pyramid is established for the grayscale image, and the radial orthogonal mean FAST algorithm is used to extract feature points. Then, the local color histogram descriptor is fused with the rBRIEF descriptor to improve the robustness and accuracy of feature point matching.

[0008] In order to achieve the above purpose, the present invention adopts the following specific technical solutions to solve the problem:

[0009] 1. A method for fusion matching of multiple features of ORB vision for unmanned vehicles based on radial orthogonal mean, characterized by comprising the following steps:

[0010] S1: Convert the input visual RGB image into a grayscale image and establish a feature scale pyramid. The number of layers L of the pyramid is determined by the image size and the scale of the feature points described. The calculation formula is as follows:

[0011] L = [log k (min(w,h))-log k (patch_size·4)] (1)

[0012] Where k is the downsampling coefficient, w and h are the width and height of the image, respectively, and patch_size is the scale of the feature point description;

[0013] S2: Calculate the number of feature points to be extracted in each layer of the image, and use the ROM-FAST algorithm to extract feature points from the multi-scale image in S1, including the following sub-steps:

[0014] S2.1: Calculate the number of feature points to be extracted in each layer of the image based on the number of layers L of the pyramid. The number of feature points extracted in each layer It should be adjusted according to the downsampling factor k and satisfy the following equation:

[0015]

[0016] Where N f is the total number of feature points to be extracted;

[0017] S2.2: For each pixel in the image, extract a 7×7 region around the pixel p as the center, then decompose the region into two groups of orthogonal patches, calculate the average of the radial pixels at all orthogonal positions, and after generating the average, fill each 3×3 block with four orthogonal pixels, and finally fill the orthogonal pixels around the center pixel;

[0018] S2.3: In order to avoid the fixed threshold affecting the feature point extraction when the ambient light and contrast change, a method of dynamically calculating the threshold T is adopted. The calculation formula is shown as follows:

[0019]

[0020] Where σ is the adaptive parameter, I max is the maximum pixel value on the circle, I min is the minimum pixel on the circle, I aver It is the average value after removing the maximum pixel value and the minimum pixel value;

[0021] S2.4: For pixel p, there are 8 adjacent pixels on a circle with one pixel as radius, which are marked from integers 1 to 8 in a clockwise direction. Then the central pixel is compared with the mean of all orthogonal pixels, and the difference between it and the surrounding 8 pixels is calculated. If there are N consecutive points whose pixel difference is greater than or equal to or less than or equal to the threshold T, it is judged as a feature point;

[0022] S2.5: The number of feature points per layer extracted based on the ROM-FAST algorithm with a predetermined number of feature points for that layer For comparison, if Select Top The best point if According to the remaining layers Allocate the missing feature points proportionally;

[0023] S3: Convert the image into multiple separate feature channels, then process each channel independently, and finally concatenate the outputs of all channels at each feature point to form the final local color histogram descriptor, which specifically includes the following sub-steps:

[0024] S3.1: convert the collected RGB image into HSV color space and quantize it;

[0025] S3.2: Take each feature point in the quantized image as the center of the circular neighborhood, and count the number of occurrences of pixels with different intensity values ​​in the circular neighborhood. The calculation formula is as follows:

[0026]

[0027] In the formula, h(x c ,y c , r) is a c ,y c ) as the center and the radius is r. It is a binary image. When a pixel (x, y) in the image belongs to the kth gray level, its value is 1;

[0028] S3.3: Repeat the steps in S3.2 with different radii centered at the feature point, and finally concatenate and normalize the outputs at different radii as the local color histogram descriptor of the feature point;

[0029]

[0030] In the formula, S represents the color space, K represents the vector dimension (grayscale), Indicates that K i The LH descriptor in the i-th channel of the quantization bin;

[0031] S4: Locate the detected feature points in the grayscale image and use the information of the grayscale image to generate the rBRIEF descriptor with rotation invariance, which specifically includes the following sub-steps:

[0032] S4.1: Use descriptors with the same patch_size to describe features of different scales on each feature layer, and extract equivalent feature scale patch_size from each feature layer r Satisfied:

[0033] patch_size r =patch_size·k i-1 (1≤i≤L) (6)

[0034] S4.2: For each feature point, compare the size relationship between two random pixels near it, and define the operation τ as:

[0035]

[0036] Where: p(x) and p(y) are the pixel values ​​of random points x and y respectively;

[0037] For the convenience of description, four points are selected and recorded as:

[0038] p1(x, y), p2(x, y), p3(x, y), p4(x, y)

[0039] Perform τ operations on randomly selected points respectively and combine the calculated results:

[0040]

[0041] The final descriptor is: 1001;

[0042] S4.3: For any feature point, in patch_size r The location in the neighborhood of (x i ,y i ) is represented by a 2×n matrix:

[0043]

[0044] Using the main direction θ of the feature point and the corresponding rotation matrix R θ , calculate S θ To represent S:

[0045]

[0046] So we get a directional descriptor:

[0047] g n (p, θ) = f n (p)|(x i ,y i )∈S θ (11)

[0048] Where: f n (p) = ∑ 1≤i≤n 2 i-1 τ(p;x i ,y i ) is the descriptor of BRIEF;

[0049] S5: Establish the FAST-SCORE graph of the underlying image in each scale range, and obtain the coordinate correction amount of the feature points extracted from the upper layer of the image pyramid to the underlying image, which specifically includes the following sub-steps:

[0050] S5.1: Using box filters y(x, y, s) of different scales i ) Convolve the original image I(x, y) to obtain each layer image I i (x, y), as shown below:

[0051]

[0052] In the formula, I(x, y) is the original image, s i is the filter size;

[0053] S5.2: Perform weighted difference calculation on the L images, and then perform difference calculation with the original image I(x, y) to obtain L approximate FAST-SCORE graphs. The approximate FAST-SCORE of each layer is calculated by the following formula:

[0054]

[0055] In the formula, δ i+1→i Represents the weighting coefficient between the i+1th layer image of the filtering layer and the ith layer image of the difference layer;

[0056] S5.3: When the coordinates of the feature points in the upper image are mapped to the lower image, the coordinates in the corresponding ROI area in the FAST-SCORE map are weighted according to the score distribution in the area, and the geometric moments under different weight coefficients χ are calculated. The specific formula is as follows:

[0057]

[0058] For a two-dimensional discrete function representing an image, the above formula can be rewritten as:

[0059]

[0060] Therefore, the centroid C of the score within the ROI area χ It can be obtained by the following formula:

[0061]

[0062] In the formula, χ is the weight coefficient indicating the correlation between the score distribution in the region and the mapping result;

[0063] S5.4: As χ changes, C represents the target position under different selection methods, which affects the mapping effect. Therefore, the momentum coefficient η is introduced to enhance the mapping effect. Therefore, the mapping correction relationship based on the approximate FAST-SCORE map is as follows:

[0064]

[0065] S6: Concatenate the rBRIEF descriptor and the local color histogram obtained above to obtain the final feature point descriptor, and use Hamming distance matching, which specifically includes the following sub-steps:

[0066] S6.1: The specific steps of using the Hamming distance to roughly match feature points include the following: The main idea of ​​the Hamming distance is to select two binary descriptors g1 and g2 from the two images to be matched, perform an XOR operation on them, and calculate the Hamming distance Hdis between g1 and g2. 12 , the formula is as follows:

[0067]

[0068] Hdis 12 With the preset threshold H th Compare, if it is greater than the preset threshold, it means the match is successful, otherwise the match fails;

[0069] S6.2: The present invention calculates the ratio of the closest absolute value distance to the second closest absolute value distance between feature vectors as the matching basis, that is, traverses and calculates the Euclidean distance between each feature point descriptor in the image to be matched and all feature descriptors in the reference image, and finds the feature points closest and second closest to a feature point. When the ratio of the closest distance value to the second closest distance value is less than a threshold, it is defined as a correct matching point pair.

[0070] The present invention has the following beneficial effects:

[0071] 1. Aiming at the problem that the traditional FAST algorithm has poor feature point extraction effect in low-light environment, the present invention proposes a FAST algorithm based on radial orthogonal mean for feature point detection. The method first calculates the number of pyramid layers according to the image size, and then dynamically calculates the number of feature points to be extracted in each layer according to the number of pyramid layers. A dynamic calculation threshold method is adopted to extract rich feature points even in dark light and obvious contrast changes. In the original method, only one feature point can be correctly matched, while in this patent, the number of feature points extracted in darker places increases by more than 200, and the number of matches increases by 17.

[0072] 2. Color and texture are important features for distinguishing images. The present invention first converts the image into multiple separate channels and quantizes them, then calculates the distribution of pixel intensities around feature points in different channels, and finally concatenates the vectors of all channels as a local color histogram descriptor. This method can apply the color and texture information of the image to the image matching process while retaining the advantages of the ORB algorithm, thereby improving the problem of the traditional ORB algorithm in lack of color and texture. At the same time, using the HSV color space instead of the RGB color space for quantization can effectively improve the impact of light changes, successfully eliminate the two mismatched points in the image, and retain all correct matching points. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0074] Figure 1 It is the overall flow chart of the scheme of the present invention;

[0075] Figure 2 It is a schematic diagram of radial orthogonal mean;

[0076] Figure 3 It is a schematic diagram of ROM-FAST;

[0077] Figure 4 It is a schematic diagram of LH descriptor;

[0078] Figure 5 The FAST-SCORE difference diagram of the 5-layer pyramid;

[0079] Figure 6 This is the data picture used in this simulation experiment;

[0080] Figure 7This is a schematic diagram showing the effect comparison of feature matching of the traditional ORB algorithm;

[0081] Figure 8 A comparison diagram of matching using the ROM-FAST algorithm;

[0082] Fig. 9 This is the effect diagram of the feature matching proposed in the present invention. DETAILED DESCRIPTION

[0083] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, an unmanned vehicle ORB visual multi-feature fusion matching method based on radial orthogonal mean is provided, and the overall structure is as follows Figure 1 As shown, the following steps are included:

[0084] 1. A method for fusion matching of multiple features of ORB vision for unmanned vehicles based on radial orthogonal mean, characterized by comprising the following steps:

[0085] S1: Convert the input visual RGB image into a grayscale image and build a feature scale pyramid for the image, which includes the following sub-steps:

[0086] S1.1: Calculate the number of layers L in the pyramid. The number of layers L in the pyramid is mainly determined by the size of the image and the size of the feature point description scale. The calculation is as follows:

[0087] L = [log k (min(w,h))-log k (patch_size·4)] (19)

[0088] Where κ is the downsampling coefficient, w and h are the width and height of the image respectively, and patch_size is the scale of the feature point description;

[0089] S1.2: Perform Gaussian filtering and downsampling on the input image, and repeat this operation until L+1 images are obtained. The sizes of the images are [(w / 2 L , h / 2 L ), …, (w, h)];

[0090] S2: Use the adaptive ROM-FAST algorithm to extract feature points from the multi-scale image in S1, including the following sub-steps:

[0091] S2.1: For each pixel in the image, features are extracted in a 7x7 region centered on the pixel. The region is decomposed into two groups of orthogonal patches. Then, an average is generated between the radial pixels at all orthogonal positions of the two patches. After the average is generated, each 3x3 block is filled with four orthogonal pixels. Then, the orthogonal pixels are filled around the central pixel, as shown in Figure 2As shown;

[0092] S2.2: For Figure 2 There are 8 adjacent feature points on the circle with a pixel point as the radius, which are marked from integers 1 to 8 in a clockwise direction, such as Figure 3 As shown;

[0093] S2.3: In order to avoid the fixed threshold affecting the feature point extraction when the ambient light and contrast change, a method of dynamically calculating the threshold T is adopted. The calculation formula is shown as follows:

[0094]

[0095] Where σ is the adaptive parameter, I max is the maximum pixel value on the circle, I min is the minimum pixel on the circle, I aver It is the average value after removing the maximum pixel value and the minimum pixel value;

[0096] S2.4: Compare the average values ​​of all orthogonal pixels with the central pixel, take the four feature points 1, 3, 5, and 7 located in the four directions of up, down, left, and right, respectively, and calculate the pixel difference between these four points and the central point P, define a threshold T, and if the pixel difference of three of the four points is greater than the threshold, the central point P is judged to be a potential feature point, otherwise it is removed;

[0097]

[0098] S2.5: Select the real feature points from the potential feature points selected in S2.3. If P is already a potential feature point, calculate the pixel difference between point P and the surrounding 8 feature points. If there are N consecutive points with pixel differences greater than or equal to or less than the threshold T, then it is judged as a real feature point, otherwise it is removed. So far, the feature points in each feature layer have been selected;

[0099]

[0100] S2.6: Use the ROM-FAST algorithm to extract feature points from each layer of the pyramid image. To ensure that the feature points are evenly distributed in each layer, the number of feature points extracted from each layer is It should be adjusted according to the downsampling factor k and satisfy the following equation:

[0101]

[0102] Where N f is the total number of feature points to be extracted;

[0103] S2.7: For each layer of the image, the number of feature points extracted based on the ROM-FAST algorithm with a predetermined number of feature points for that layer Compare; if Select Top The best point if And according to the remaining layers Allocate the missing feature points proportionally;

[0104] S3: Convert the image into multiple separate feature channels, then process each channel independently, and finally, connect the outputs of all channels at each pixel to form the final local color histogram descriptor, which specifically includes the following sub-steps:

[0105] S3.1: Convert the collected RGB image to HSV color space. HSV color space consists of three components: hue H, saturation S, and brightness V. The formula for converting color images from RGB to HSV color space is as follows:

[0106]

[0107] In the formula, I H Describing the hue, I S Depict saturation, I V Describe brightness;

[0108] S3.2: Quantify the HSV image in S3.1. The specific rules of quantization are as follows:

[0109]

[0110] S3.3: Take each feature point pixel in the given quantized image as the center of the circular neighborhood and count the number of occurrences of pixels with different intensity values ​​in the circular neighborhood. The specific formula is:

[0111]

[0112] In the formula, h(x c ,y c , r) is a c ,y c ) as the center and the radius is r. It is a binary image. When a pixel (x, y) in the image belongs to the kth gray level, its value is 1;

[0113] S3.4: Repeat the steps in S3.3 with different radii centered at the feature point. Finally, the output connections of all channels at each pixel are normalized as the final feature vector.

[0114]

[0115] In the formula, S represents the color space, K represents the vector dimension (grayscale), Indicates that K i The local color histogram descriptor in the i-th channel of the quantization bin;

[0116] S4: Locate the detected feature points in the grayscale image and use the information of the grayscale image to generate the rBRIEF descriptor with rotation invariance, which specifically includes the following sub-steps:

[0117] S4.1: Use descriptors with the same patch_size to describe features of different scales on each feature layer, and extract equivalent feature scale patch_size from each feature layer r Satisfied:

[0118] patch_size r =patch_size·k i-1 (1≤i≤L) (28)

[0119] S4.2: For each feature point, compare the size relationship between two random pixels near it, and define the operation τ as:

[0120]

[0121] Where: p(x) and p(y) are the pixel values ​​of random points x and y respectively;

[0122] For the convenience of description, four points are selected and recorded as:

[0123] p1(x, y), p2(x, y), p3(x, y), p4(x, y),

[0124] Perform τ operations on randomly selected points respectively and combine the calculated results:

[0125]

[0126] The final descriptor is: 1001;

[0127] S4.3: For any feature point, in patch_size r The location in the neighborhood of (x i ,y i ) is represented by a 2×n matrix:

[0128]

[0129] Using the main direction θ of the feature point and the corresponding rotation matrix Rθ , calculate S θ To represent S:

[0130]

[0131] So we get a directional descriptor:

[0132] g n (p, θ) = f n (p)|(x i ,y i )∈S θ (33)

[0133] Where: f n (p) = ∑ 1≤i≤n 2 i-1 τ(p;x i ,y i ) is the descriptor of BRIEF;

[0134] S5: Obtain the coordinate correction amount of the feature points extracted from the upper layer of the image pyramid to the underlying image, and establish the FAST-SCORE graph of the underlying image in various scale ranges, which specifically includes the following sub-steps:

[0135] S5.1: Using box filters of different scales f(x, y, s i ) and perform convolution of different scales on the original image I(x, y) according to the number of layers of the previous feature scale pyramid to obtain the images I i (x, y), as shown below:

[0136]

[0137] S5.2: In order to efficiently obtain FAST-SCORE images of different scales, weighted difference calculation is performed on L images, and then difference calculation is performed with the original image I(x, y) to obtain L approximate FAST-SCORE images. The approximate FAST-SCORE of each layer is calculated by the following formula:

[0138]

[0139] In the formula, δ i+1→i Represents the weighting coefficient between the i+1th layer image of the filtering layer and the ith layer image of the difference layer;

[0140] S5.3: When the coordinates of the feature points in the upper image are mapped to the lower image, the coordinates in the corresponding ROI area in the FAST-SCORE map are weighted according to the score distribution in the area, and the geometric moments under different weight coefficients χ are calculated. The specific formula is as follows:

[0141]

[0142] For a two-dimensional discrete function representing an image, the above formula can be rewritten as:

[0143]

[0144] Therefore, the centroid C of the score within the ROI area χ It can be obtained by the following formula:

[0145]

[0146] Wherein, the weight coefficient χ represents the correlation between the score distribution in the region and the mapping result.

[0147] S5.4: As χ changes, C represents the target position under different selection methods, which affects the mapping effect. Therefore, the momentum coefficient η is introduced to enhance the mapping effect. Therefore, the mapping correction relationship based on the approximate FAST-SCORE map is as follows:

[0148]

[0149] S6: The rBRIEF descriptor and the local color histogram obtained above are concatenated to obtain the final feature point descriptor, and a rough match is performed using the Hamming distance and brute force matching, which specifically includes the following sub-steps:

[0150] S6.1: The specific steps of using the Hamming distance to roughly match feature points include the following: The main idea of ​​the Hamming distance is to select two binary descriptors g1 and g2 from the two images to be matched, perform an XOR operation on them, and calculate the Hamming distance Hdis between g1 and g2. 12 , the formula is as follows:

[0151]

[0152] Hdis 12 With the preset threshold H th Compare, if it is greater than the preset threshold, it means the match is successful, otherwise the match fails;

[0153] S6.2: The present invention calculates the ratio of the closest absolute distance to the second closest absolute distance between feature vectors as the matching basis, that is, traverses and calculates the Euclidean distance between each feature point descriptor in the image to be matched and all feature descriptors in the reference image, and finds the feature points that are closest and second closest to a certain feature point. When the ratio of the closest distance value to the second closest distance value is less than a threshold, it is defined as a correct matching point pair.

[0154] The algorithm is numerically simulated, including the effect verification of adaptive ROM-FAST and the effect verification of improved ORB algorithm. The specific steps are as follows:

[0155] The simulation experiment software environment is ubuntu18.04+vscode+python, and the hardware environment is AMD Ryzen5 3600+NVIDAGeForce RTX 1060+8GB memory. The feature point detection algorithm parameter comparison table is shown in Table 1, the feature point matching parameter comparison table is shown in Table 2, and some detection results are shown in Table 3.

[0156] Table 1 Feature point detection simulation parameter settings

[0157]

[0158] Table 2 Feature matching simulation parameter settings

[0159]

[0160] Table 3 Test results

[0161]

[0162] In this simulation experiment, the Leuven images in the Mikolajczyk dataset are used for verification under different lighting conditions. Figure 6 As shown, Figure 6 -(Left) is a schematic diagram under normal lighting. Figure 6 -(right) is a schematic diagram under dark light, and the height and width of the image are 298 and 448 respectively. At the same time, the improvement effect of introducing the ROM-FAST method and the local color histogram descriptor on the traditional ORB algorithm is verified by comparative experiments.

[0163] Figure 7 and Figure 8 The results of feature point detection using the traditional ORB algorithm and the ROM-FAST method are shown. Figure 7 It shows that the number of feature points detected by the traditional ORB algorithm is small and concentrated on the edges and obvious textures. The detection effect is poor under dark lighting conditions and there is only one correct matching point. However, Figure 8 As shown in the figure, after adopting the ROM-FAST algorithm, the number of feature points and the number of correct matches have been significantly improved. The number of extractions in darker areas has increased by more than 200, and the number of correct matches has increased by 14. Figure 8 and Fig. 9 The matching effect of feature point detection using the ROM-FAST method is demonstrated, where: Fig. 9 The local color histogram descriptor is introduced. Figure 8Although the number of matches is Fig. 9 has 5 more matches, but there are obvious mismatches; Fig. 9 After introducing the local color histogram descriptor, the two mismatched points were successfully eliminated.

[0164] The specific implementation scheme described above further illustrates the purpose of the invention and the technical solution of the present invention. The above embodiments are only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Those skilled in the art should understand that any modification or equivalent substitution of the technical solution of the present invention is included in the scope of protection of the present invention.

Claims

1. A multi-feature fusion matching method for unmanned vehicle ORB vision based on radial orthogonal mean, characterized in that: The following steps are involved: S1: Convert the input visual RGB image into a grayscale image and establish a feature scale pyramid. The number of layers L of the pyramid is determined by the image size and the scale of the feature points described. The calculation formula is as follows: L=[log k (min(ω,h))-log k (patch_size·4)] (1) Where k is the downsampling coefficient, w and h are the width and height of the image, respectively, and patch_size is the scale of the feature point description; S2: Calculate the number of feature points to be extracted in each layer of the image, and use the adaptive ROM-FAST algorithm to extract feature points from the multi-scale image in S1, including the following sub-steps: S2.1: Calculate the number of feature points to be extracted in each layer of the image based on the number of layers L of the pyramid. The number of feature points extracted in each layer It should be adjusted according to the downsampling factor k and satisfy the following equation: Where N f is the total number of feature points to be extracted; S2.2: For each pixel in the image, extract a 7×7 region around the pixel p as the center, then decompose the region into two groups of orthogonal patches, calculate the average of the radial pixels at all orthogonal positions, and after generating the average, fill each 3×3 block with four orthogonal pixels, and finally fill the orthogonal pixels around the center pixel; S2.3: In order to avoid the fixed threshold affecting the feature point extraction when the ambient light and contrast change, a method of dynamically calculating the threshold T is adopted. The calculation formula is shown as follows: Where σ is the adaptive parameter, I max is the maximum pixel value on the circle, I min is the minimum pixel on the circle, I aver It is the average value after removing the maximum pixel value and the minimum pixel value; S2.:4: For pixel p, there are 8 adjacent pixels on a circle with one pixel as radius, which are marked from integers 1 to 8 in a clockwise direction. Then the central pixel is compared with the mean of all orthogonal pixels, and the difference between it and the surrounding 8 pixels is calculated. If there are N consecutive points whose pixel difference is greater than or equal to or less than or equal to the threshold T, it is judged as a feature point; S2.5: The number of feature points per layer extracted based on the adaptive ROM-FAST algorithm with a predetermined number of feature points for that layer For comparison, if Select Top The best point if According to the remaining layers Allocate the missing feature points proportionally; S3: Convert the image into multiple separate feature channels, then process each channel independently, and finally concatenate the outputs of all channels at each feature point to form the final local color histogram descriptor, which specifically includes the following sub-steps: S3.1: convert the collected RGB image into HSV color space and quantize it; S3.2: Take each feature point in the quantized image as the center of the circular neighborhood, and count the number of occurrences of pixels with different intensity values ​​in the circular neighborhood. The calculation formula is as follows: In the formula, h(x c ,y c , r) is a c ,y c ) as the center and the radius is r. It is a binary image. When a pixel (x, y) in the image belongs to the kth gray level, its value is 1; S:3.3: Repeat the steps in S3.2 with different radii centered at the feature point, and finally concatenate and normalize the outputs at different radii as the local color histogram descriptor of the feature point; In the formula, S represents the color space, K represents the vector dimension (grayscale), Indicates that K i The local color histogram descriptor in the i-th channel of the quantization bin; S4: Locate the detected feature points in the grayscale image and use the information of the grayscale image to generate the rBRIEF descriptor with rotation invariance, which specifically includes the following sub-steps: S4.1: Use descriptors with the same patch_size to describe features of different scales on each feature layer, and extract equivalent feature scale patch_size from each feature layer r Satisfied: patch_size r =patch_size·k i-1 (1≤i≤L) (6) S4.2: For each feature point, compare the size relationship between two random pixels near it, and define the operation τ as: Where: p(x) and p(y) are the pixel values ​​of random points x and y respectively; For the convenience of description, four points are selected and recorded as: p1(x,y), p2(x,y), p3(x,y), p4(x,y) Perform τ operations on randomly selected points respectively and combine the calculated results: The final descriptor is: 1001; S4.3: For any feature point, in patch_size r The location in the neighborhood of (x i ,y i ) is represented by a 2×n matrix: Using the main direction θ of the feature point and the corresponding rotation matrix R θ , calculate S θ To represent S: So we get a directional descriptor: g n (p,θ)=f n (p)|(x i ,y i )∈S θ (11) Where: f n (p) = ∑ 1≤i≤n 2 i-1 τ(p;x i ,y i ) is the descriptor of BRIEF; S5: Establish the FAST-SCORE graph of the underlying image in each scale range, and obtain the coordinate correction amount of the feature points extracted from the upper layer of the image pyramid to the underlying image, which specifically includes the following sub-steps: S5.1: Using box filters of different scales f(x, y, s i ) Convolve the original image I(x, y) to obtain each layer image I i (x, y), as shown below: In the formula, I(x,y) is the original image, s i is the filter size; S5.2: Perform weighted difference calculation on the L images, and then perform difference calculation with the original image I(x, y) to obtain L approximate FAST-SCORE graphs. The approximate FAST-SCORE of each layer is calculated by the following formula: In the formula, δ i+1→i Represents the weighting coefficient between the i+1th layer image of the filtering layer and the ith layer image of the difference layer; S5.3: When the coordinates of the feature points in the upper image are mapped to the lower image, the coordinates in the ROI region of the corresponding feature point in the FAST-SCORE map are weighted according to the score distribution in the region, and the geometric moments under different weight coefficients χ are calculated. The specific formula is as follows: For a two-dimensional discrete function representing an image, the above formula can be rewritten as: Therefore, the centroid Cχ of the score within the ROI area can be obtained by the following formula: In the formula, χ is the weight coefficient indicating the correlation between the score distribution in the region and the mapping result; S5.4: As χ changes, C represents the target position under different selection methods, which affects the mapping effect. Therefore, the momentum coefficient η is introduced to enhance the mapping effect. Therefore, the mapping correction relationship based on the approximate FAST-SCORE map is as follows: S6: Concatenate the rBRIEF descriptor and the local color histogram obtained above to obtain the final feature point descriptor, and use Hamming distance matching, which specifically includes the following sub-steps: S6.1: The specific steps of using the Hamming distance to roughly match feature points include the following: The main idea of ​​the Hamming distance is to select two binary descriptors g1 and g2 from the two images to be matched, perform an XOR operation on them, and calculate the Hamming distance Hdis between g1 and g2. 12 , the formula is as follows: Hdis 12 With the preset threshold H th Compare, if it is greater than the preset threshold, it means the match is successful, otherwise the match fails; S6.2: The present invention calculates the ratio of the closest absolute distance to the second closest absolute distance between feature vectors as the matching basis, that is, traverses and calculates the Euclidean distance between each feature point descriptor in the image to be matched and all feature descriptors in the reference image, and finds the feature points that are closest and second closest to a certain feature point. When the ratio of the closest distance value to the second closest distance value is less than a threshold, it is defined as a correct matching point pair.

Images