An rgb feature matching method based on information entropy

Abstract

The application provides an RGB feature matching method based on information entropy, comprising: 1) image acquisition and image gray pretreatment based on an RGB feature extraction algorithm; 2) pretreated image is used to collect feature information in the image by using the RGB feature extraction algorithm based on information entropy, and a feature point descriptor is generated; 3) feature coarse matching is carried out by using Hamming distance; 4) feature matching is carried out by combining the RGB feature descriptor and the PROSAC algorithm; and 5) image feature matching points and matching relations are outputted. The RGB feature matching method based on the application can select a suitable weight in the image RGB feature gray process, thereby retaining the original feature information of the image to the greatest extent, avoiding error feature information, removing the key points prone to be extracted in advance in the image feature collection stage by means of the RGB feature matching advantage, and ensuring the matching precision when the light and the view angle change greatly, so that the precision and the robustness of the system feature recognition are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to an RGB feature matching method based on information entropy. BACKGROUND

[0002] The ORB feature matching method is an algorithm for screening and matching features, mainly including four stages of image feature acquisition, image feature detection, feature descriptor generation, and feature matching, and the main task is to extract ORB feature points in the visual odometer and calculate the feature descriptors, and then the motion between the adjacent two frames of images is calculated through the matching of the corresponding feature points. The ORB feature matching algorithm adopts the FAST and BRIEF algorithms for key point detection and feature description, has high calculation speed, and considers the influence of image rotation and scale transformation, so that the algorithm has strong robustness. However, the ORB feature matching algorithm is prone to feature matching failure, feature mismatch and other problems under strong light, strong shaking and large scale change. Although there are improved versions, the problems of feature matching failure and the like have not been effectively solved. SUMMARY

[0003] The present application aims at the deficiencies of the prior art, and provides an RGB feature matching method based on information entropy, which mainly introduces information entropy into the feature extraction stage, uses information entropy-based gray-scale processing on the RGB image, and then uses the PROSAC algorithm to remove feature mismatch, thereby increasing the feature recognition accuracy and robustness of the system.

[0004] To achieve the above object, the present application adopts the following technical scheme:

[0005] An RGB feature matching method based on information entropy comprises the following steps:

[0006] Step S1. Image acquisition and RGB feature extraction algorithm-based image gray preprocessing: the pre-processing of the input RGB image includes using gray-scale simplification to simplify subsequent processing, using Gaussian filtering to reduce image noise, and using image gradient calculation to obtain image local features;

[0007] Step S2. The pre-processed image uses an RGB feature extraction algorithm based on information entropy to collect feature information in the image, and generates a feature point descriptor;

[0008] Step S3. Using Hamming distance for feature coarse matching;

[0009] Step S4. Combining the RGB feature descriptor and the PROSAC algorithm for feature matching;

[0010] Step S5. Outputting the image feature matching points and the matching relationship.

[0011] Specifically, the step S1 includes the following steps of pre-processing the image gray based on the RGB feature extraction algorithm:

[0012] Step S11. First, pre-process the input RGB image, including using gray scale to simplify subsequent processing, Gaussian filtering to reduce image noise, and image gradient calculation to obtain image local features;

[0013] Step S12. Feature point candidate extraction: use the FAST algorithm to extract candidate feature points, and through the calculation and analysis of the horizontal and vertical directions of the image gradient, extract the feature point candidates for each pixel point in the image;

[0014] Step S13. Feature point candidate screening: screen the feature points through non-maximum suppression (NMS), and for each pixel point, compare the gradient size with the surrounding pixel points, and keep the point with the largest gradient as the feature point;

[0015] Step S14. Adaptive strategy is used to optimize the candidate feature points: according to the variance and mean of the gray values of the pixel points around the feature points, the threshold of the feature points is adaptively adjusted, and the feature points with representative and discriminative are reserved, then the gradient direction histogram of the pixel points around the reserved feature points is established, the peak position is found, and if the peak information of the detected feature points is different from the surrounding points, the point is determined as a false detection point.

[0016] Specifically, the step S2 includes the following steps of generating the feature point descriptor:

[0017] Step S21: express the image feature information at this time as an MxNx3 array, and arrange the RGB three channels from top to bottom, then randomly select a pair of points in the neighborhood of any point in the layer, perform random sampling, and then compare the pixel size of the up-sampled points of p i =(x i ,y i ) and p′ i =(x′ i ,y′ i ), and assign 0 or 1 according to the pixel value size;

[0018] Step S22: repeat step S21 to generate a binary code, i.e., generate a feature point descriptor.

[0019] Specifically, the step S3 includes the following contents of using the Hamming distance to perform feature coarse matching:

[0020] The image performing feature matching outputs corresponding equal-length binary feature descriptors according to an RGB feature extraction algorithm. Assuming that the strings of the two are a and b, the Hamming distance is d(a, b), which is calculated using a Hamming distance calculation formula. The smaller the Hamming distance, the more similar the two feature points or feature vectors, and the greater the matching possibility.

[0021] Specifically, the combination of the RGB feature descriptor and the PROSAC algorithm for feature matching in step S4 includes feature mismatch point elimination and feature fine matching.

[0022] PROSAC arranges a pair of matching points according to the Hamming distance similarity, sorts, samples, and calculates a pair of matching target models H in the two images, and finally verifies the remaining matching points through the target model to remove the mismatched points. If the number of matching point pairs is set to N, the Hamming distance similarity becomes higher, and the quality of the matching point pairs becomes higher. The matching point pair set in descending order of Hamming distance similarity is defined as a set sequence M consisting of n groups of high-quality matching point pairs. The candidate model obtained by sampling calculation is F, and the inner point error threshold W, the maximum number of inner points N, and the maximum number of iterations K are set. The specific steps of the PROSAC algorithm are as follows:

[0023] Step S41. Arrange the Hamming distance similarity matching point pairs in descending order to obtain a matching pair set UN.

[0024] Step S42. Select n pairs of high-quality matching point pairs in UN to form a matching pair subset M.

[0025] Step S43. Solve the candidate model F through the subset M, and set (x, y, 1) and (x', y', 1) as the homogeneous coordinates of a pair of correctly coinciding points a(x, y) and a'(x', y') in the stereo image pair, and substitute the M groups of matching point pairs into the equation to obtain the f matrix parameters.

[0026] Step S44: Detect the remaining matching point pairs through the candidate model f and the error threshold W, and obtain the matching point pairs that satisfy the candidate model f and the number of matching point pairs that satisfy the candidate model f.

[0027] Step S45. Determine whether the iteration process is terminated based on the end condition, and repeat the process steps S42-S45; the end condition: if the internal score exceeds the set maximum number of inner points, or the number of inner points after K times sampling does not increase compared with the number of inner points after K-1 times sampling.

[0028] Specifically, the output of the image feature matching points and the matching relationship in step S5 is represented by a matrix, where the element in the ith row and jth column represents the matching relationship between the ith feature point in the first image and the jth feature point in the second image.

[0029] Compared with the prior art, the present application has the beneficial effects that:

[0030] The present application is based on the RGB feature matching method, which selects appropriate weights in the image RGB feature gray processing to maximize the preservation of the original feature information of the image, while avoiding false feature information. In the image feature acquisition stage, the RGB feature matching advantage can be used to exclude the key points that are easy to be misextracted in advance. The matching accuracy is ensured when the light and the viewing angle change greatly, and the precision and robustness of the system feature recognition are effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is the flowchart of the RGB feature matching method based on information entropy of the present application;

[0032] Figure 2 is the original image of the experiment in the embodiment of the present application;

[0033] Figure 3 is the matching situation of the algorithm under different conditions in the embodiment of the present application;

[0034] Figure 4 is the predicted trajectory obtained based on the RGB feature matching method in the embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be described clearly and completely below in combination with the drawings of the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0036] As shown in Figure 1 , the present application provides a RGB feature matching method based on information entropy, and the specific implementation of each part includes the following steps:

[0037] Step 1. Image acquisition and image gray processing

[0038] Step 1.1, select the scene of image acquisition, respectively acquire images of the same scene under insufficient light and normal light, and then acquire images under different scenes using insufficient light, as shown in Figure 2 , the selected image in the present application is a corner of a semi-open scene;

[0039] Step 1.2, set up the camera position, align the camera to the same scene, and then move the scene to acquire images;

[0040] Step 1.3, according to the sequence, respectively, on the same scene under the insufficient light and normal light collected images and the same scene moving collected images are grayed, using the method based on RGB color vector to realize the three primary color gray information entropy formula:

[0041]

[0042] Then configure the weight

[0043]

[0044] Finally, using the gray scale conversion formula of three primary color images:

[0045] f GREY =w r f r +w g f g +w b f b (3)

[0046] Gray processing.

[0047] Step 1.4, in addition to the light change and moving scene change comparison, the image feature matching under the scale change, translation and rotation processing is carried out, and the processed image is filed to facilitate the subsequent feature extraction and feature matching work.

[0048] Step 2, using the preprocessed image using information entropy based RGB feature extraction algorithm to collect image features (key points, edges, texture features, etc.), the specific steps are:

[0049] Step 2.1, according to the image feature information at this time can be expressed as MxNx3 array, RGB three channels from top to bottom arrangement:

[0050]

[0051] In the formula: M, N represent the maximum value in the plane horizontal and vertical coordinate system, r MN , g MN , b MN respectively represent the corresponding pixel point information in three layers.

[0052] Take SxSx3 solid neighborhood with g MN Some two-dimensional feature point (x, y) as the center, then randomly select a pair of points in the neighborhood. Random sampling rules are as follows: uniform random sampling is adopted on R, G, B three layers; Gaussian two-step random sampling is adopted for pixels on the layer, that is, first sample x i According to the Gaussian distribution Sampling, then yi According to Gaussian distribution Sampling, and x i and y i are independent, that is, the horizontal and vertical coordinates of the sampling points have no correlation. Then the pixel size of the sampling point on p i = (x i , y i ) is compared with the pixel size of the sampling point on p' i = (x' i , y' i ), and the following assignments are made.

[0053]

[0054] Step 2.2, repeat step 2.1 to generate a binary code, that is, to generate a feature point descriptor.

[0055] Step 3, perform feature coarse matching using the Hamming distance, as follows:

[0056] The images for which feature matching is performed output corresponding equal-length binary feature descriptors according to the RGB feature extraction algorithm. Assuming that the strings of the two are a and b, the Hamming distance is d(a, b), and the Hamming distance calculation formula is:

[0057]

[0058] In the formula, n represents the length of the string, and f(x, y) is a function for determining whether two characters are the same. When a[i] and b[i] are the same, the value of f(a[i], b[i]) is 0, otherwise it is 1.

[0059] The smaller the Hamming distance calculated is, the more similar the two feature points or feature vectors are, and the greater the possibility of matching is.

[0060] Step 4, combine the RGB feature extraction algorithm and the PROSAC algorithm to perform feature false matching elimination and feature fine matching;

[0061] PROSAC arranges a pair of matching points according to the Hamming distance similarity, sorts, samples and calculates a pair of matching target models H in two images, and finally verifies the remaining matching points through the target model to remove the unmatched points. If the number of matching point pairs is set to N, the Hamming distance similarity is higher, and the quality of the matching point pairs is higher. The Hamming distance similarity is used as a quantitative representation of the matching quality. The matching point pair set in descending order of Hamming distance similarity is defined as a set sequence M consisting of n groups of high-quality matching point pairs. The candidate model obtained by sampling and calculation is F, and the inner point error threshold W, the maximum number of inner points N, and the maximum number of iterations K are set. The specific steps of the algorithm are as follows:

[0062] Step 4.1, rank the matching point pairs in descending order of Hamming distance similarity to obtain a matching pair set UN.

[0063] Step 4.2, select n pairs of high-quality matching point pairs in UN to constitute a matching pair subset M.

[0064] Step 4.3, solve the candidate model F from M, and solve the candidate model F,

[0065]

[0066] (x, y, 1) and (x', y', 1) are set as the homogeneous coordinates of a pair of correctly coinciding points a(x, y) and a'(x', y') in the stereo image pair, and the M matching point pairs are substituted into the equation to obtain the f matrix parameters.

[0067] Step 4.4, detect the remaining matching point pairs by the candidate model f and the error threshold W, and obtain the matching point pairs satisfying the candidate model f and the number of matching point pairs satisfying the candidate model f (inliers d).

[0068] Step 4.5, judge whether the iteration process is terminated based on the end condition, and repeat the process steps 4.2-4.5.

[0069] End condition: if the internal score exceeds the set maximum number of inliers, or the number of inliers after K sampling does not increase compared with the number of inliers after K-1 sampling.

[0070] Step 5, output the image feature matching points and the matching relationship, which is represented by a matrix, wherein the element in the ith row and jth column represents the matching relationship between the ith feature point in the first image and the jth feature point in the second image.

[0071] The improvement effect of the method of the application is described below through a comparative test.

[0072] First, according to the RGB feature extraction and matching algorithm based on information entropy and the ORB-SLAM2 algorithm, the trajectory generation under the influence of factors such as large-scale changes and camera jitter is compared and verified on the TUM sub-datasets fr1_xyz, fr1_floor, fr2_pioneer_360 and fr2_pioneer_slam, and the specific trajectory graph is as shown in Figure 4 Then, the root mean square error of the predicted trajectory generated by the dataset fr1_xyz and fr1_room is compared with the real trajectory:

[0073]

[0074] In the above formula: d i represents and the nearest point Euclidean distance between the two points, l i denotes the shortest distance between the trajectory X b , the shortest distance between the trajectory X

[0075] Table 1 Relative trajectory RMSE comparison

[0076]

[0077] The comparative test verification shows that the feature matching algorithm based on the RGB information entropy has the relative trajectory displacement error improvement ratio of more than 10% in the fr1_xyz data set, and the relative trajectory rotation error improvement ratio is also more than 10%; the relative trajectory displacement error improvement ratio and the relative trajectory rotation error improvement ratio are also more than 20% in the fr1_room data set.

[0078] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, but not to limit it; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An RGB feature matching method based on information entropy, characterized in that, Includes the following steps: Step S1. Image acquisition and grayscale preprocessing based on RGB feature extraction algorithm: The input RGB image is preprocessed, including using grayscale to simplify subsequent processing, Gaussian filtering to reduce image noise, and image gradient calculation to obtain local image features; Step S2. The preprocessed image is used to collect feature information within the image using an RGB feature extraction algorithm based on information entropy, and feature point descriptors are generated; Step S21. Represent the image feature information at this time as follows: The array is arranged with RGB channels from top to bottom. Then, a pair of points is randomly selected within the neighborhood of any point in the layer, and random sampling is performed. Finally, the two pairs of points are compared. and The pixel size of the upsampling point is assigned a value of 0 or 1 based on the pixel values ​​of the two samples. Step S22. Repeat step S21 to generate binary encoding, i.e., generate feature point descriptors; Step S3. Perform coarse feature matching using Hamming distance; Step S4. Combine RGB feature descriptors with the PROSAC algorithm for feature matching, including the removal of feature mismatch points and fine feature matching; PROSAC assigns a pair of matching points based on Hamming distance similarity. It sorts, samples, and calculates a pair of matching target models H in two images, and finally verifies the remaining matching points using the target model, thus removing mismatched points. If the number of matching point pairs is set to N, the Hamming distance similarity increases, and the quality of the matching point pairs improves. The set of matching point pairs in descending order of Hamming distance similarity is defined as a sequence M consisting of n high-quality matching point pairs. The candidate model obtained through sampling calculation is F. The inlier error threshold W, the maximum number of inliers N, and the maximum number of iterations K are set. The specific steps of the PROSAC algorithm are as follows: Step S41. Sort the Hamming distance similarity matching point pairs in descending order to obtain the matching pair set UN; Step S42. Select n pairs of high-quality matching points in UN to form a subset M of matching pairs; Step S43. Solve for the candidate model F using subset M. and Set as a pair of correctly coincident points in a stereo image pair and The homogeneous coordinates of M matching point pairs are substituted into the equation to obtain the parameters of the f matrix. Step S44: Detect the remaining matching point pairs using the candidate model f and the error threshold W, and obtain the matching point pairs that satisfy the candidate model f and the number of matching point pairs that satisfy the candidate model f; Step S45. Determine whether the iteration process terminates based on the termination condition, and repeat steps S42-S45; Termination condition: If the internal score exceeds the set maximum number of internal points, or the number of internal points after K samplings has not increased compared to the number of internal points after K-1 samplings; Step S5. Output image feature matching points and matching relationships.

2. The RGB feature matching method based on information entropy according to claim 1, characterized in that, Step S1 involves preprocessing the image grayscale based on the RGB feature extraction algorithm, including the following steps: Step S11. First, preprocess the input RGB image, including using grayscale to simplify subsequent processing, Gaussian filtering to reduce image noise, and image gradient calculation to obtain local image features; Step S12. Feature point candidate extraction: The FAST algorithm is used to extract candidate feature points. By calculating and analyzing the horizontal and vertical gradients of the image, candidate feature points are extracted for each pixel in the image. Step S13. Feature point candidate screening: Feature points are screened using non-maximum suppression (NMS). Specifically, for each pixel, its gradient magnitude is compared with that of its surrounding pixels, and the point with the largest gradient is retained as the feature point. Step S14. Optimize candidate feature points using an adaptive strategy: Based on the variance and average of the gray values ​​of the pixels around the feature point, adaptively adjust the feature point threshold to retain representative and discriminative feature points. Then, establish a gradient direction histogram for the pixels around the retained feature points to find the peak position. If the peak information of the detected feature point is different from that of the surrounding points, then the point is determined to be a false detection point.

3. The RGB feature matching method based on information entropy according to claim 1, characterized in that, Step S3, which involves using Hamming distance for coarse feature matching, includes the following: The images undergoing feature matching output corresponding equal-length binary feature descriptors based on the RGB feature extraction algorithm. Assuming the strings for the two descriptors are 'a' and 'b' respectively, the Hamming distance is... The Hamming distance is calculated using the Hamming distance formula; the smaller the Hamming distance, the more similar the two feature points or feature vectors are, and the greater the probability of a match.

4. The RGB feature matching method based on information entropy according to claim 1, characterized in that, The output image feature matching points and matching relationships mentioned in step S5 are represented by a matrix, where the first... i Line 1 j The elements of the column represent the first image. i The first feature point is similar to the first feature point in the second image. j Matching relationships between feature points.

Images