Method for determining main direction of FAST key points in image based on RILBP operator
Through the image FAST key point main direction determination method based on RILBP operator, the problems of poor real-time performance and low image registration accuracy in the visual positioning algorithm are solved, and faster calculation speed and better High positioning accuracy.
Patent Information
- Application Number
- CN202210420116.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-20
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-04-20
AI Technical Summary
In the prior art, when the camera rotates violently, the main direction determination method for FAST key points has problems such as poor real-time performance and low image registration accuracy, mainly due to the excessive number of pixels involved in the operation and the lack of significant main direction characteristics. Reduces positioning accuracy.
The main direction determination method for image FAST key points based on RILBP operator is used, and the key points are extracted through boundary mirror expansion and Gaussian smoothing processing, and the RILBP feature value of each pixel is calculated using RILBP operator to record the first occurrence Eigenvalues are numbered to determine the main direction, reduce the number of pixels involved in the operation and improve the uniqueness of the main direction.
It improves the real-time performance of the visual positioning algorithm and image registration accuracy, ensures that the positioning accuracy and registration accuracy are maintained when the image rotates, reduces the calculation time and eliminates the ambiguity of the main direction information.
Smart Images

Figure CN114913227B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of visual positioning, and relates to simultaneous positioning and mapping SLAM, and specifically is a method for determining the main direction of FAST key points of an image based on a RILBP operator, which is used for visual positioning. Background Art
[0002] In the visual positioning algorithm using the camera as the data acquisition sensor, the FAST key point of the acceleration segment detection feature has been widely used due to its fast detection speed and strong real-time performance. However, in order to ensure that the visual positioning algorithm can maintain a good positioning state when the camera rotates violently, it is necessary to add a main direction to the FAST key point of the acceleration segment detection feature to make it rotation invariant.
[0003] At present, there are several existing methods for determining the main direction of key points: First, the grayscale centroid method is used, but the number of pixels involved in the calculation is too large, which slows down the operation of the visual positioning algorithm and weakens the real-time performance. Second, the gradient of the neighborhood of the key point is calculated, but the main direction feature is not significant, which is easy to cause mismatching. Third, the Haar wavelet feature method of the neighborhood of the key point is calculated, but it is too dependent on the gradient direction of the pixels in the local area of the image, resulting in inaccurate main direction, which in turn affects the positioning accuracy of the visual positioning algorithm.
[0004] The first method can determine the main direction of the key point. For example, the patent application with application publication number CN112907580A and titled "An image feature extraction and matching algorithm for comprehensive point and line features in weak texture scenes" discloses a method for determining the main direction of feature points using the grayscale centroid method. This method uses the feature point as the geometric center, and determines the centroid of the circular neighborhood by calculating the zero-order moment and first-order moment of all pixels in the circular neighborhood of the feature point. Then, the geometric center and the centroid are connected to obtain a direction vector, and the direction of the direction vector is used as the main direction of the feature point. However, this method requires all pixels in the circular neighborhood to participate in the calculation, resulting in a large amount of calculation and slow calculation speed.
[0005] The second method can determine the main direction of the key point. For example, the patent application with the application publication number CN114119686A and the name "Multi-source remote sensing image registration method for spatial layout similarity calculation" discloses a method for determining the main direction of a feature point. This method draws on the idea of SIFT feature points of scale-invariant feature transformation, and uses the Gaussian weighted histogram method to count the maximum value of the pixel grayscale gradient in the 360-degree angular direction of the feature point, and takes the maximum value as the main direction. However, this method may lead to the existence of multiple maximum values, or the maximum value and the second maximum value are very different, making the main direction feature not significant enough, and ultimately affecting the accuracy of image registration.
[0006] At present, various visual positioning algorithms include a variety of methods for determining the main direction for key points, but these methods have different defects, which can be divided into two aspects: First, during the calculation process, due to too many pixels involved in the operation, the running speed is slow, which in turn leads to reduced real-time performance of the visual positioning algorithm; Second, the main direction determined for the feature points is not unique, and even ambiguous, which will reduce the accuracy of image registration and ultimately lead to reduced positioning accuracy of the visual positioning algorithm. Summary of the invention
[0007] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and to propose a method for determining the main direction of FAST key points of an image based on a RILBP operator, which has stronger real-time performance and higher image registration accuracy.
[0008] The present invention is a method for determining the main direction of FAST key points of an image based on a RILBP operator, which is characterized by comprising the following steps:
[0009] (1) Obtain the original image A: The size of the original image A is W×H, where W is the width of the original image A and H is the height of the original image A:
[0010] (2) Extend the original image A by performing a boundary mirroring operation: Extend the top, bottom, left, and right boundaries of the original image A by a length of 3 pixels to obtain the extended image A. 1 , its size is (W+6)×(H+6);
[0011] (3) For the expanded image A 1 Perform Gaussian smoothing: for the expanded image A 1 Perform Gaussian smoothing to obtain Gaussian smoothing result image A 2 , where the area where the original image A is located is recorded as B;
[0012] (4) Define an empty set: Define an empty set F C , using the empty set F C Store all FAST key points extracted from area B;
[0013] (5) Extract key points from area B using the FAST algorithm: For any pixel P in area B, first, draw a circle with pixel P as the center and 3 pixels as the radius to determine the circumference C. Then, starting from the top of pixel P, number the 16 pixels on the circumference C in a clockwise direction. Then, set a grayscale difference threshold t to compare the absolute value of the difference between the grayscale value of all pixels on the circumference C and the grayscale value of pixel P, so as to determine whether pixel P is a FAST key point in steps, including:
[0014] (5a) Determine the circumference C for any pixel P in region B: For the Gaussian smoothed image A2 A pixel P in area B has its gray value represented as I P , draw a circle with pixel P as the center and 3 pixels as the radius, and get the circumference C about pixel P;
[0015] (5b) The pixels and their grayscale values on the circumference C are numbered respectively: there are 16 pixels distributed on the circumference C. Starting from the point directly above the pixel P, the 16 pixels are numbered 1, 2, ..., 15, 16 in a clockwise direction, and the corresponding grayscale values are also numbered I, II, III, IV, V ... 1 ,I 2 ,...,I 15 ,I 16 ;
[0016] (5c) Setting a grayscale difference threshold t: Setting a grayscale difference threshold t is used to compare the absolute value of the difference between the grayscale values of all pixels on the circumference C and the grayscale value of the pixel P, and step by step determine whether the pixel P is a FAST key point. The grayscale difference threshold t is empirically set to 40;
[0017] (5d) Preliminary determination of whether pixel P is a candidate for a FAST keypoint: Calculate I 1 ,I 9 with I P The absolute value of the difference between 1 、a 9 If a 1 and a 9 If both are greater than the grayscale difference threshold t, then it is preliminarily determined that the pixel P is a candidate point of the FAST key point and the step (5e) is performed; otherwise, it is determined that the pixel P is not a FAST key point and the step (7) is performed;
[0018] (5e) Determine again whether pixel P is a candidate point for a FAST key point: calculate I 5 ,I 13 with I P The absolute value of the difference between 5 、a 13 If a 5 and a 13 If the number m of pixels greater than the grayscale difference threshold t is not zero, the pixel P is determined to be a candidate point of the FAST key point again, and step (5f) is performed; otherwise, the pixel P is determined not to be a FAST key point, and step (7) is performed;
[0019] (5f) Finally, determine whether pixel P is a FAST key point: Calculate the grayscale values of the remaining 12 pixels on the circle C except for those numbered 1, 5, 9, and 13 and the grayscale values of I PThe absolute value of the difference between t and t is recorded as n. If m+n≥10, the pixel P is finally determined to be a FAST key point and step (6) is executed. Otherwise, the pixel P is not a FAST key point.
[0020] (6) Classify FAST key points into set F C :If pixel P is determined to be a FAST key point, then pixel P is included in the set F C , proceed to step (7); if pixel P is determined not to be a FAST key point, proceed directly to step (7);
[0021] (7) Extract all FAST key points in region B: Select another pixel P in region B, return to step (5a), and repeatedly execute steps (5a) to (6) until all FAST key points in region B are obtained and are included in set F. C middle;
[0022] (8) Calculate the set F using the RILBP operator C The RILBP feature value of each pixel in: C For any pixel in the set F, take the pixel as the origin of the coordinate, and take the vertical axis as the 0 degree coordinate axis, and use the RILBP operator to calculate its RILBP eigenvalue. C Any pixel P in c First, take pixel P c A circle S is determined with a center and a radius of 1 pixel, and a circle P is defined with a length of 1 pixel. c The upper right corner pixel is taken as the starting point, and the 8 pixels on the circle S are numbered in a clockwise direction. Then an 8-bit binary sequence is initialized, and the binary sequence is cyclically shifted right bit by bit using the RILBP operator. The decimal numbers corresponding to all binary sequences that appear during the cyclic right shift are calculated and numbered respectively, and the smallest decimal number is used as the pixel P. c RILBP eigenvalue;
[0023] (9) Calculate and determine pixel P c The main direction of: During the circular bitwise right shift, the pixel P c The RILBP feature value may appear multiple times. The first RILBP feature value is recorded as k, and the pixel numbered k on the circle S is recorded as P k , connect pixel P c Center and pixel P k Center, get the direction vector Calculate the direction vector The angle with the 0 degree coordinate axis is taken as the pixel P c The main direction θ;
[0024] (10) Calculate and determine the set F C The main direction of all pixels in the set F C Select another pixel P c , repeatedly perform steps (8) to (9) until the set F is determined C The main direction of all pixels in .
[0025] The present invention solves the problems of poor real-time performance and low image registration accuracy in the prior art.
[0026] Compared with the prior art, the present invention has the following advantages:
[0027] The real-time performance of determining the main direction of the key point is better: when determining the main direction for the FAST key point, the present invention uses the FAST key point as the center and 1 pixel length as the radius to obtain the circumference of the FAST key point. During the calculation process, it is clarified that the pixels that need to participate in the calculation only include the pixels on this circumference, while the existing method involves all pixels inside the entire circle during calculation. Therefore, compared with the existing method, the method for determining the main direction for the FAST key point proposed by the present invention reduces the number of pixels involved in the calculation. Under the premise of comparable calculation complexity, the present invention has a faster calculation speed and is more in line with the real-time requirements of the visual positioning algorithm.
[0028] Improved image registration accuracy: When determining the main direction for a FAST key point, the present invention first calculates the RILBP eigenvalue of the FAST key point using the RILBP operator, records the number of the first RILBP eigenvalue that appears during the calculation process, and finally uses this number to uniquely determine the main direction for each FAST key point through a simple calculation. Compared with the existing method, this main direction does not cause ambiguity and has sufficient uniqueness. Relying on this significant main direction, rotation invariance is successfully achieved for the FAST key point, and the registration accuracy can be improved when performing image registration. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 It is a flowchart of the implementation process of the present invention;
[0030] Figure 2 This is a simulation effect diagram of image registration with a running time of 3.04ms in the present invention;
[0031] Figure 3 This is a simulation effect diagram of image registration with a running time of 1.30 ms. DETAILED DESCRIPTION
[0032] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.
[0033] Example 1
[0034] At present, in visual positioning, when determining the main direction for the key points of an image, there are disadvantages such as poor real-time performance due to the large amount of calculation and low image registration accuracy due to the lack of significant main direction. In view of the existing problems, the present invention proposes a method for determining the main direction of the key points of an image FAST based on the RILBP operator after research and experiments. Figure 1 , Figure 1 The implementation flow chart of the present invention includes the following steps:
[0035] (1) Obtaining the original image A: The size of the original image A is W×H, where W is the width of the original image A and H is the height of the original image A. In actual visual positioning applications, different requirements and different cameras are used, and the sizes of the original image A taken may also be different. For example, in some actual applications, W=640, H=480, while in some actual applications, W=1920, H=1080. However, the present invention is applicable to original images A of any size. In this embodiment, W=1920, H=1080.
[0036] (2) Extend the original image A by performing a boundary mirroring operation: Extend the top, bottom, left, and right boundaries of the original image A by a length of 3 pixels to obtain the extended image A. 1 , whose size is (W+6)×(H+6). In the subsequent extraction of FAST key points, it meets the radius requirement of 3 pixels when drawing a circle.
[0037] (3) For the expanded image A 1 Perform Gaussian smoothing: for the expanded image A 1 Perform Gaussian smoothing to obtain Gaussian smoothing result image A 2 , filtering out the expanded image A 1 The area where the original image A is located is recorded as B.
[0038] (4) Define an empty set: Define an empty set F C , using the empty set F C Store all FAST key points extracted from region B. Since the number of FAST key points extracted from different images is different, but in most cases less than 1500, in this embodiment, the empty set F C The capacity is 1500.
[0039] (5) Extract key points from region B using the FAST algorithm: Since the FAST algorithm has a simple extraction concept and a fast extraction speed among existing key point extraction algorithms, the present invention uses the FAST algorithm to extract key points. The specific concept is: first define an empty set to store all FAST key points extracted from region B, and for any pixel P in region B, draw a circle with pixel P as the center and 3 pixels as the radius, determine the circumference C, and take the top of pixel P as the starting point, and number the 16 pixels on the circumference C in a clockwise direction, and then set a grayscale difference threshold t, which is used to compare the absolute value of the difference between the grayscale values of all pixels on the circumference C and the grayscale value of pixel P, so as to determine in steps whether pixel P is a FAST key point, which specifically includes:
[0040] (5a) Determine the circumference C for any pixel P in region B: For the preprocessed image A 2 A pixel P in area B has its gray value represented as I P , draw a circle with pixel P as the center and 3 pixels as the radius, and get the circumference C about pixel P.
[0041] (5b) The pixels and their grayscale values on the circumference C are numbered respectively: there are 16 pixels distributed on the circumference C. Starting from the point directly above the pixel P, the 16 pixels are numbered 1, 2, ..., 15, 16 in a clockwise direction, and the corresponding grayscale values are also numbered I, II, III, IV, V ... 1 ,I 2 ,…,I 15 ,I 16 .
[0042] (5c) Setting grayscale difference threshold t: A grayscale difference threshold t is set to compare the absolute value of the difference between the grayscale values of all pixels on the circumference C and the grayscale value of pixel P, and step by step determine whether pixel P is a FAST key point. Studies have shown that when the grayscale difference threshold t is 40, the quality of the extracted FAST key point is higher. In this embodiment, t=40.
[0043] (5d) Preliminary determination of whether pixel P is a candidate for a FAST keypoint: Calculate the grayscale values I corresponding to the pixels numbered 1 and 9 on the circle C respectively. 1 ,I 9 with I P The absolute value of the difference between 1 、a 9 If a 1 and a 9 If both are greater than the grayscale difference threshold t, then it is preliminarily determined that the pixel P is a candidate point of the FAST key point and the step (5e) is performed; otherwise, it is directly determined that the pixel P is not a FAST key point and the step (7) is performed.
[0044] (5e) Determine again whether pixel P is a candidate point of a FAST key point: Calculate the grayscale value I corresponding to the pixels numbered 5 and 13 on the circumference C respectively. 5 ,I 13 with I P The absolute value of the difference between 5 、a 13 If a 5 and a 13 If the number m of pixels greater than the grayscale difference threshold t is not zero, the pixel P is determined to be a candidate point of the FAST key point again and step (5f) is performed; otherwise, the pixel P is directly determined not to be a FAST key point and step (7) is performed.
[0045] (5f) Finally, determine whether pixel P is a FAST key point: Calculate the grayscale values of the remaining 12 pixels on the circle C except the 12 pixels numbered 1, 5, 9 and 13 and the grayscale values of I P The absolute value of the difference between the grayscale difference and the grayscale difference threshold t is recorded as n. If m+n≥10, the pixel P is finally determined to be a FAST key point and step (6) is executed. Otherwise, the pixel P is directly determined not to be a FAST key point and step (7) is executed.
[0046] (6) Classify FAST key points into set F C :If pixel P is determined to be a FAST key point, then pixel P is included in the set F C , proceed to step (7); if pixel P is determined not to be a FAST key point, proceed directly to step (7).
[0047] (7) Extract all FAST key points in region B: Select another pixel P in region B, return to step (5a), and repeatedly execute steps (5a) to (6) until all FAST key points in region B are obtained and are included in set F. C In this step, the extraction of all FAST key points in region B is completed, and step (8) is performed.
[0048] (8) Calculate the set F using the RILBP operator C The RILBP feature value of each pixel in the set F C All pixels in the image are oriented vertically upward as the 0 degree coordinate axis, and the RILBP operator is used to calculate the RILBP eigenvalues of each. In the field of visual positioning, the purpose of determining the main direction for the key points is to add rotation invariance to the key points so that the image registration accuracy is still high when the image is rotated. The RILBP operator itself has rotation invariance, so the present invention integrates the RILBP operator to determine the main direction for the FAST key points. Specifically: for the set F CAny pixel P in c First, take pixel P c A circle S is determined with a center and a radius of 1 pixel, and a circle P is defined with a length of 1 pixel. c The upper right corner pixel is taken as the starting point, and the 8 pixels on the circle S are numbered in a clockwise direction. Then an 8-bit binary sequence is initialized, and the binary sequence is cyclically shifted right bit by bit using the RILBP operator. The decimal numbers corresponding to all binary sequences that appear are calculated and numbered respectively, and the pixel P is calculated with the smallest decimal number. c The RILBP eigenvalue of .
[0049] (9) Calculate and determine pixel P c The main direction of: During the circular bitwise right shift, the pixel P c The RILBP feature value may appear multiple times. The first RILBP feature value is recorded as k, and the pixel numbered k on the circle S is recorded as P k , connect pixel P c Center and pixel P k Center, get the direction vector Calculate the direction vector The angle with the 0 degree coordinate axis is taken as the pixel P c The main direction θ.
[0050] (10) Calculate and determine the set F C The main direction of all pixels in the set F C Select another pixel P c , repeatedly perform steps (8) to (9) until the set F is determined C The main direction of all pixels in .
[0051] The present invention provides an overall technical solution for determining the main direction of FAST key points of an image based on a RILBP operator. In order to solve the problems of weak real-time performance and low image registration accuracy in the prior art, the present invention has carried out research and thinking. The idea of the present invention is: first, an original image is subjected to a boundary mirror extension of 3 pixels in length and a Gaussian smoothing process to obtain a Gaussian smoothing process result image, and the region where the original image is located in the Gaussian smoothing process result image is recorded as B. Then, the key points of region B are extracted using a simple and fast FAST algorithm, specifically: first, an empty set is defined to store all FAST key points extracted from region B, and for any pixel P in region B, a circle is drawn with pixel P as the center and 3 pixels as the radius to determine the circumference C, and the 16 pixels on the circumference C are numbered in sequence from 1 to 16 in a clockwise direction with the pixel P directly above the pixel P as the starting point. Then, a grayscale difference threshold t is set to compare the absolute value of the difference between the grayscale values of all pixels on the circumference C and the grayscale value of pixel P, so as to determine in steps whether pixel P is a FAST key point. In essence, the purpose of determining the main direction for FAST key points is to achieve rotation invariance for FAST key points. Therefore, the rotation invariance of the RILBP operator itself is used to determine the main direction for the set F C Calculate the main direction of all pixels in the set F C Any pixel P in c First, take pixel P c The circumference S is determined by the center and the radius is 1 pixel long, and the pixel P is c The upper right corner pixel is taken as the starting point, and the 8 pixels on the circumference S are numbered from 1 to 8 in a clockwise direction. Then an 8-bit binary sequence is initialized, and the binary sequence is cyclically shifted right bit by bit using the RILBP operator. The decimal numbers corresponding to all binary sequences that appear are calculated and numbered respectively, and the pixel P is calculated with the smallest decimal number. c The RILBP feature value of pixel P c The RILBP feature value may appear multiple times. In order to ensure that the pixel P c The present invention records the number of the first RILBP feature value, and determines the pixel numbered k on the circle S as P according to the number. k , connect pixel P c Center and pixel P k Center, get the direction vector Finally calculate the direction vector The angle with the 0 degree coordinate axis is taken as the pixel P cTherefore, compared with the popular key point main direction determination method in the field of visual positioning, the method for determining the main direction of FAST key points based on the RILBP operator image proposed in the present invention reduces the number of pixels involved in the calculation by combining the RILBP operator, and gives the FAST key points more significant main direction information with a shorter calculation time, thereby improving the image registration accuracy. The method of the present invention is applied to the field of visual positioning, such as in warehouse logistics robots, to improve the real-time performance and positioning accuracy of the visual positioning algorithm, and ensure that the robot works with high efficiency and high accuracy.
[0052] Example 2
[0053] The method for determining the main direction of the FAST key point of the image based on the RILBP operator is the same as that in Example 1. In step (8) of the present invention, the RILBP operator is used to calculate and determine the set F C The RILBP feature value of each pixel in , specifically includes the following steps:
[0054] (8a) For the set F C Any pixel P in c Determine the circumference S: For the set F C Any pixel P in c , and its gray value is represented as I Pc , and in pixels P c As the center of the circle, with 1 pixel length as the radius, get pixel P c In the preprocessed image result image A 2 The circumference S in .
[0055] (8b) Number the 8 pixels in the circle S: There are 8 pixels distributed on the circle S, with pixel P c As the coordinate origin, with the vertical upward as the 0 degree coordinate axis, and the upper right corner pixel as the starting point, the 8 pixels on the circumference S are numbered P in a clockwise direction. 1 ,P 2 ,...,P 7 ,P 8 , and the corresponding grayscale values are also expressed as
[0056] (8c) Initialize an 8-bit binary sequence seq1: seq1 expresses and The relative size of the pixel P c The RILBP eigenvalue of seq1 is seq1=00000000.
[0057] (8d) Encode each bit of seq1: Compare them separately and The size of Then the i-th bit of seq1 is encoded as 1, otherwise the i-th bit of seq1 remains 0.
[0058] (8e) Perform a circular right shift operation on seq1: After the circular right shift operation, 8 binary sequences are obtained, which are represented as seq1, seq2, ..., seq7, seq8, where seqj means that seq1 has undergone a circular right shift operation of j-1 bits.
[0059] (8f) Convert seq1, seq2, ..., seq7, seq8 to decimal numbers: Represent the decimal numbers corresponding to seq1, seq2, ..., seq7, seq8 as dec1, dec2, ..., dec7, dec8 respectively.
[0060] (8g) Determine pixel P c RILBP eigenvalue: The RILBP eigenvalue represents the minimum value among dec1, dec2, ..., dec7, dec8. The minimum value among dec1, dec2, ..., dec7, dec8 is dec_min, and dec_min is used as the pixel P c The RILBP eigenvalue of .
[0061] The present invention is to set F C Any pixel P in c When determining the main direction, the pixels involved in the calculation only include pixels on the circumference, not pixels inside the circle. During the calculation, the number of pixels involved in the calculation is reduced, and the first RILBP eigenvalue that appears is recorded. This eigenvalue is unique and is calculated based on the pixel P. c The number corresponding to the RILBP feature value determines the pixel P c The main direction θ makes the pixel P c The main direction feature is more significant, eliminating the ambiguity of the main direction information in the prior art, with less calculation, enhancing the real-time performance of the visual positioning algorithm and improving the accuracy of image registration. The method of the present invention is used in unmanned driving, drone inspection and other occasions to improve positioning accuracy and provide protection for safe driving.
[0062] Example 3
[0063] The method for determining the main direction of the FAST key point image based on the RILBP operator is the same as that in Embodiment 1-2, characterized in that the calculation and determination of the pixel P in step (9) c The main direction is: get the value of dec_min that appears first in dec1, dec2, ..., dec7, dec8, denoted as deck, k represents the pixel P cThe number corresponding to the RILBP eigenvalue is k∈[1,2,3,4,5,6,7,8], and the pixel P c The calculation formula for the main direction θ is:
[0064]
[0065] The present invention records the first RILBP feature value, which eliminates the ambiguity in the existing methods in the field of visual positioning, and according to the pixel P c The number corresponding to the RILBP feature value is realized by simple calculation. c The main direction is determined by making the pixel P c It has a more significant main direction, which improves the accuracy of image registration. In the visual positioning algorithm, it can improve the positioning accuracy of the algorithm and provide accurate location information for more practical application functions such as navigation and obstacle avoidance.
[0066] A more specific example is given below to further illustrate the present invention.
[0067] Example 4
[0068] The method for determining the main direction of FAST key points of an image based on the RILBP operator is the same as that of Embodiments 1-3, and includes the following steps:
[0069] (1) Obtain an original image A, whose size is W×H, where W is the width of the original image A and H is the height of the original image A. In fact, the present invention is applicable to original images A of any size. In this embodiment, W=640 and H=480.
[0070] (2) The top, bottom, left, and right edges of the original image A are mirrored and extended by 3 pixels to obtain the expanded image A. 1 , whose size is (W+6)×(H+6). In the subsequent extraction of FAST key points, it meets the radius requirement of 3 pixels when drawing a circle.
[0071] (3) For the expanded image A 1 Perform Gaussian smoothing to obtain Gaussian smoothing result image A 2 , filtering out the expanded image A 1 The area where the original image A is located is recorded as B.
[0072] (4) Define an empty set: Define an empty set F C , using the empty set F C Store all FAST key points extracted from region B. Since the number of FAST key points extracted from different images is different, but generally less than 2000, in this embodiment, an empty set F is defined. CThe capacity is 2000.
[0073] (5) Extracting key points from region B using the FAST algorithm: In the field of visual positioning, in order to meet the requirements of real-time positioning, the visual positioning algorithm does not track and locate all pixels of an image, but extracts key points from an image, thereby reducing the number of pixels involved in tracking and positioning and improving real-time performance. The present invention not only extracts key points, but also adopts the FAST algorithm, which has a simple extraction idea and fast extraction speed among the existing key point extraction algorithms. However, the present invention is also applicable to all key point extraction algorithms. The specific idea of the FAST algorithm is as follows: first, an empty set is defined to store all FAST key points extracted from region B. For any pixel P in region B, a circle is drawn with pixel P as the center and 3 pixels as the radius to determine the circumference C. The 16 pixels on circumference C are numbered in a clockwise direction with the top of pixel P as the starting point. Then, a grayscale difference threshold t is set to compare the absolute value of the difference between the grayscale values of all pixels on circumference C and the grayscale value of pixel P, so as to determine in steps whether pixel P is a FAST key point. Moreover, the determination steps have different priorities, so the order of the steps is fixed, which speeds up the extraction of key points. Specifically include:
[0074] (5a) Determine the circumference C for any pixel P in region B: For the preprocessed image A 2 A pixel P in area B has its gray value represented as I P , draw a circle with pixel P as the center and 3 pixels as the radius, and get the circumference C about pixel P.
[0075] (5b) The pixels and their grayscale values on the circumference C are numbered respectively: there are 16 pixels distributed on the circumference C. Starting from the point directly above the pixel P, the 16 pixels are numbered 1, 2, ..., 15, 16 in a clockwise direction, and the corresponding grayscale values are also numbered I, II, III, IV, V ... 1 ,I 2 ,...,I 15 ,I 16 .
[0076] (5c) Setting grayscale difference threshold t: A grayscale difference threshold t is set to compare the absolute value of the difference between the grayscale values of all pixels on the circumference C and the grayscale value of pixel P, and step by step determine whether pixel P is a FAST key point. Studies have shown that when the grayscale difference threshold t is 40, the quality of the extracted FAST key point is higher. In this embodiment, t=40.
[0077] (5d) Preliminary determination of whether pixel P is a candidate for a FAST keypoint: Calculate the grayscale values I corresponding to the pixels numbered 1 and 9 on the circle C respectively. 1 ,I9 with I P The absolute value of the difference between 1 、a 9 If a 1 and a 9 If both are greater than the grayscale difference threshold t, then it is preliminarily determined that the pixel P is a candidate point of the FAST key point and the step (5e) is performed; otherwise, it is directly determined that the pixel P is not a FAST key point and the step (7) is performed.
[0078] (5e) Determine again whether pixel P is a candidate point of a FAST key point: Calculate the grayscale value I corresponding to the pixels numbered 5 and 13 on the circumference C respectively. 5 ,I 13 with I P The absolute value of the difference between 5 、a 13 If a 5 and a 13 If the number m of pixels greater than the grayscale difference threshold t is not zero, the pixel P is determined to be a candidate point of the FAST key point again and step (5f) is performed; otherwise, the pixel P is directly determined not to be a FAST key point and step (7) is performed.
[0079] (5f) Finally, determine whether pixel P is a FAST key point: Calculate the grayscale values of the remaining 12 pixels on the circle C except the 12 pixels numbered 1, 5, 9 and 13 and the grayscale values of I P The absolute value of the difference between the grayscale difference threshold t and the ... C Otherwise, directly determine that pixel P is not a FAST key point and proceed to step (7).
[0080] (6) Classify FAST key points into set F C :If pixel P is determined to be a FAST key point, then pixel P is included in the set F C , proceed to step (7). If pixel P is determined not to be a FAST key point, proceed directly to step (7).
[0081] (7) Extract all FAST key points in region B: Select another pixel P in region B, return to step (5a), and execute steps (5a) to (6) until all FAST key points in region B are obtained and are included in set F. C middle.
[0082] (8) Calculate the set F using the RILBP operator C The RILBP feature value of each pixel in the set F CAll pixels in the set are taken as the 0 degree vertical axis, and the respective RILBP eigenvalues are calculated using the RILBP operator. All FAST key points extracted in step (5) do not have rotation invariance, which will result in a decrease in the image registration accuracy when the image is rotated. In the visual positioning algorithm, the accuracy of the visual positioning system will be reduced, which is not conducive to other tasks such as obstacle avoidance and navigation in practical applications. In the field of visual positioning, adding rotation invariance to key points is generally done by determining the main direction for the key points so that a high image registration accuracy is maintained when the image is rotated. The RILBP operator itself has rotation invariance, so the present invention integrates the RILBP operator to determine the main direction for the FAST key points, and performs simulation verification. The simulation results also illustrate the feasibility and effectiveness of the present invention incorporating the RILBP operator into the determination of the main direction of the FAST key points. Specifically: for the set F C Any pixel P in c First, take pixel P c A circle S is determined with a center and a radius of 1 pixel, and a circle P is defined with a length of 1 pixel. c The upper right corner pixel is the starting point, and the 8 pixels on the circumference S are numbered in a clockwise direction. Then the RILBP operator is used to calculate the pixel P c In the calculation process, multiple RILBP feature values may appear. The present invention records the number of the first RILBP feature value, and the pixel numbered k on the circle S is recorded as P. k , connect pixel P c Center and pixel P k Center, get the direction vector Finally calculate the direction vector The angle with the 0 degree coordinate axis is taken as the pixel P c The main direction θ. Specifically include:
[0083] (8a) For the set F C Any pixel P in c Determine the circumference S: For the set F C Any pixel P in c , and its gray value is expressed as And the pixel P c As the center of the circle, with 1 pixel length as the radius, get pixel P c The preprocessed image results in image A 2 The circumference S in .
[0084] (8b) Number the 8 pixels in the circle S: There are 8 pixels distributed on the circle S, with pixel P c As the origin, with the vertical upward as the 0 degree coordinate axis, and the upper right corner pixel as the starting point, the 8 pixels on the circumference S are numbered P in a clockwise direction.1 ,P 2 ,...,P 7 ,P 8 , and the corresponding grayscale values are also expressed as
[0085] (8c) Initialize an 8-bit binary sequence seq1: seq1 expresses and The relative size of the pixel P c The RILBP eigenvalue of seq1 is seq1=00000000.
[0086] (8d) Encode each bit of seq1: Compare them separately and The size of Then the i-th bit of seq1 is encoded as 1, otherwise the i-th bit of seq1 remains 0.
[0087] (8e) Perform a circular right shift operation on seq1: After the circular right shift operation, 8 binary sequences are obtained, which are represented as seq1, seq2, ..., seq7, seq8, where seqj means that seq1 has undergone a circular right shift operation of j-1 bits.
[0088] (8f) Convert seq1, seq2, ..., seq7, seq8 to decimal numbers: Represent the decimal numbers corresponding to seq1, seq2, ..., seq7, seq8 as dec1, dec2, ..., dec7, dec8 respectively.
[0089] (8g) Determine pixel P c RILBP eigenvalue: The RILBP eigenvalue represents the minimum value among dec1, dec2, ..., dec7, dec8. The minimum value among dec1, dec2, ..., dec7, dec8 is dec_min, and dec_min is used as the pixel P c The RILBP eigenvalue of .
[0090] (9) Calculate and determine pixel P c The main direction θ: dec_min may appear multiple times in dec1, dec2, ..., dec7, dec8. In order to ensure that the pixel P c The uniqueness of the main direction θ is obtained, and the value of dec_min that appears first in dec1, dec2, ..., dec7, dec8 is recorded as deck. c The number k corresponding to the RILBP eigenvalue determines the pixel P cThe main direction θ is calculated as:
[0091]
[0092] Where k represents the pixel P c The number corresponding to the RILBP eigenvalue is k∈[1,2,3,4,5,6,7,8].
[0093] (10) Calculate and determine the set F C The main direction of all pixels in the set F C Select another pixel P c , repeatedly perform steps (8) to (9) until the set F is determined C The main direction of all pixels in the image is determined by the RILBP operator to determine the main direction of the FAST key point.
[0094] The method for determining the main direction of a FAST key point based on a RILBP operator of the present invention comprises the following steps: obtaining an original image A; performing boundary mirroring expansion on the original image A; and performing boundary mirroring expansion on the expanded image A. 1 Perform Gaussian smoothing; define an empty set; extract key points from region B using the FAST algorithm; assign the FAST key points to set F C ; Extract all FAST key points in region B; Use RILBP operator to calculate set F C The RILBP feature value of each pixel in is calculated and determined. c The main direction of the set F is calculated and determined C The main direction of all pixels in the image is used to determine the main direction of the FAST key point based on the RILBP operator in area B, adding rotation invariance to the FAST key point, so that the visual positioning algorithm can still maintain a good tracking and positioning state when the image rotates. The present invention combines the RILBP operator to limit the pixels involved in the operation to a circle, rather than all the pixels inside the entire circle, thereby accelerating the running speed of the visual SLAM algorithm and enhancing the real-time performance of the visual SLAM algorithm. At the same time, according to the number of the first RILBP eigenvalue, a unique main direction is determined for the FAST key point, eliminating the defect of ambiguity in the main direction information in the prior art, and improving the image registration accuracy of the visual SLAM algorithm.
[0095] The technical effects of the present invention are further illustrated below through simulation and its results.
[0096] Example 5
[0097] The method for determining the main direction of the FAST key point of the image based on the RILBP operator is the same as that of Embodiments 1-4. c When the main direction of the
[0098] Simulation conditions: The simulation conditions are Intel Corei5-5200U@2.20GHz, 8GRAM, Windows operating system, and Python programming language.
[0099] Simulation content: The simulation content is to select two images with relative motion relationship for image registration and record them as pixels P c Determine the runtime in the main direction.
[0100] Simulation results and analysis: see Figure 2 , Figure 2 This is the image registration simulation effect diagram of the present invention with a running time of 3.04ms. Figure 2 There are two images with relative motion relationship, both of which contain objects such as tables, monitors, tea cups, and books. Figure 2 Each line segment in represents the line connecting the two-dimensional projection points of the same three-dimensional space point in two images with relative motion relationship after image registration. Some of these line segments are obviously not parallel to the vast majority of line segments, which is a case of misregistration, while the vast majority of line segments that are parallel to each other are a case of correct registration. Figure 2 The image registration accuracy is 79.88%. Figure 2 The method used to determine the main direction for FAST key points is the grayscale centroid method. Figure 2 It shows that when determining the main direction for all FAST key points, the grayscale centroid method takes 3.04ms to run, 323 FAST key points are extracted, and an average of 9.41ns is required to determine the main direction for each FAST key point.
[0101] Example 6
[0102] The method for determining the main direction of the FAST key point of the image based on the RILBP operator is the same as that of Examples 1-4, and the simulation conditions and contents are the same as those of Example 5.
[0103] Simulation results and analysis: see Figure 3 , Figure 3 This is the image registration simulation effect diagram of the present invention with a running time of 1.30ms. Figure 3 Included with Figure 2 The same two images with relative motion relationship, Figure 3 The meaning of the line segments in Figure 2 same. Figure 3 When determining the main direction for the FAST key point, the method for determining the main direction of the FAST key point based on the RILBP operator image proposed in the present invention is adopted. Figure 2 , Figure 3There are fewer line segments that are obviously not parallel to the vast majority of line segments, indicating a higher image registration rate. Figure 3 The image registration accuracy is 87.43%. Figure 3 It shows that when determining the main direction for all FAST key points, the method for determining the main direction of FAST key points based on the RILBP operator image proposed by the present invention takes 1.30ms to run, extracts 358 FAST key points, and takes an average of 3.63ns to determine the main direction for each FAST key point, indicating that the method of the present invention has a shorter running time and a faster computing speed, and achieves a higher image registration accuracy with a shorter running time, making the visual positioning algorithm less computationally intensive, more real-time, and more accurate in image registration. In applications such as unmanned driving and sweeping robots, it can improve positioning accuracy and better perform tasks such as driving and cleaning.
[0104] In summary, the method for determining the main direction of FAST key points based on the RILBP operator of the present invention solves the problems of weak real-time performance and low image registration accuracy when determining the main direction of FAST key points in the prior art. The implementation steps include: obtaining the original image A; performing boundary mirroring expansion on the original image A; 1 Perform Gaussian smoothing; define an empty set; extract key points from region B using the FAST algorithm; assign the FAST key points to set F C ; Extract all FAST key points in region B; Use RILBP operator to calculate set F C The RILBP feature value of each pixel in is calculated and determined. c The main direction of the set F is calculated and determined C The main direction of all pixels in the image is determined to complete the determination of the main direction of the FAST key point in area B based on the RILBP operator. The present invention adds rotation invariance to the FAST key point, so that the visual positioning algorithm can still maintain a good tracking and positioning state when the image rotates. In combination with the RILBP operator, the pixels involved in the operation are limited to a circle, rather than all the pixels inside the entire circle, which speeds up the operation speed of the visual SLAM algorithm and enhances the real-time performance of the visual SLAM algorithm. At the same time, according to the numbering of the first RILBP eigenvalue, the present invention determines a unique main direction for the FAST key point, eliminates the defect of ambiguity in the main direction information in the prior art, and improves the image registration accuracy of the visual SLAM algorithm. The method of the present invention is used in mobile systems such as automatic driving and restaurant service robots to improve the positioning accuracy of the system and enhance the stability and sustainability of the system.
Claims
1. A method for determining the main direction of FAST key points in an image based on a RILBP operator, characterized in that: The steps include: (1) Obtain the original image A: The size of the original image A is W×H, where W is the width of the original image A and H is the height of the original image A: (2) Extend the original image A by performing a boundary mirroring operation: The top, bottom, left, and right boundaries of the original image A are all extended by a boundary mirroring operation with a length of 3 pixels to obtain an extended image A1, whose size is (W+6)×(H+6); (3) Gaussian smoothing of the expanded image A1: Gaussian smoothing is performed on the expanded image A1 to obtain a Gaussian smoothed result image A2, where the area where the original image A is located is denoted as B; (4) Define an empty set: Define an empty set F C , using the empty set F C Store all FAST key points extracted from area B; (5) Extract key points from area B using the FAST algorithm: For any pixel P in area B, first, draw a circle with pixel P as the center and 3 pixels as the radius to determine the circumference C. Then, starting from the top of pixel P, number the 16 pixels on the circumference C in a clockwise direction. Then, set a grayscale difference threshold t to compare the absolute value of the difference between the grayscale value of all pixels on the circumference C and the grayscale value of pixel P, so as to determine whether pixel P is a FAST key point in steps, including: (5a) Determine the circumference C for any pixel P in region B: For a pixel P in region B in the Gaussian smoothing result image A2, its grayscale value is expressed as I P , draw a circle with pixel P as the center and 3 pixels as the radius, and get the circumference C about pixel P; (5b) The pixels and their grayscale values on the circumference C are numbered respectively: there are 16 pixels distributed on the circumference C. Starting from the point just above the pixel P, the 16 pixels are numbered 1, 2, ..., 15, 16 in a clockwise direction, and the corresponding grayscale values are also numbered I1, I2, ..., I 15 ,I 16 ; (5c) Setting a grayscale difference threshold t: Setting a grayscale difference threshold t is used to compare the absolute value of the difference between the grayscale values of all pixels on the circumference C and the grayscale value of the pixel P, and step by step determine whether the pixel P is a FAST key point. The grayscale difference threshold t is empirically set to 40; (5d) Preliminary determination of whether pixel P is a candidate for a FAST keypoint: Calculate I1, I9 and I P The absolute value of the difference is recorded as a1 and a9; if a1 and a9 are both greater than the grayscale difference threshold t, the pixel P is preliminarily determined to be a candidate point of the FAST key point, and step (5e) is performed; otherwise, the pixel P is determined not to be a FAST key point, and step (7) is performed; (5e) Determine again whether pixel P is a candidate point for a FAST key point: calculate I5, I 13 with I P The absolute value of the difference is denoted as a5, a 13 ; If a5 and a 13 If the number m of pixels greater than the grayscale difference threshold t is not zero, the pixel P is determined to be a candidate point of the FAST key point again, and step (5f) is performed; otherwise, the pixel P is determined not to be a FAST key point, and step (7) is performed; (5f) Finally, determine whether pixel P is a FAST key point: Calculate the grayscale values of the remaining 12 pixels on the circle C except for those numbered 1, 5, 9, and 13 and the grayscale values of I P The absolute value of the difference between t and t is recorded as n. If m+n≥10, the pixel P is finally determined to be a FAST key point and step (6) is executed. Otherwise, the pixel P is not a FAST key point. (6) Classify FAST key points into set F C :If pixel P is determined to be a FAST key point, then pixel P is included in the set F C , proceed to step (7); if pixel P is determined not to be a FAST key point, proceed directly to step (7); (7) Extract all FAST key points in region B: Select another pixel P in region B, return to step (5a), and repeatedly execute steps (5a) to (6) until all FAST key points in region B are obtained and are included in set F. C middle; (8) Calculate the set F using the RILBP operator C The RILBP feature value of each pixel in: C For any pixel in the set F, take the pixel as the origin of the coordinate, and take the vertical axis as the 0 degree coordinate axis, and use the RILBP operator to calculate its RILBP eigenvalue. C Any pixel P in c First, take pixel P c A circle S is determined with a center and a radius of 1 pixel, and a circle P is defined with a length of 1 pixel. c The upper right corner pixel is taken as the starting point, and the 8 pixels on the circle S are numbered in a clockwise direction. Then an 8-bit binary sequence is initialized, and the binary sequence is cyclically shifted right bit by bit using the RILBP operator. The decimal numbers corresponding to all binary sequences that appear during the cyclic right shift are calculated and numbered respectively, and the smallest decimal number is used as the pixel P. c RILBP eigenvalue; (9) Calculate and determine pixel P c The main direction of: During the circular bitwise right shift, the pixel P c The RILBP feature value may appear multiple times. The first RILBP feature value is recorded as k, and the pixel numbered k on the circle S is recorded as P k , connect pixel P c Center and pixel P k Center, get the direction vector Calculate the direction vector The angle with the 0 degree coordinate axis is taken as the pixel P c The main direction θ; (10) Calculate and determine the set F C The main direction of all pixels in the set F C Select another pixel P c , repeat steps (8) to (9) until the set F is determined C The main direction of all pixels in .
2. The method for determining the main direction of FAST key points of an image based on a RILBP operator according to claim 1, characterized in that: The RILBP operator is used to calculate the set F described in step (8) C The RILBP feature value of each pixel in includes: (8a) For the set F C Any pixel P in c Determine the circumference S: For the set F C Any pixel P in c , and its gray value is expressed as And the pixel P c As the center of the circle, with 1 pixel length as the radius, get pixel P c The circumference S in the Gaussian smoothed image result image A2; (8b) Number the 8 pixels in the circle S: There are 8 pixels distributed on the circle S, with pixel P c As the coordinate origin, with the vertical upward as the 0 degree coordinate axis, and the upper right corner pixel as the starting point, the 8 pixels on the circumference S are numbered P1, P2, ..., P7, P8 in a clockwise direction, and the corresponding grayscale values are also expressed as (8c) Initialize an 8-bit binary sequence seq1: let the initial state of seq1 be seq1=00000000; (8d) Encode each bit of seq1: Compare them separately and The size of Then the i-th bit of seq1 is encoded as 1, otherwise the i-th bit of seq1 remains 0; (8e) seq1 is subjected to a cyclic right shift operation: 8 binary sequences are obtained through the cyclic right shift operation, which are represented as seq1, seq2, ..., seq7, seq8, respectively, where seqj indicates that seq1 has been subjected to a j-1 bit cyclic right shift operation; (8f) Convert seq1, seq2, ..., seq7, seq8 to decimal numbers: represent the decimal numbers corresponding to seq1, seq2, ..., seq7, seq8 as dec1, dec2, ..., dec7, dec8 respectively; (8g) Determine pixel P c RILBP feature value: record the minimum value among dec1, dec2, ..., dec7, dec8 as dec_min, and use dec_min as pixel P c The RILBP eigenvalue of .
3. The method for determining the main direction of FAST key points of an image based on a RILBP operator according to claim 1, characterized in that: The calculation described in step (9) determines the pixel P c The main direction θ is as follows: Get the value of dec_min that appears first in dec1, dec2, ..., dec7, dec8, denoted as deck, k represents the pixel P c The number corresponding to the RILBP eigenvalue is k∈[1,2,3,4,5,6,7,8], and the pixel P c The calculation formula for the main direction θ is: Where k represents the pixel P c The number corresponding to the RILBP eigenvalue is k∈[1,2,3,4,5,6,7,8].
