A method for generating a rotation-invariant multi-scale ring-shaped feature descriptor

Abstract

The application discloses a kind of generation methods of rotation invariant multi-scale annular feature descriptor, to overcome the challenge that feature recognition technology is difficult to accurately describe key features due to image rotation problem in image recognition field.First, multi-scale annular domain is constructed, for the segmentation local feature around feature point, to obtain multi-scale annular feature;Then the eigenvalue of annular feature is calculated with annular feature as basic unit;Finally, the eigenvalue obtained from multiple scales is sorted to form a feature vector with rotation invariance.The feature description scheme provided by the application avoids obtaining a feature vector with rotation invariance by estimating the dominant direction of the feature point.Compared with the scheme based on the dominant direction, the feature vector provided by the application will not be disturbed by the error from the dominant direction, avoids the operation of rotating all local features to the same direction according to the dominant direction, simplifies the calculation steps of describing local features, and reduces the algorithm complexity.

Description

Technical Field

[0001] This invention belongs to the field of computer vision research, specifically relating to a method for generating rotation-invariant multi-scale ring feature descriptors. Background Technology

[0002] The change in angle between the imaging device and the target object is a common scenario encountered during image acquisition. This change in angle results in varying degrees of rotation in the acquired image. Because the rotation angle is unknown and difficult to predict, it can easily interfere with image feature description, posing a challenge to many image recognition-based methods in the field of computer vision, including image registration, camera tracking, 3D reconstruction, and object recognition.

[0003] To address the image rotation problem, taking image matching as an example, a typical application of existing schemes is to guide feature description by estimating the dominant orientation of features. Specifically, SIFT was the first to propose estimating the gradient direction and magnitude of the image around a feature point, and statistically identifying the most significant direction as the dominant orientation of that feature point. Subsequently, the local features around the feature point are rotated according to the dominant orientation, or the coordinate axes are rotated to be in the same direction as the dominant orientation, with the aim of guiding the subsequent feature description process. Finally, feature descriptions are performed on feature points with unified orientations to obtain feature vectors. As one of the most representative local image feature descriptors, SIFT has profoundly influenced many improved algorithms, including SURF, ORB, and KAZE. These methods all employ the aforementioned scheme based on dominant orientation estimation to construct rotation-invariant feature descriptors. However, this scheme heavily relies on accurate estimation of the dominant orientation; errors in the dominant orientation will lead to inaccurate feature descriptions, resulting in decreased robustness in subsequent image recognition tasks, especially hindering the demand for robust feature descriptors in multimodal image recognition tasks. Furthermore, estimating the dominant orientation of each feature point and rotating all dominant orientations to the same direction is considered computationally time-consuming and costly. Summary of the Invention

[0004] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention discloses a method for generating rotation-invariant multi-scale ring feature descriptors. The method obtains ring features by segmenting local features through a ring domain, calculates feature values ​​using the ring features as basic units, and then sorts the feature values ​​to obtain rotation-invariant feature vectors, which are used to describe image characteristics or attributes, thereby improving the recognition accuracy and efficiency of rotated images.

[0005] This invention discloses a method for generating local feature descriptors of an image, comprising the following steps:

[0006] (1) Extract local features around the feature points;

[0007] (2) Segment local features into multiple annular regions;

[0008] (3) Analyze the eigenvalues ​​within the annular domain;

[0009] (4) Sort multiple feature values ​​to construct a feature vector with rotation invariance, i.e., feature descriptor.

[0010] Preferably, step (1) includes:

[0011] The feature points, also known as key points, are obtained by feature detection methods based on image processing technology. These feature detection methods include, but are not limited to, FAST, SIFT, and SURF algorithms, or image pixels can be used directly as feature points without using feature detection algorithms.

[0012] The number of the feature points in an image is multiple, and at least one of them must be present.

[0013] The local features are distributed in a circular or ring shape with the feature points as the center.

[0014] The local features are obtained by feature extraction algorithms based on image processing technology. These feature extraction algorithms include, but are not limited to, brightness extraction algorithms, texture extraction algorithms, contrast extraction algorithms, gradient extraction algorithms, phase extraction algorithms, amplitude value extraction algorithms, or image pixels can be used directly as local features without using feature extraction algorithms.

[0015] The local range of the local feature is not fixed, but it will not be less than a distance of 1 pixel and will not exceed the size range of the entire image.

[0016] Preferably, step (2) includes the following sub-steps:

[0017] (2-1) The center of the annular domain is (a, b) with the center of the local feature's range as the center of the annular domain, and the radius of the local feature is the radius R of the outer circle of the largest annular domain in each scale layer. h ;

[0018] (2-2) At the same scale level, a ring decision graph is constructed by stipulating that the area of ​​each ring domain is equal;

[0019] The equal areas of the annular domains in the same scale layer are subject to the following condition:

[0020]

[0021] Where i∈[1,l] are positive integers; l is the number of given annular regions; S m S represents the area of ​​the circular decision graph; h It is the area of ​​the outer circle of the largest annular domain in each scale layer;

[0022] The area of ​​the annular region is also the area of ​​the annular decision graph, and its formula is:

[0023]

[0024] Where (p,q) are the coordinates of the non-zero values ​​in the circular decision graph; (R i+1 ,R i ) represent the radii of the outer and inner circles of the annular domain, respectively; M is the annular decision graph; D p D q It refers to the neighborhood range of (p,q);

[0025] The radii of the outer and inner circles of the annular domain are characterized by satisfying the formula:

[0026] R i+1 2 :R i 2 =(i+1):i

[0027] The expression for the circular decision diagram is:

[0028] M = (C B -C S )>0

[0029] Among them, C B =(xa) 2 +(yb) 2 <R i+1 2 and C S =(xa) 2 +(yb) 2 <R i 2 These represent the feature or pixel regions contained within the outer and inner circles of the annular domain, respectively; (x, y) are the coordinates of the circular trajectory.

[0030] (2-3) Extend the method of step (2-2) to multiple scale layers, and construct a multi-scale ring decision graph by stipulating that the area of ​​each ring domain in each scale layer is equal;

[0031] (2-4) By constructing a multi-scale annular decision graph, local features are segmented into multiple annular domains to obtain multi-scale annular features;

[0032] The method for obtaining the multi-scale ring features follows the formula below:

[0033]

[0034] Where n is the number of multi-scale layers; F represents the local feature; and E is the obtained multi-scale ring feature.

[0035] Furthermore, the annular domain is geometrically annular, specifically the smallest annular domain constructed around the feature point, which is geometrically circular.

[0036] Preferably, step (3) includes:

[0037] The eigenvalues ​​within the annular domain are obtained by analyzing the multi-scale annular feature E. The eigenvalue analysis method includes the following features:

[0038] Multi-scale ring features segmented from the ring domain are used as the basic unit for analyzing eigenvalues;

[0039] All feature values ​​or key feature information in the multi-scale ring feature are used as the data source for analyzing feature values. The key information includes, but is not limited to, pixel mean, image brightness, gradient, contrast, and texture structure.

[0040] The feature value obtained from a single ring feature can be a single numerical value or multiple numerical values;

[0041] The number of feature values ​​obtained from all the annular features contained in multiple scale layers satisfies multiple conditions.

[0042] Preferably, step (4) includes:

[0043] The rotation-invariant eigenvector is obtained by sorting the multiple eigenvalues ​​in step 3 using a sorting method, which includes the following sub-steps:

[0044] (4-1) Sort the feature values ​​obtained from the circular feature to obtain the feature value sequence O1 in the circular feature;

[0045] (4-2) Using the feature value sequence O1 in step (4-1) as the basic unit for sorting, sort the feature value sequence units in the same scale layer to obtain the scale layer feature value sequence O2.

[0046] (4-3) Using the scale layer feature value sequence O2 in step (4-2) as the basic unit for sorting, sort the scale layer feature value sequence O2 units in all scale layers to obtain the feature vector V with rotation invariance.

[0047] Furthermore, the sorting is based on the spatial distribution relationship of the annular domain, and a fixed order is determined among the feature points to be compared to sort the feature values.

[0048] Preferably, it also includes

[0049] A normalization algorithm is used to normalize each feature vector.

[0050] Furthermore, normalization algorithms include, but are not limited to, Z-Score normalization and Linear normalization.

[0051] In summary, the technical solution disclosed in this invention has the following beneficial effects compared with existing methods:

[0052] This invention addresses the challenge of accurately describing key features in image recognition due to image rotation. The invention obtains ring features by segmenting local features into ring domains, calculates feature values ​​using these ring features as basic units, and then sorts the feature values ​​to obtain rotation-invariant feature vectors. This invention is the first to avoid obtaining rotation-invariant feature vectors by estimating the dominant direction of feature points. To our knowledge, the feature vector constructed by this invention is the first design to possess rotation-invariant feature encoding without involving the dominant direction. Therefore, it has the following two advantages: (1) The feature vectors provided by this invention are not affected by errors from the dominant direction, and the constructed feature vectors accurately characterize image features or attributes, improving the recognition accuracy of rotated images; (2) The feature vectors provided by this invention achieve rotation invariance by directly analyzing the ring feature units to obtain the feature values ​​for constructing the feature vectors, avoiding the operation of rotating all local features to the same direction according to the dominant direction, simplifying the calculation steps for describing local features, and improving the recognition efficiency of rotated images. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of a multi-scale ring feature descriptor method for a rotation-invariant multi-scale ring feature descriptor provided in an embodiment of the present invention.

[0054] Figure 2 This is a flowchart illustrating the application of a rotation-invariant multi-scale ring feature descriptor method provided in this embodiment of the invention to image registration tasks.

[0055] Figure 3 This is a schematic diagram illustrating the working principle of a rotation-invariant multi-scale ring feature descriptor method provided in this embodiment of the invention, applied to image registration tasks.

[0056] Figure 4 This invention provides a method for displaying registration results using a rotation-invariant multi-scale ring feature descriptor in an image registration task. Detailed Implementation

[0057] To more clearly understand the objectives, technical solutions, and advantages of this invention, the following description, in conjunction with the accompanying drawings and embodiments, further elaborates on the invention. It should be understood that the embodiments used to illustrate the invention are for illustrative purposes only and are not intended to limit the invention. Furthermore, this invention can be applied to multiple fields related to computer vision; other fields not illustrated in the embodiments are also applicable to this invention. All content related to the claims of this invention is within the scope of protection of this invention.

[0058] See Figures 1 to 4 The present invention is illustrated in the field of image registration, and is described in detail below:

[0059] like Figure 2 As shown, the main steps of the embodiments provided by the present invention are as follows:

[0060] Step 1, extract local features around the feature points:

[0061] (1-1) Obtain two source images for image registration, and use the FAST corner detection algorithm to obtain feature points (or key points) on the two images that have relevant corner feature information;

[0062] (1-2) The source image is smoothed using a guided filtering algorithm to obtain the image brightness feature, and then the brightness feature is subtracted from the source image to obtain the image texture feature F; one or both of the brightness feature and texture feature are used as the feature information of the source image; for ease of description, this embodiment takes the texture feature F as an example for explanation;

[0063] (1-3) Set the radius of the circle to R. o =25 pixels, centered on the feature point with R o Use R as the radius to extract local features of the circular region surrounding the feature point; or set the outer radius of the annular circle to R. o =25 pixels, the inner circle radius is r=5 pixels, and the local features of the annular area around the feature point are extracted with the feature point as the center point of the ring. For ease of description, this embodiment takes the local features of the circular area as an example for explanation.

[0064] Step 2, as follows Figure 3 The local features are segmented into multiple annular regions as shown:

[0065] (2-1) Taking the center of the local feature's range (i.e., the location of the feature point) as the center (a, b) of the annular domain, and the radius R of the local feature... o =25 pixels is the radius R of the outer circle of the largest annular domain in each scale layer. h =25 pixels;

[0066] (2-2) At the same scale level, a ring decision graph is constructed by stipulating that the area of ​​each ring domain is equal;

[0067] The equal areas of the annular domains in the same scale layer are subject to the following condition:

[0068]

[0069] Where i∈[1,l] are positive integers; l is the number of given annular regions; S m S represents the area of ​​the circular decision graph; h It is the area of ​​the outer circle of the largest annular domain in each scale layer;

[0070] The area of ​​the annular region is also the area of ​​the annular decision graph, and its formula is:

[0071]

[0072] Where (p,q) are the coordinates of the non-zero values ​​in the circular decision graph; (R i+1 ,R i ) represent the radii of the outer and inner circles of the annular domain, respectively; M is the annular decision graph; D p D q It refers to the neighborhood range of (p,q);

[0073] The radii of the outer and inner circles of the annular domain are characterized by satisfying the formula:

[0074] R i+1 2 :R i 2 =(i+1):i

[0075] The expression for the circular decision diagram is:

[0076] M = (C B -C S )>0

[0077] Among them, C B =(xa) 2 +(yb) 2 <R i+1 2 and C S =(xa) 2 +(yb) 2 <R i 2 These represent the feature or pixel regions contained within the outer and inner circles of the annular domain, respectively; (x, y) are the coordinates of the circular trajectory.

[0078] (2-3) such as Figure 1 As shown, the method of step (2-2) is extended to multiple scale layers, and the area of ​​each annular domain in each scale layer is equal to construct a multi-scale annular decision graph.

[0079] By setting the number of annular domains in the nth scale layer to l=8, an 8-ring decision graph is obtained;

[0080] By halving the number of ring domains between the (n-1)th scale layers, four new ring decision graphs are obtained.

[0081] Similarly, the number of ring domains between n-2 scale layers is halved to obtain two new ring decision graphs;

[0082] Similarly, a circular decision graph can be obtained from the first scale layer.

[0083] Based on the method of halving the number of ring domains in adjacent scale layers, we can deduce that n = l / 2 = 4 layers, and obtain a total of 15 ring decision graphs.

[0084] (2-4) By constructing a multi-scale annular decision graph, local features are segmented into multiple annular domains to obtain multi-scale annular features;

[0085] The method for obtaining the multi-scale ring features follows the formula below:

[0086]

[0087] Where n is the number of multi-scale layers; F represents the local texture feature; and E is the obtained multi-scale ring texture feature.

[0088] Step 3, as follows Figure 3 As shown, the eigenvalues ​​within the ring domain are analyzed:

[0089] (3-1) Using the ring-shaped texture features as the basic unit, a Gaussian filtering algorithm is used to remove all noise from the ring-shaped texture features obtained in step 2, thus obtaining the key texture feature unit W (e.g., Figure 3 (as shown);

[0090] (3-2) Calculate the average value T1 of each texture feature unit W, using the formula:

[0091] T1 = mean(W η )

[0092] Where η is the number of elements in the current texture feature unit; specifically, for the case of directly using all elements in the annular texture feature E, η is numerically equal to the area of ​​the annular decision graph.

[0093] (3-3) Using T1 as the threshold, texture feature information is classified as follows:

[0094] W1∈[W <T]

[0095] W2∈[W≥T]

[0096] Where W1 represents the set of elements in texture feature unit W whose values ​​are less than T1; W2 represents the set of elements in texture feature unit W whose values ​​are greater than or equal to T1;

[0097] (3-4) Calculate the average values ​​T2 and T3 of sets W1 and W2, using the formula:

[0098] T2 = mean(W1)

[0099] T3 = mean(W1)

[0100] (3-5) The three feature values ​​obtained from each annular texture feature unit are T1, T2, and T3.

[0101] Step 4, as follows Figure 3 As shown, sorting multiple feature values ​​constructs a rotation-invariant feature vector:

[0102] (4-1) Sort the feature values ​​T1, T2, T3 obtained from the ring feature to obtain the feature value sequence O1 = [T1, T2, T3];

[0103] (4-2) Using the feature value sequence O1 from step (4-1) as the basic unit for sorting, sort the feature value sequence O1 units in the same scale layer to obtain the scale layer feature value sequence O2, whose expression is:

[0104]

[0105] Where l is the number of ring domains in the nth scale layer;

[0106] (4-3) Using the scale layer feature value sequence O2 from step (4-2) as the basic unit for sorting, sort the scale layer feature value sequence O2 units in all scale layers to obtain the rotation-invariant feature vector V, whose expression is:

[0107]

[0108] Where n = 4 is the number of scale layers;

[0109] Step 4 provides a method for sorting feature values ​​to construct feature vectors. Other unlisted sorting methods share the same feature value as step 4: if the feature values ​​contained in the feature vector of a feature point are sorted according to a certain rule, then all other feature points will obtain valid feature vectors by sorting the feature values ​​according to this rule.

[0110] Step 5: Using the methods described in steps 1 to 4, obtain feature vectors for multiple feature points. Then, normalize the feature vectors using the Z-Score normalization algorithm. The normalization formula is as follows:

[0111]

[0112] Where μ represents the mean of the elements contained in the eigenvector V, and σ represents the standard deviation of the elements contained in the eigenvector V. This represents the normalized eigenvector.

[0113] Step 6, as follows Figure 3 As shown, the similarity level between feature vectors is measured using Euclidean distance:

[0114] The distance between feature vectors is measured by calculating the Euclidean distance between corresponding elements of the feature vectors and summing all distances. This distance represents the similarity level between feature points. The similarity level between feature points corresponds to the distance between feature vectors; a smaller distance indicates a higher similarity level, and a larger distance indicates a lower similarity level.

[0115] Following the main steps shown in steps 1 to 4, the outer radius R of the largest annular domain in each scale layer is set. h In a scenario where the pixel size is 25 and the maximum number of ring domains in a single scale layer is l=8, 15 ring features can be obtained. Each ring feature can be calculated to have 3 feature values, and each feature point can be described by a 45-dimensional feature vector.

[0116] like Figure 4 The image registration results shown are illustrated using the method provided in this example to register medical images (CT and MRI images) and remote sensing images (visible light and panchromatic remote sensing images), respectively. The results demonstrate that the rotation-invariant multi-scale ring feature descriptor method disclosed in this invention is not only applicable to various image types, but also robustly maintains image recognition accuracy when dealing with image rotation at multiple angles.

[0117] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating a rotation-invariant multi-scale ring-shaped feature descriptor, characterized in that, The method comprises the following steps: (1) extracting local features around the feature points; (2) segmenting the local features into multiple annular domains; Step (2) comprises the following sub-steps: (2-1) Taking the center of the range where the local feature is located as the center of the annular domain Taking the radius of the local feature as the radius of the outer circle of the largest annular domain in each scale layer ; (2-2) in the same scale layer, constructing an annular decision graph by setting the area of each annular domain to be equal; (2-3) applying the method of step (2-2) to multiple scale layers, and constructing a multi-scale annular decision graph by setting the area of each annular domain in each scale layer to be equal; (2-4) segmenting the local features into multiple annular domains by using the constructed multi-scale annular decision graph to obtain multi-scale annular features; (3) analyzing the feature values in the annular domains; (4) sorting the multiple feature values to construct a feature vector with rotation invariance, i.e., a feature descriptor; Step (4) comprises: The feature vector with rotation invariance is obtained by sorting the multiple feature values in step (3) using a sorting method, and the sorting method comprises the following sub-steps: (4-1) sorting the feature values obtained from the ring feature to obtain a sequence of feature values in the ring feature ; (4-2) the sequence of eigenvalues in the step (4-1) For the basic unit of sorting, the sequence of eigenvalues in the same scale layer is sorted to obtain the sequence of eigenvalues in the scale layer ; (4-3) obtaining the scale layer feature value sequence in the step (4-2) For the basic unit of sorting, the scale layer feature value sequence in all scale layers Sort the unit to obtain the feature vector with rotation invariance .

2. The method of claim 1, wherein: Step (1) comprises: The feature points, also known as key points, are obtained by a feature detection method based on image processing technology, and the feature detection method includes FAST, SIFT, SURF algorithm, or directly uses image pixels as feature points without using a feature detection algorithm; The number of feature points in an image is multiple; The local features are circularly or annularly distributed around the feature points; The local features are obtained by a feature extraction algorithm based on image processing technology, and the feature extraction algorithm includes brightness extraction algorithm, texture extraction algorithm, contrast extraction algorithm, gradient extraction algorithm, phase extraction algorithm, amplitude value extraction algorithm, or directly uses image pixels as local features without using a feature extraction algorithm; The local range of the local features is not fixed, and the minimum range is not less than 1 pixel distance, and the maximum range is not more than the size of the whole image.

3. The method of claim 1, wherein: The area of the annular domain in the same scale layer is equal, which follows the following conditions: wherein is a positive integer; is the number of ring-shaped domains given; represents the area of the ring-shaped decision graph; is the maximum outer circle area of the ring-shaped domains in each scale layer; The area of the annular domain is also the area of the annular decision graph, and the formula is: wherein, is the coordinate of non-zero value in the annular decision graph; are the outer and inner circle radii of the annular domain, respectively; is the annular decision graph; refers to neighborhood range; The radii of the outer and inner circles of the annular domain satisfy the formula: The expression of the annular decision graph is: wherein, and are the features or pixel areas contained by the outer and inner circles of the annular domain, respectively; are the circular trajectory coordinates; The acquisition method of the multi-scale annular features follows the following formula: wherein, is a number of scales; denotes local features; is the obtained multi-scale annular features.

4. The method of claim 1, wherein: The annular domain is annular in geometry, and the smallest annular domain is circular in geometry.

5. The method of claim 3, wherein: Step (3) comprises: said analyzing eigenvalues within the annular domain, by analyzing multi-scale annular features obtained, said eigenvalue analysis method comprising: Using the multi-scale annular features segmented from the annular domain as the basic unit for analyzing the feature values; Using all the feature values or key feature information in the multi-scale annular features as the data source for analyzing the feature values, and the key information includes pixel mean, image brightness, gradient, contrast, and texture structure; The feature values obtained from a single annular feature are one or multiple numerical values; The number of feature values obtained from all the annular features in multiple scale layers is multiple.

6. The method of claim 1, wherein: The sorting is based on the spatial distribution relationship of the annular domains to determine a fixed order between the feature points to be compared for feature value sorting.

7. The method of claim 1, wherein: Step 4 further comprises using a normalization algorithm to normalize each feature vector.

8. The method of claim 7, wherein: Normalization algorithms include Z-Score normalization, Linear normalization.

Images