Matching method based on directional fast rotation BRIEF algorithm features

Abstract

The invention belongs to the technical field of image matching, and particularly relates to a matching method based on directional fast rotation BRIEF algorithm features, which comprises the following steps: S1, inputting a reference image and a target image, and graying the reference image and the target image; s2, performing BH feature enhancement on the two images after graying; the features of a reference image and a target image are enhanced by combining a Butterworth frequency domain filtering method and a Haar wavelet transform method, the frequency domain features and local structure features of the images are balanced, and the visibility and contrast of the image features are highlighted and enhanced. According to the method, Butterworth filtering and Haar wavelet transform are introduced for feature enhancement, target contour features are effectively enhanced, and detail information of image edges and textures is highlighted; the amount of information carried by each feature point is quantitatively evaluated by using the adaptive information entropy, so that the matching efficiency and precision are remarkably improved; bRIEF descriptors are replaced with BEBLID descriptors, efficient feature point matching is carried out in combination with an FLANN algorithm, and the robustness and accuracy of the matching process are further improved.

Description

Technical Field

[0001] The invention relates to the technical field of image matching, and in particular to a matching method based on directional rapid rotation BRIEF algorithm features. Background Art

[0002] Image matching can be broadly defined as the mapping of two images in space and grayscale. It is the basis of image processing processes such as image stitching and recognition, and is widely used in medical image matching, 3D reconstruction and other scenarios. Feature point-based image matching algorithms usually include feature point detection, feature point description, feature point matching and other steps. Feature point detection and description are generally referred to as feature extraction.

[0003] The Directed Rapid Rotation BRIEF (ORB) algorithm is a commonly used feature extraction algorithm that can detect the feature points of an image using FAST and describe them using the BRIEF descriptor. The FAST algorithm has a fast detection speed and good robustness. It determines whether it is a corner point by comparing the grayscale value of the pixel point, and generates the corresponding feature point after the corner point is determined; the BRIEF descriptor has a fast generation speed and small memory consumption, and can generate a stable feature descriptor. It generates a fixed-length feature descriptor by binary encoding the pixels at a specific position.

[0004] After feature extraction, the image matching is realized by using the feature point matching algorithm. The Fast Library for Approximate Nearest Neighbors (FLANN) algorithm is one of the algorithms based on feature point matching. It has high robustness to transformations such as rotation and scale, and is suitable for matching of various types of images. Therefore, the FLANN algorithm is selected to realize feature point matching.

[0005] However, in the entire image matching process, when faced with complex changes such as image rotation, scale, blur, brightness, etc., the ORB algorithm's limited extraction ability leads to problems such as mismatching and low matching accuracy. Therefore, we propose a matching method based on the features of the directional rapid rotation BRIEF algorithm to solve the above problems. Summary of the invention

[0006] 1. Technical issues to be resolved

[0007] In view of the deficiencies in the prior art, the present invention provides a matching method based on the characteristics of a directional rapid rotation BRIEF algorithm, which solves the problems raised in the above background technology.

[0008] (II) Technical solution

[0009] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:

[0010] A matching method based on the characteristics of a directional rapid rotation BRIEF algorithm comprises the following steps:

[0011] S1: Input the reference image and the target image and convert them into grayscale;

[0012] S2: Perform BH feature enhancement on the two grayscale images; combine Butterworth frequency domain filtering and Haar wavelet transform to enhance the features of the reference image and the target image, balance the frequency domain characteristics and local structural characteristics of the image, and highlight and enhance the visibility and contrast of the image features;

[0013] S3: For the reference image and the target image after BH feature enhancement, feature point detection is performed on the two images based on the FAST detection part in the ORB feature extraction algorithm;

[0014] S4: For the FAST detection part in the ORB feature extraction process, a feature detection algorithm based on adaptive information entropy is introduced to detect feature points, which is used to quantify the amount of information carried by the feature points and filter out feature points with lower quality to reduce interference with the subsequent matching process;

[0015] S5: Combine BEBLID descriptor and FLANN algorithm to achieve feature description and matching.

[0016] Furthermore, the specific process of S2 is as follows: S2.1: Butterworth frequency domain filtering uses frequency domain characteristics to filter high-frequency interference to enhance low-frequency characteristics, smooth the frequency response of the image, and reduce interference. The frequency response H(u,v) is calculated as follows:

[0017]

[0018] Where D 0 is the cutoff frequency; D(u,v) is the distance from the point (u,v) in the frequency domain to the center of the frequency domain (u 0 ,v 0 ) distance; the coordinates (x, y) in the spatial domain are transformed into the coordinates (u, v) in the frequency domain through Fourier transformation;

[0019] S2.2: Haar wavelet transform uses two-dimensional Haar wavelet as basis function to perform wavelet transform, extract features in the image and enhance its details; Haar transform obtains the transformed approximate information A, horizontal detail information H, vertical detail information V and diagonal detail information C through low-pass and high-pass filtering in both horizontal and vertical directions, and then performs inverse wavelet transform on the wavelet coefficients, merges the approximate information and detail coefficients to reconstruct an enhanced new image and highlight the image details.

[0020] Furthermore, the specific content of S3 is: using the FAST detection algorithm to detect feature points, when a pixel has a number of consecutive pixels in its neighborhood whose grayscale value or color value is much higher or lower than that of the pixel, the pixel is considered to be a feature point; FAST performs local operations on each pixel point, does not require global calculation or traversal, and is relatively fast. The specific steps are:

[0021] S3.1: Select pixel P and set its brightness to B P ;

[0022] S3.2: Set a threshold T to determine whether the change in pixel value is significant;

[0023] S3.3: Select a neighborhood size and select 16 pixels on a circle with a radius of 3 and centered at pixel P;

[0024] S3.4: If there are N consecutive points on the selected circle with brightness greater than B P +T or less than B P -T, then the pixel P is marked as a corner point, i.e., a feature point;

[0025] S3.5: Repeat the above four steps and perform the same operation on each pixel to obtain the feature point.

[0026] Furthermore, the specific process of detecting feature points by the feature detection algorithm based on adaptive information entropy in S4 is as follows: S4.1: After the FAST algorithm of ORB detects feature points, the gray level at the pixel point (i, j) in the image is calculated as follows:

[0027]

[0028] Where p(i,j) is the gray level at the point (i,j) in the image, I ij is the probability distribution of gray level at the feature point (i, j); the information entropy H of the M×N image block where the feature point is located can be calculated using the gray level probability distribution, as shown in the following formula:

[0029]

[0030] The calculated information entropy value reflects the complexity or texture diversity of the local area, that is, the degree of pixel change; the larger the entropy value, the richer the texture and the greater the amount of information, indicating that the quality of the feature points in the area is higher;

[0031] S4.2: In the process of FAST feature point detection algorithm, the higher the local entropy value of the image block where the feature point is located, the more information the feature point contains, the more effective the detection is, and the image block is retained; conversely, the lower the local information entropy value, the worse the effect of detecting the feature point, and the image block is filtered; therefore, for each feature point, the entropy value of an image block centered on it is calculated as the information entropy value of the point; the details are as follows:

[0032] Ⅰ: According to the entropy values ​​of all N feature points calculated, the average value is calculated and then multiplied by a proportional coefficient k as the adaptive threshold (0<k≤1), as shown in the following formula:

[0033]

[0034] Ⅱ: Use this threshold thresh, k = 0.8, to filter feature points. Feature points above the threshold are retained, and those below the threshold are filtered out. The larger the k value, the higher the quality of the filtered feature points. If fewer feature points are detected, adjust the k value to a smaller number, then k = 0.5, and relax the filtering conditions.

[0035] Furthermore, the specific content of the feature description using the BEBLID descriptor in S5 is:

[0036] Ⅰ: After detecting feature points in the image, construct an integral image to speed up the subsequent calculation of the grayscale value of the local area;

[0037] II: For each feature point, define a set of weak classifiers. The vector of the kth weak classifier in the image is:

[0038] h k (z) = h k (z; F, T)

[0039] In the formula, z represents the input feature of the image block where the feature point is located; F is the feature extraction function (F: x→R), which is used to calculate the average gray value difference of the image block area. When the local image block x and the classification threshold T are given, we can get:

[0040]

[0041] The key to the BEBLID descriptor is the feature extraction function F(x) which is highly discriminative and fast to calculate, and is defined as the average grayscale value difference between the pixels of two image blocks, as follows:

[0042]

[0043] Where F(x) is the feature extraction function, I(q) and I(r) are the grayscale values ​​of image blocks q and r respectively, and R(p,s) is the square neighborhood with pixel p as the center and side length s;

[0044] III: BEBLID descriptor combines the output of each weak classifier and binarizes it to form a binary descriptor. By comparing F(x) with T, BEBLID converts the -1 output to 0 and the +1 output to 1, and finally obtains a binary descriptor:

[0045]

[0046] Furthermore, the specific content of using the FLANN algorithm for feature point matching in S5 is: the FLANN algorithm searches from the nearest neighbor of the feature point to the nearest point based on the Euclidean distance, and finds the corresponding feature point by calculating the Euclidean distance of each feature point of the two images, as shown in the following formula:

[0047]

[0048] In the formula, (x i , y i ) represents the feature descriptor vector, D(x,y) is the Euclidean distance between two feature points, and i is the vector dimension; the FLANN matcher accelerates the nearest neighbor search by establishing an index-based structure and ultimately achieves image matching.

[0049] (III) Beneficial effects

[0050] Compared with the prior art, the present invention provides a matching method based on the characteristics of the directional rapid rotation BRIEF algorithm, which has the following beneficial effects:

[0051] The present invention introduces Butterworth filtering and Haar wavelet transform for feature enhancement, which effectively enhances the target contour features and highlights the detailed information of image edges and textures; uses adaptive information entropy to quantitatively evaluate the amount of information carried by each feature point, screens and eliminates low-quality feature points that cause interference in the matching process, thereby significantly improving the matching efficiency and accuracy; uses BEBLID descriptor to replace BRIEF descriptor, which has the advantage of enhancing the distinguishing ability of feature description when the image undergoes complex changes such as brightness, rotation and scale; and combines the FLANN algorithm for efficient feature point matching, further improving the robustness and accuracy of the matching process. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A flowchart of a matching method based on the directional rapid rotation BRIEF algorithm feature of the present invention;

[0053] Figure 2 It is a flowchart of the feature detection algorithm based on adaptive information entropy of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] Example

[0056] like Figure 1-2 As shown, a matching method based on the directional rapid rotation BRIEF algorithm feature proposed in one embodiment of the present invention includes the following steps:

[0057] S1: Input the reference image and the target image and convert them into grayscale;

[0058] S2: Perform BH feature enhancement on the two grayscale images; combine Butterworth frequency domain filtering and Haar wavelet transform to enhance the features of the reference image and the target image, better balance the frequency domain characteristics and local structural features of the image, highlight and enhance the visibility and contrast of the image features, thereby improving the accuracy and stability of matching;

[0059] The specific process is as follows: S2.1: Butterworth frequency domain filtering uses frequency domain characteristics to filter high-frequency interference to enhance low-frequency features, smooth the frequency response of the image, and reduce interference. The frequency response H(u,v) is calculated as follows:

[0060]

[0061] Where D 0 is the cutoff frequency; D(u,v) is the distance from the point (u,v) in the frequency domain to the center of the frequency domain (u 0 ,v 0 ) distance; the coordinates (x, y) in the spatial domain are transformed into the coordinates (u, v) in the frequency domain through Fourier transformation;

[0062] S2.2: Haar wavelet transform uses two-dimensional Haar wavelet as basis function to perform wavelet transform, extract features in the image and enhance its details; Haar transform obtains the transformed approximate information A, horizontal detail information H, vertical detail information V and diagonal detail information C through low-pass and high-pass filtering in both horizontal and vertical directions, and then performs inverse wavelet transform on the wavelet coefficients, merges the approximate information and detail coefficients to reconstruct an enhanced new image and highlight the image details.

[0063] S3: For the reference image and the target image after BH feature enhancement, feature point detection is performed on the two images based on the FAST detection part in the ORB feature extraction algorithm;

[0064] The specific content is: use the FAST detection algorithm to detect feature points. When there are several consecutive pixels in the neighborhood of a pixel whose grayscale value or color value is much higher or lower than that of the pixel, the pixel is considered to be a feature point. FAST performs local operations on each pixel point, does not require global calculation or traversal, and is fast. The specific steps are:

[0065] S3.1: Select pixel P and set its brightness to B P ;

[0066] S3.2: Set a threshold T to determine whether the change in pixel value is significant;

[0067] S3.3: Select a neighborhood size and select 16 pixels on a circle with a radius of 3 and centered at pixel P;

[0068] S3.4: If there are N consecutive points on the selected circle with brightness greater than B P +T or less than B P -T, then the pixel P is marked as a corner point, i.e., a feature point;

[0069] S3.5: Repeat the above four steps and perform the same operation on each pixel to obtain the feature point.

[0070] S4: For the FAST detection part in the ORB feature extraction process, a feature detection algorithm based on adaptive information entropy is introduced to detect feature points, which is used to quantify the amount of information carried by the feature points and filter out feature points with lower quality to reduce interference with the subsequent matching process;

[0071] The specific process of detecting feature points based on the feature detection algorithm based on adaptive information entropy is as follows:

[0072] S4.1: After the FAST algorithm of ORB detects the feature points, the gray level at the pixel point (i, j) in the image is calculated as follows:

[0073]

[0074] Where p(i,j) is the gray level at the point (i,j) in the image, I ij is the probability distribution of gray level at the feature point (i, j); the information entropy H of the M×N image block where the feature point is located can be calculated using the gray level probability distribution, as shown in the following formula:

[0075]

[0076] The calculated information entropy value reflects the complexity or texture diversity of the local area, that is, the degree of pixel change; the larger the entropy value, the richer the texture and the greater the amount of information, indicating that the quality of the feature points in the area is higher;

[0077] S4.2: In the process of FAST feature point detection algorithm, the higher the local entropy value of the image block where the feature point is located, the more information the feature point contains, the more effective the detection is, and the image block is retained; conversely, the lower the local information entropy value, the worse the effect of detecting the feature point, and the image block is filtered; therefore, for each feature point, the entropy value of an image block centered on it is calculated as the information entropy value of the point; the details are as follows:

[0078] Ⅰ: According to the entropy values ​​of all N feature points calculated, the average value is calculated and then multiplied by a proportional coefficient k as the adaptive threshold (0<k≤1), as shown in the following formula:

[0079]

[0080] This adaptability dynamically adjusts the threshold according to different images and feature point distribution, avoiding the problem of insufficient or redundant feature points that may be caused by a fixed threshold;

[0081] Ⅱ: Use this threshold thresh, k = 0.8, to filter feature points. Feature points above the threshold are retained, and those below the threshold are filtered out. The larger the k value, the higher the quality of the filtered feature points. If fewer feature points are detected, adjust the k value to a smaller number, then k = 0.5, and relax the filtering conditions.

[0082] S5: Combining BEBLID descriptor and FLANN algorithm to achieve feature description and matching;

[0083] The BEBLID descriptor is a binary descriptor that uses an integral image to calculate the difference between the average grayscale values ​​of a pair of square regions in an image, selects weak classifiers using the AdaBoost algorithm, and then combines all weak classifiers to form stronger description information. The specific content of feature description using the BEBLID descriptor is as follows:

[0084] Ⅰ: After detecting feature points in the image, construct an integral image to speed up the subsequent calculation of the grayscale value of the local area;

[0085] II: For each feature point, define a set of weak classifiers. The vector of the kth weak classifier in the image is:

[0086] h k (z) = h k (z; F, T)

[0087] In the formula, z represents the input feature of the image block where the feature point is located; F is the feature extraction function (F: x→R), which is used to calculate the average gray value difference of the image block area. When the local image block x and the classification threshold T are given, we can get:

[0088]

[0089] The key to the BEBLID descriptor is the feature extraction function F(x) which is highly discriminative and fast to calculate, and is defined as the average grayscale value difference between the pixels of two image blocks, as follows:

[0090]

[0091] Where F(x) is the feature extraction function, I(q) and I(r) are the grayscale values ​​of image blocks q and r respectively, and R(p,s) is the square neighborhood with pixel p as the center and side length s;

[0092] III: BEBLID descriptor combines the output of each weak classifier and binarizes it to form a binary descriptor. By comparing F(x) with T, BEBLID converts the -1 output to 0 and the +1 output to 1, and finally obtains a binary descriptor:

[0093]

[0094] Thus, the BEBLID descriptor is generated. It enhances the local feature optimization and normalization processing, strengthens the descriptor's ability to express local features, has better robustness and efficiency, can better adapt to image changes in different scenes, and enhances the distinguishing ability of feature descriptors.

[0095] The specific content of using the FLANN algorithm for feature point matching is as follows: The FLANN algorithm searches from the nearest neighbor of the feature point to the nearest point based on the Euclidean distance, and finds the corresponding feature points by calculating the Euclidean distance of each feature point of the two images, as shown in the following formula:

[0096]

[0097] In the formula, (x i , y i ) represents the feature descriptor vector, D(x,y) is the Euclidean distance between two feature points, and i is the vector dimension; the FLANN matcher accelerates the nearest neighbor search by establishing an index-based structure and ultimately achieves image matching.

[0098] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A matching method based on the characteristics of the directional rapid rotation BRIEF algorithm, characterized in that: The steps include: S1: Input the reference image and the target image and convert them into grayscale; S2: Perform BH feature enhancement on the two grayscale images; Combining Butterworth frequency domain filtering and Haar wavelet transform methods to enhance the features of reference and target images, balance the frequency domain characteristics and local structural features of images, and highlight and enhance the visibility and contrast of image features; S3: For the reference image and the target image after BH feature enhancement, feature point detection is performed on the two images based on the FAST detection part in the ORB feature extraction algorithm; S4: For the FAST detection part in the ORB feature extraction process, a feature detection algorithm based on adaptive information entropy is introduced to detect feature points, which is used to quantify the amount of information carried by the feature points and filter out feature points with lower quality to reduce interference with the subsequent matching process; S5: Combine BEBLID descriptor and FLANN algorithm to achieve feature description and matching.

2. The matching method based on the directional rapid rotation BRIEF algorithm feature according to claim 1 is characterized in that: The specific process of S2 is as follows: S2.1: Butterworth frequency domain filtering uses frequency domain characteristics to filter high-frequency interference to enhance low-frequency characteristics, smooth the frequency response of the image, and reduce interference. The frequency response H(u,v) is calculated as follows: Where D0 is the cutoff frequency; D(u,v) is the distance from the point (u,v) in the frequency domain to the center (u0,v0) of the frequency domain; the coordinates (x,y) in the spatial domain are obtained by Fourier transform to the coordinates (u,v) in the frequency domain; S2.2: Haar wavelet transform uses two-dimensional Haar wavelet as basis function to perform wavelet transform, extract features in the image and enhance its details; Haar transform obtains the transformed approximate information A, horizontal detail information H, vertical detail information V and diagonal detail information C through low-pass and high-pass filtering in both horizontal and vertical directions, and then performs inverse wavelet transform on the wavelet coefficients, merges the approximate information and detail coefficients to reconstruct an enhanced new image and highlight the image details.

3. The matching method based on the directional rapid rotation BRIEF algorithm feature according to claim 1 is characterized in that: The specific content of S3 is: using the FAST detection algorithm to detect feature points, when there are several consecutive pixels in the neighborhood of a pixel whose grayscale value or color value is much higher or lower than that of the pixel, the pixel is considered to be a feature point; FAST performs local operations on each pixel point and does not require global calculation or traversal.

4. The matching method based on the directional rapid rotation BRIEF algorithm feature according to claim 1 is characterized in that: The specific process of detecting feature points by the feature detection algorithm based on adaptive information entropy in S4 is as follows: S4.1: After the FAST algorithm of ORB detects the feature points, the gray level at the pixel point (i, j) in the image is calculated as follows: Where p(i,j) is the gray level at the point (i,j) in the image, I ij is the probability distribution of gray level at the feature point (i, j); the information entropy H of the M×N image block where the feature point is located can be calculated using the gray level probability distribution, as shown in the following formula: The calculated information entropy value reflects the complexity or texture diversity of the local area, that is, the degree of pixel change; the larger the entropy value, the richer the texture and the greater the amount of information, indicating that the quality of the feature points in the area is higher; S4.2: In the process of FAST feature point detection algorithm, the higher the local entropy value of the image block where the feature point is located, the more information the feature point contains, the more effective the detection is, and the image block is retained; conversely, the lower the local information entropy value, the worse the effect of detecting the feature point, and the image block is filtered; therefore, for each feature point, the entropy value of an image block centered on it is calculated as the information entropy value of the point; the details are as follows: Ⅰ: According to the entropy values ​​of all N feature points calculated, the average value is calculated and then multiplied by a proportional coefficient k as the adaptive threshold (0<k≤1), as shown in the following formula: Ⅱ: Use this threshold thresh, k = 0.8, to filter feature points. Feature points above the threshold are retained, and those below the threshold are filtered out. The larger the k value, the higher the quality of the filtered feature points. If fewer feature points are detected, adjust the k value to a smaller number, then k = 0.5, and relax the filtering conditions.

5. The matching method based on the directional rapid rotation BRIEF algorithm feature according to claim 1 is characterized in that: The specific content of the feature description using the BEBLID descriptor in S5 is: Ⅰ: After detecting feature points in the image, construct an integral image to speed up the subsequent calculation of the grayscale value of the local area; II: For each feature point, define a set of weak classifiers. The vector of the kth weak classifier in the image is: h k (z)=h k (z;F,T) In the formula, z represents the input feature of the image block where the feature point is located; F is the feature extraction function (F: x→R), which is used to calculate the average gray value difference of the image block area. When the local image block x and the classification threshold T are given, we can get: The key to the BEBLID descriptor is the feature extraction function F(x) which is highly discriminative and fast to calculate, and is defined as the average grayscale value difference between the pixels of two image blocks, as follows: Where F(x) is the feature extraction function, I(q) and I(r) are the grayscale values ​​of image blocks q and r respectively, and R(p,s) is the square neighborhood with pixel p as the center and side length s; III: BEBLID descriptor combines the output of each weak classifier and binarizes it to form a binary descriptor. By comparing F(x) with T, BEBLID converts the -1 output to 0 and the +1 output to 1, and finally obtains a binary descriptor:

6. The matching method based on the directional rapid rotation BRIEF algorithm feature according to claim 1, characterized in that: The specific content of the feature point matching using the FLANN algorithm in S5 is: the FLANN algorithm searches from the nearest neighbor of the feature point to the nearest point based on the Euclidean distance, and finds the corresponding feature point by calculating the Euclidean distance of each feature point of the two images, as shown in the following formula: In the formula, (x i , y i ) represents the feature descriptor vector, D(x,y) is the Euclidean distance between two feature points, and i is the vector dimension; the FLANN matcher accelerates the nearest neighbor search by establishing an index-based structure and ultimately achieves image matching.

Images