A feature descriptor based bounding box matching method

Abstract

The application discloses a kind of deep learning prediction frame matching methods based on feature descriptor, it is related to the identification and matching technology of target object in image recognition field.The present application determines the support area of deep learning identification prediction frame, obtains the feature descriptor with scale invariance by constructing regional gradient, and the recognition result is matched using the formed feature vector.Simulation results show that the present application has strong robustness to translation, rotation, scaling and scale transformation, and has important significance for the development of target tracking, visual navigation and other fields.

Description

Technical Field

[0001] This invention pertains to the identification and matching technology of target objects in the field of image recognition, specifically relating to the fields of image recognition and image feature matching based on artificial intelligence, and more specifically to a prediction box matching method based on feature descriptors. Background Technology

[0002] With the development of artificial intelligence, deep learning networks are widely used in image recognition, and deep learning-based artificial intelligence technology is one of the future development directions of image recognition. Feature extraction networks are used to mine deep features in images, effectively improving the detection rate of target objects. However, recognition networks struggle to match target objects and lack robustness under large-scale viewpoint changes.

[0003] Therefore, how to design an algorithm that can match the recognition results is a problem that industry professionals urgently need to solve. Summary of the Invention

[0004] The technical problem solved by this invention is the difficulty of target matching in current recognition networks. By combining artificial intelligence recognition networks with feature descriptor matching algorithms, a recognition prediction box matching algorithm based on feature descriptors is proposed.

[0005] This invention proposes a prediction box matching algorithm based on feature descriptors, comprising selecting the frame support range, generating feature descriptors, and matching feature descriptors. First, the image is downsampled and Gaussian blurred to construct a scale-space pyramid. The scale-space pyramid aims to simulate the multi-scale features of image data. The prediction box for each image in the pyramid is described, and the smallest circumcircle of the prediction box is selected as the frame support range, ensuring that the pixel information contained in the sampled area remains unchanged after image rotation. This guarantees the scale invariance of the described prediction box. Then, the pixel gradient of the support range is used to determine the main direction of the descriptor. Starting from the main direction, the fan-shaped neighborhood is evenly divided into eight sectors, and the support range is divided into eight equal sectors (…). Each sector is a sub-region of the FSR. Each sub-region is treated as a component vector. The grayscale gradient projection values ​​of all pixels within the sub-region are calculated according to the eight directions from 0° to 315° in the image coordinates, forming 8*8 feature vectors, which are then normalized according to the principal direction. A Gaussian weighted operation is performed on the gradient of each pixel to reduce the influence of noise from peripheral points on the feature values, forming a 64-dimensional vector descriptor. The 64-dimensional vector is then normalized for brightness to reduce the impact of illumination changes. Target matching is achieved by comparing the predicted bounding box descriptors of two points. The similarity metric for the 128-dimensional descriptors is represented by Euclidean distance. Predicted bounding boxes are iteratively obtained from the previous frame image. For each predicted bounding box in the previous frame image, a corresponding predicted bounding box is matched in the next frame image. A threshold for the Euclidean distance between descriptors is then set, and the similarity of the predicted bounding boxes extracted from the two images is determined based on this threshold.

[0006] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0007] This invention proposes a prediction box matching method based on feature descriptors, which provides a solution to the problem that recognition networks have difficulty matching target objects and have low robustness under large-scale viewpoint changes.

[0008] The method proposed in this invention is invariant to image scaling, translation and rotation transformations, and illumination matching, and is independent of the size of the prediction box. Attached Figure Description

[0009] By referring to the accompanying drawings and reading the specific embodiments, readers will gain a clearer understanding of all aspects of the present invention.

[0010] Figure 1 The overall flowchart of the algorithm of the present invention is shown.

[0011] Figure 2 This is a schematic diagram of the supporting area.

[0012] Figure 3 To support the relationship between the number of region blocks and the correct matching rate.

[0013] Figure 4 This is a schematic diagram of the supporting area division.

[0014] Figure 5 This is a schematic diagram of the weights of the support region for the prediction box.

[0015] Figure 6 This is the gradient histogram of the feature descriptors.

[0016] Figure 7 This is a diagram showing the matching results under scaling transformation.

[0017] Figure 8 This is a diagram showing the matching results under rotation transformation.

[0018] Figure 9 This is a diagram showing the matching results under translation transformation. Detailed Implementation

[0019] Figure 1 A general flowchart of the algorithm of this invention is shown. (Refer to...) Figure 1 The specific embodiments of the present invention will be described in further detail below.

[0020] Reference Figure 1 Firstly, in the process of determining the supporting region, the image is downsampled and Gaussian blurred to construct a scale-space pyramid. The purpose of the scale-space pyramid is to simulate the multi-scale features of image data, describing the predicted bounding box for each image in the pyramid, thereby ensuring that the described predicted bounding box has scale invariance. For example... Figure 2 As shown, the smallest circumcircle of the prediction box is used as the support region for the prediction box. When the corresponding region of this shape undergoes a small proportional change, the corresponding pixels that maintain a larger proportion compared to the rectangle are the corresponding pixels, and the area of ​​the changed region is the same. During feature descriptor generation: when generating descriptors for information within the prediction box, blurring is first performed to reduce the impact of noise on matching. To avoid destroying local information, a Gaussian filtering algorithm is used instead of mean filtering; subsequently, a two-dimensional normally distributed Gaussian function is used when weighting pixels.

[0021] (1)

[0022] At each pyramid level, the gradient and orientation distribution features of pixels within the support region of their respective prediction bounding box are collected. A reference orientation is assigned to each prediction bounding box using the pixel gradients of the support region. To achieve complete rotation invariance, the gradient and orientation of a point are adaptively adjusted according to its position when calculating the gradient. For bivariate functions... At point gradient in the horizontal direction gradient in the vertical direction It can be represented as:

[0023] (2);

[0024] (3);

[0025] Then the image at the pixel point gradient magnitude With direction They are respectively:

[0026] (4)

[0027] (5)

[0028] After calculating the gradient for all pixels, the gradient magnitudes need to be summed according to the divided gradient direction intervals to construct the gradient histogram. If the gradient direction division granularity is too fine, the computational cost of constructing the descriptor will increase, and the matching will be more susceptible to noise interference; conversely, if the gradient division granularity is too coarse, image detail features will be lost, also resulting in poor matching performance. Figure 3 The relationship between the number of blocks and matching accuracy is shown. When the gradient direction is divided into 8 intervals of 45 degrees each, a good matching effect can be obtained with less computation. Figure 4 This is a schematic diagram of the support area division when there are 8 zones.

[0029] For the i-th directional interval, the sum of gradient magnitudes can be expressed as:

[0030] (6)

[0031] in, is a weighting coefficient, representing the magnitude of the pixel gradient amplitude in interval i. The calculation process is shown in formula (7). When the image rotates, the edge information of the perceptual region will change accordingly. Conversely, pixels closer to the center region can remain relatively stable. Therefore, according to the pixel position, a corresponding weighting coefficient is assigned when summing the gradient amplitude.

[0032] (7)

[0033] For the rectangular sensing region, it is divided into a 4*4 grid, and the weight coefficient of each grid is obtained according to the Gaussian function shown in formula (1). The result is as follows. Figure 5 As shown. Further, combining the approximate target perception using the circumscribed circle from step one, the network weight coefficients are adjusted based on the network coverage relationship between the circle and the rectangular frame. Its circumscribed circle support area is shown below. Figure 2 As shown. Based on the network weight coefficient grid, the sum of gradient magnitudes in each directional interval is calculated, and a gradient histogram is constructed, where the horizontal axis represents the eight gradient direction intervals, and the vertical axis represents the sum of gradient magnitudes within the corresponding interval. Then, the directional interval with the largest sum of gradient magnitudes is selected as the main directional interval, and descriptors are constructed by traversing all intervals counterclockwise from this interval. For Figure 6 The histogram shown, where the k-th interval is the main direction interval, can be represented by the following descriptor:

[0034] (8)

[0035] When an image scales, the overall number of pixels changes; when brightness changes, the overall pixel values ​​change. However, the proportion of the sum of gradient magnitudes in each directional interval remains relatively stable. Therefore, the descriptor D is normalized. Let the maximum value in D be... The minimum value is Then the normalized descriptor D can be represented as:

[0036] (9)

[0037] Finally, the predicted bounding boxes in the two images are matched, and a set of predicted bounding box descriptors is established using feature descriptors. Target matching is accomplished by comparing the predicted bounding box descriptors in two point sets. The similarity measure of the 128-dimensional descriptors is represented by Euclidean distance. Predicted bounding boxes are iteratively obtained from the previous frame image, and for each predicted bounding box in the previous frame image, the corresponding predicted bounding box is matched in the subsequent frame image. Next, a threshold for the Euclidean distance between descriptors is set. Considering the differences in vector lengths between different descriptors, it is difficult to set a general threshold; therefore, relative distance is used to measure the matching effect. For the descriptors... When performing a match, if its best match is The condition for a match to be valid is:

[0038] (10)

[0039] A threshold is used to determine whether the predicted bounding boxes extracted from two images are similar. Simultaneously, the nearest neighbor ratio method is used to reduce false matches, and the matching results are as follows: Figure 7-9 As shown, the correct matching rate was 83.05% and the mismatch rate was 1.69% under translational changes; the correct matching rate was 81.36% and the mismatch rate was 6.78% under rotational changes; and the correct matching rate was 89.25% and the mismatch rate was 2.94% under scaling changes.

[0040] In summary, the method described in this invention performs bounding box matching on images identified by a deep learning network. As can be seen from the final results, the proposed bounding box matching method based on feature descriptors achieves bounding box matching, which is of great significance for target recognition, target tracking, and visual navigation.

[0041] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above content to create equivalent embodiments for application in other fields. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for predicting bounding boxes based on feature descriptors, characterized in that, include: Step A: Select the support area of ​​the prediction box; The circular region with the diagonal of the prediction box as its diameter and the intersection point as its center is used as the prediction box support region (FSR). The FSR contains all pixel information inside the prediction box. Step B: Generate feature descriptors from the extracted prediction boxes, including the following steps: B1. Construct a box descriptor in the support region; B11 proposes a method for constructing descriptors within a circular support region using sector-shaped region segmentation. When generating descriptors from information within the predicted bounding box, blurring is first performed to reduce the impact of noise on matching. To avoid destroying local information, a Gaussian filtering algorithm is used instead of mean filtering. Finally, a two-dimensional normally distributed Gaussian function is used when weighting pixels. ； At each pyramid level, the gradient and orientation distribution features of pixels within the support region of their respective prediction bounding boxes are collected, and the highest gradient and orientation of the support region pixels are used as the principal orientation of the prediction bounding box. To achieve complete rotation invariance, the gradient and orientation of a point are adaptively adjusted according to its position when calculating the gradient. For binary functions... At point gradient in the horizontal direction gradient in the vertical direction It can be represented as: ； ； Then the image at the pixel point gradient magnitude With direction They are respectively: ； ； Calculate the lateral gradient of each pixel separately. and Then, gradient vectors are synthesized according to the formula in B11 to obtain the gradient direction and magnitude of the pixel. B12. After calculating the gradient for all pixels, the gradient magnitudes need to be summed according to the divided gradient direction intervals to construct the gradient histogram. If the gradient direction division granularity is too fine, the computational cost of constructing the descriptor will increase, and the matching will be more susceptible to noise interference. Conversely, if the gradient division granularity is too coarse, image detail features will be lost, also resulting in poor matching performance. Experiments have shown that dividing the gradient direction into 8 intervals of 45 degrees each yields better matching results with less computation. For the i-th directional interval, the sum of gradient magnitudes can be expressed as: ， in, The weighting coefficient represents the magnitude of the pixel gradient's influence in the i-th interval, and is normalized by a Gaussian function. The calculation shows that when an image undergoes rotation, the gradient magnitude shifts more towards the central region due to the weighting coefficients. Gradient formation; ， Step C: Perform bounding box matching on the generated feature descriptors.

2. The method for determining the support region according to claim 1, characterized in that: A1. The smallest outer circle of the prediction box is used as the support area of ​​the prediction box. When the descriptor of this shape rotates in its corresponding area, each sector can retain a larger proportion of the initial pixels compared to the rectangle, and the area of ​​the changed area is the same. A2. The circular sampling region uses Gaussian weighting to reduce the influence of the outer sub-regions, thus making the descriptor scale invariant. Under the effect of the Gaussian pyramid, the descriptor also has scale invariance.

3. The matching method according to claim 1: In step C, matching based on descriptors includes the following steps: C1. Establish a set of prediction box descriptors using feature descriptors; C2. The similarity measure for 128-dimensional descriptors is represented by Euclidean distance. Predicted bounding boxes are iteratively obtained from the previous frame image. For each predicted bounding box in the previous frame, the corresponding predicted bounding box is matched in the next frame. Secondly, a threshold for the Euclidean distance between descriptors is set. Considering the differences in vector lengths between different descriptors, it is difficult to set a general threshold; therefore, relative distance is used to measure the matching effect. When performing a match, if its best match is The condition for a match to be valid is: 。

Images