A pest identification method incorporating attention correction

Abstract

The application discloses a pest identification method introducing attention correction, comprising the following steps: taking a target frame belonging to a background class as a query vector, taking a target frame belonging to a target class as a key value, when the similarity between the target frame of the target class and the target frame of the background class is greater than a first preset standard similarity, reclassifying the target frame of the current target class into the background class; taking the target frame belonging to the target class as the query vector, taking the target frame belonging to the background class as the key value, when the similarity between the target frame of the background class and the target frame of the target class is greater than a second preset standard similarity, exchanging the confidence of the target frame of the current background class with the confidence of the target frame of the target class with the highest similarity; step 5: outputting the target frame after optimization and correction through a layer of feedforward neural network, and predicting the position and class of the pest. In the target frame classification process, the attention mechanism is used to correct the classification of the target frame twice, effectively improving the accuracy of target detection.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically to a pest identification method that incorporates attention correction. Background Technology

[0002] In the field of computer vision, Google pioneered the introduction of the Transformer architecture. In 2020, Facebook successfully implemented a one-stage object detection method based on the Transformer architecture, proposing a complete object detection encoder architecture (DetectionTransformer, DETR). Due to long-term evolution, most pests have developed camouflage similar to their environmental background. This camouflage not only reduces the threat from predators but also makes it difficult for modern deep learning-based object detection methods to accurately identify pests in complex natural background images. Therefore, the original DETR method suffers from significant false detections. Existing research has seen some researchers improve the DETR object detection method by using clustering analysis based on scale information. Other studies have used histogram equalization in the data preprocessing stage to reduce the false detection rate.

[0003] Current methods for reducing false detection rates are not direct enough and fail to accurately and intuitively help models learn the features of objects that are prone to false detection. Summary of the Invention

[0004] This invention provides a pest identification method that incorporates attention correction to solve the technical problem in the prior art where false detections are easily made due to the high similarity between pests and the background, resulting in low detection accuracy.

[0005] This invention provides a pest identification method incorporating attention correction, comprising the following steps:

[0006] Step 1: Obtain the feature map of the original image;

[0007] Step 2: Generate several bounding boxes in the feature map of the original image, and calculate the category of each bounding box. The categories include: background class and target class.

[0008] Step 3: Use the bounding boxes belonging to the background class as query vectors and the bounding boxes belonging to the target class as keys. Then, use an attention mechanism to obtain the similarity between each bounding box of the target class and each bounding box of the background class.

[0009] When the similarity between the bounding box of the target class and the bounding box of the background class is greater than the first preset standard similarity, the bounding box of the current target class will be reclassified into the background class.

[0010] Step 4: Use the bounding boxes belonging to the target class as query vectors and the bounding boxes belonging to the background class as keys. Then, use an attention mechanism to obtain the similarity between each bounding box in the background class and each bounding box in the target class.

[0011] When the similarity between the target box of the background class and the target box of the target class is greater than the second preset standard similarity, the confidence of the current target box of the background class is swapped with the confidence of the target box of the target class with the highest similarity.

[0012] Step 5: The optimized and corrected target box is output through a feedforward neural network to predict the location and category of the pest.

[0013] Furthermore, in step 3, the query matrix for the attention mechanism is:

[0014] Querymatrix1(i)=query1(i)·W1 q

[0015] In the formula, query1(i) is the query vector corresponding to the target bounding box of the target class; W1 q The first learnable query matrix corresponding to the target bounding box of the target class;

[0016] The key-value matrix of the attention mechanism is:

[0017] Keymatrix1(j) = key1(j)·W1 k

[0018] In the formula, key1(j) is the key-value vector corresponding to the target bounding box of the background class; W1 k The first learnable key-value matrix corresponding to the target bounding box of the background class;

[0019] In step 4, the query matrix for the attention mechanism is:

[0020] Querymatrix2(i)=query2(i)·W2 q

[0021] In the formula, query2(i) is the query vector corresponding to the target box of the background class; W2 q The second learnable query matrix corresponding to the target bounding box of the background class;

[0022] The key-value matrix of the attention mechanism is:

[0023] Keymatrix2(j) = key2(j)·W2 k

[0024] In the formula, key2(j) is the key-value vector corresponding to the target bounding box of the background class; W2 kThis is the second learnable key-value matrix corresponding to the target bounding box of the target class.

[0025] Furthermore, the first learnable query matrix, the first learnable key-value matrix, the second learnable query matrix, and the second learnable key-value matrix are all k*m matrices, where k is the number of target boxes required by the Transformer decoder, and m is the dimension parameter in the Transformer architecture.

[0026] Furthermore, the initial values ​​of each element in the first learnable query matrix, the first learnable key-value matrix, the second learnable query matrix, and the second learnable key-value matrix are set using the following formula:

[0027] ω=c*(ba)+a

[0028] In the formula, a and b are the maximum and minimum values ​​in the matrix, which are set manually; c is a random number.

[0029] Furthermore, in steps 3 and 4, when performing the dot product between the query vector corresponding to the target box and the learnable query matrix, and the dot product between the key vector corresponding to the target box and the learnable key matrix, the number of elements contained in each dimension of the vector is first compared with the number of target boxes required by the Transformer's decoder.

[0030] When the number of elements in each dimension vector is less than the number of target boxes required by the Transformer's decoder, zeros are padded in the one-dimensional vector until the number of elements is the same as the number of target boxes required.

[0031] When the number of elements in each dimension vector exceeds the number of target boxes required by the Transformer's decoder, the last element is removed from the one-dimensional vector until the number of elements is the same as the required number of target boxes.

[0032] Furthermore, the procedure before performing step 3 includes:

[0033] The first preset standard similarity, the first learnable query matrix, and the first learnable key-value matrix are optimized.

[0034] Furthermore, the procedure before performing step 4 includes:

[0035] The second preset standard similarity, the second learnable query matrix, and the second learnable key-value matrix are optimized.

[0036] Furthermore, the optimization method is as follows: optimization is performed using the Adam optimizer.

[0037] The beneficial effects of this invention are:

[0038] This invention employs an attention mechanism to correct the classification of target bounding boxes twice during the target bounding box classification process. This correction scheme effectively improves the detection accuracy, especially when the target has camouflage similar to the environment, where the accuracy is far higher than traditional target detection methods.

[0039] This invention employs a multi-scale feature fusion method, which can extract features of large and small targets in a more refined manner.

[0040] In this invention, a learnable key-value matrix and query matrix are used, which allows the model to learn the feature information of target boxes that are easy to misdetect during multiple iterations of calculation and helps the model to make corrections, thereby improving the accuracy of recognition. Attached Figure Description

[0041] The features and advantages of the invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the invention in any way. In the drawings:

[0042] Figure 1 This is a schematic diagram of the detection process according to a specific embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram of multi-scale feature fusion in a specific embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the attention mechanism in a specific embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram of the target box category correction method in a specific embodiment of the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] This invention provides a pest identification method incorporating attention correction, such as... Figure 1 As shown, it includes the following steps:

[0048] Step 1: Obtain the feature map of the original image;

[0049] The original image is input into three convolutional neural networks at three different scales to extract features from the original image at multiple scales.

[0050] like Figure 2 As shown, the input RGB format image is first converted into tensor form, and then the input information is processed using ResNet50 networks with three convolution kernel sizes of 9*9, 6*6, and 3*3. At the same time, feature information of large, medium, and small receptive fields is extracted. After feature extraction, the average of the three extracted results is calculated to complete the fusion of large, medium, and small receptive field features.

[0051] Using X = {x i j ∈R H×W×C |i∈[1,N],j∈[1,C]} represents the feature tensor of all images in a batch.

[0052] Where x represents the tensor information of each image after conversion; H, W, and C represent the height, width, and number of channels of each image, respectively; and N represents the number of images in each batch.

[0053] Inputting three convolutional neural networks of different scales yields three feature tensors representing image features.

[0054]

[0055] Feature fusion will be performed on feature information extracted from neural network models at multiple scales;

[0056] The feature tensor Z, which integrates multi-size features, is obtained by calculating the mean of the three tensors.

[0057]

[0058] Transform the feature information in tensor form into vector form;

[0059] First, reduce the dimensionality to make it a three-dimensional tensor Z' = Z. reshape The input vector (B, C, H × W) is then subjected to mean pooling to obtain the feature vector x.

[0060]

[0061] The original image is encoded with absolute position, and the encoded result is added to the feature information in vector form;

[0062] Perform absolute position encoding on each position (i,j) in the image:

[0063]

[0064] The position feature vector is added to the pixel feature vector, and the encoder performs feature extraction based on the feature vector to extract its feature map.

[0065] The encoder structure is shown in the diagram. The feature vector is input into the encoder to obtain a feature vector z of size N×d representing the image's feature information. The encoder calculates the attention using the following formula:

[0066]

[0067] Where Q and K are the learnable query vector and key-value pair, respectively;

[0068] In this step, the complete formula for calculating the eigenvector is:

[0069]

[0070] Step 2: Generate several bounding boxes in the feature map of the original image. Use the bounding boxes generated by the query vector to query the feature vector generated by the encoder in the decoder, and calculate the matrix y = Decoder(z,Q) containing the bounding boxes and target categories. The decoder output y is a matrix of size N*(C+5), where N is the number of bounding boxes, C is the number of categories, each row corresponds to one target, the first C+1 columns are the similarity between the bounding box and each class, the extra class represents the background, and the last 4 numbers represent the position information of each bounding box (center coordinates, x, y, width and height w, h).

[0071] Steps 3 and 4 involve two modifications to the target bounding box category, where, for example... Figure 3 As shown, the matrix in the attention mechanism has also been optimized.

[0072] Step 3: Optimize the first preset standard similarity, the first learnable query matrix, and the first learnable key-value matrix using the Adam optimizer;

[0073] The target bounding boxes belonging to the background class are used as query vectors, and the target bounding boxes belonging to the target class are used as keys. An attention mechanism is used to obtain the similarity between each target bounding box and each background bounding box.

[0074] When the similarity between the bounding box of the target class and the bounding box of the background class is greater than the first preset standard similarity, the bounding box of the current target class will be reclassified into the background class.

[0075] The query matrix for the attention mechanism is as follows:

[0076] Querymatrix1(i)=query1(i)·W1 q

[0077] In the formula, query1(i) is the query vector corresponding to the target bounding box of the target class; W1q The first learnable query matrix corresponding to the target box of the target class is initialized by Xavier. As learning progresses, the optimizer can optimize it multiple times so that the model can learn the false detection query features of the background box through this matrix.

[0078] The key-value matrix of the attention mechanism is:

[0079] Keymatrix1(j) = key1(j)·W1 k

[0080] In the formula, key1(j) is the key-value vector corresponding to the target bounding box of the background class; W1 k This is the first learnable key-value matrix corresponding to the target box of the background class. It is initialized by Xavier. As learning progresses, the optimizer can optimize it multiple times so that the model can learn the false detection key-value features of the background box through this matrix.

[0081] The similarity scores between the background bounding box and all target bounding boxes are calculated using the formula α(i)(j)=dot(Querymatrix1(i),Keymatrix1(j)). These similarity scores are then compared to a first preset standard similarity P1. All similarities greater than P1 are calculated, along with their corresponding target and background bounding boxes, and the classification information of the target bounding boxes is then corrected. For example... Figure 4 As shown: In the confidence list during the object detection process, the confidence of the wrong category is swapped with the confidence of the background category. This is how the process of correcting the misclassified object box is completed.

[0082] Step 4: Optimize the second preset standard similarity, the second learnable query matrix, and the second learnable key-value matrix using the Adam optimizer.

[0083] The bounding boxes belonging to the target class are used as query vectors, and the bounding boxes belonging to the background class are used as keys. An attention mechanism is used to obtain the similarity between each bounding box of the background class and each bounding box of the target class.

[0084] When the similarity between the target box of the background class and the target box of the target class is greater than the second preset standard similarity, the confidence of the current target box of the background class is swapped with the confidence of the target box of the target class with the highest similarity.

[0085] The query matrix for the attention mechanism is as follows:

[0086] Querymatrix2(i)=query2(i)·W2 q

[0087] In the formula, query2(i) is the query vector corresponding to the target box of the background class; W2q The second learnable query matrix corresponding to the target bounding box of the background class;

[0088] The key-value matrix of the attention mechanism is:

[0089] Keymatrix2(j) = key2(j)·W2 k

[0090] In the formula, key2(j) is the key-value vector corresponding to the target bounding box of the background class; W2 k This is the second learnable key-value matrix corresponding to the target bounding box of the target class.

[0091] The similarity score between the target class and all background classes is calculated using the formula β(a)(b)=dot(Querymatrix2(a),Keymatrix2(b)). Then, all similarities greater than the second preset standard similarity P2 and their corresponding target and background bounding boxes are calculated. The classification information of the background bounding box is corrected by reclassifying it to the same category as the target bounding box with excessively high similarity in the second correction. In the confidence list during the target detection process, the confidence scores of the background class and the target class that are too similar are swapped. This method completes the correction process for misclassified background bounding boxes.

[0092] Step 5: The optimized and corrected target box is output through a feedforward neural network to predict the location and category of the pest.

[0093] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A pest recognition method introducing attention correction, characterized by, The method comprises the following steps: Step 1: obtaining a feature map of an original image; Step 2: generating a plurality of target boxes in the feature map of the original image, and calculating the category to which each target box belongs, wherein the category comprises a background category and a target category; Step 3: taking the target box belonging to the background category as a query vector and taking the target box belonging to the target category as a key value, and obtaining the similarity between the target box of each target category and the target box of each background category through an attention mechanism, when the similarity between the target box of the target category and the target box of the background category is greater than a first preset standard similarity, reclassifying the target box of the current target category into the background category; Step 4: taking the target box belonging to the target category as a query vector and taking the target box belonging to the background category as a key value, and obtaining the similarity between the target box of each background category and the target box of each target category through an attention mechanism, when the similarity between the target box of the background category and the target box of the target category is greater than a second preset standard similarity, exchanging the confidence of the target box of the current background category with the confidence of the target box of the target category with the highest similarity; Step 5: outputting the target box after optimization and correction through a layer of feedforward neural network to predict the position and category of the pest.

2. The pest recognition method incorporating attention revision as claimed in claim 1, wherein, In the step 3, the query matrix of the attention mechanism is: Querymatrix1(i) = query1(i) • W1 q In the formula, query1(i) is a query vector corresponding to the target frame of the target class; W1 q is the first learnable query matrix corresponding to the target frame of the target class; The key value matrix of the attention mechanism is: Keymatrix1(j) = key1(j) • W1 k In the formula, key1(j) is a key value vector corresponding to the target frame of the background class; W1 k is the first learnable key value matrix corresponding to the target frame of the background class; In the step 4, the query matrix of the attention mechanism is: Querymatrix2(i) = query2(i) • W2 q In the formula, query2(i) is a query vector corresponding to the target frame of the background class; W2 q is the second learnable query matrix corresponding to the target frame of the background class; The key value matrix of the attention mechanism is: Keymatrix2(j) = key2(j) - W2 k In the formula, key2(j) is a key value vector corresponding to the target frame of the background class; W2 k is a second learnable key value matrix corresponding to the target frame of the target class.

3. The pest recognition method incorporating attention revision of claim 1, wherein, The first learnable query matrix, the first learnable key value matrix, the second learnable query matrix and the second learnable key value matrix are all k*m size matrices, wherein k is the number of target boxes required by the decoder of the Transformer, and m is a dimension parameter in the Transformer architecture.

4. The pest recognition method incorporating attention revision of claim 1, wherein, The initial values of the elements in the first learnable query matrix, the first learnable key value matrix, the second learnable query matrix and the second learnable key value matrix are set through the following formula: ω=c*(b-a)+a In the formula, a and b are the maximum and minimum values in the matrix artificially set, and c is a random number.

5. The pest recognition method incorporating attention revision as claimed in claim 2, wherein, In the steps 3 and 4, when the query vector corresponding to the target box is multiplied by the learnable query matrix and the key value vector corresponding to the target box is multiplied by the learnable key value matrix, first, compare the number of elements contained in each one-dimensional vector with the number of target boxes required by the decoder of the Transformer, when the number of elements contained in each one-dimensional vector is less than the number of target boxes required by the decoder of the Transformer, supplement 0 in the one-dimensional vector until the number of elements is the same as the number of required target boxes; when the number of elements contained in each one-dimensional vector is greater than the number of target boxes required by the decoder of the Transformer, delete the last element from the one-dimensional vector until the number of elements is the same as the number of required target boxes.

6. The pest recognition method introducing attention correction according to claim 1, characterized by, Before the step 3 is performed, the first preset standard similarity, the first learnable query matrix and the first learnable key value matrix are optimized. Before the step 4 is performed, the second preset standard similarity, the second learnable query matrix and the second learnable key value matrix are optimized.

7. The pest recognition method incorporating attention revision as claimed in claim 1, wherein, ​ ​ 8. The pest recognition method incorporating attention revision as claimed in claim 6 or 7, wherein, The optimization method is: optimization by an Adam optimizer. The optimization method is: optimization by an Adam optimizer.

Images