Two-stage insect fine granularity identification method based on key part cooperative localization

Abstract

The invention discloses an insect two-stage fine granularity identification method based on key part cooperative localization. According to the method, the relative size of a key part is increased by means of a detection model, the effective characteristics of a target insect can be better identified in a multi-scale range, and the identification capability of easily confused varieties is improved; meanwhile, global context information is dynamically adjusted and fused through a global context enhancement module (GCE), so that feature details of insects can be better captured when the model processes a complex background, and the classification robustness is further improved; and a collaborative mechanism is introduced to carry out comprehensive calculation on the result, so that the comprehensiveness and accuracy of the classification result are ensured.

Description

Technical Field

[0001] The invention belongs to the technical field of agricultural image detection and classification, and specifically relates to a two-stage insect fine-grained recognition method based on key part collaborative positioning. Background Art

[0002] With the increasing complexity of ecosystems and the acceleration of globalization, accurate identification of insect species is particularly important as they are key components of ecosystems. Accurate insect classification not only helps to gain a deeper understanding of the structure and function of biodiversity, but also plays an indispensable role in preventing and controlling invasive alien species and maintaining ecological balance. Among them, image-based insect identification has gradually become the mainstream due to its simplicity, speed, and ease of operation. With the development of deep learning models such as convolutional neural networks (CNNs), the efficiency and accuracy of classification have been greatly improved, and the labor cost has been significantly reduced, adapting to the needs of large-scale data processing.

[0003] In order to more accurately identify insect images of similar species, fine-grained recognition models and methods are introduced to improve the accuracy of insect recognition (Loris et al. integrated CNN networks based on different topological structures to identify small Deng pest datasets in 2021, Md et al. constructed a convolutional neural network based on the ResNet152V2 architecture to classify field pest insect images, and Zhu et al. added an attention mechanism to the backbone network of yolov5 in 2022). However, these methods cannot fully and effectively utilize the local features of the target object, and cannot capture the characteristic details of the target object well when the background of the image is more complex. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and propose a two-stage fine-grained insect recognition method based on the collaborative positioning of key parts. The method uses a detection model (a fine-grained recognition model in that stage) to increase the relative size of key parts, which can better identify the effective characteristics of target insects in a multi-scale range and improve the recognition ability of easily confused varieties; at the same time, the global context enhancement module is used to dynamically adjust and fuse the global context information, so that the model can better capture the characteristic details of insects when processing complex backgrounds, thereby further improving the robustness of classification; a collaborative mechanism is introduced to comprehensively calculate the results, ensuring the comprehensiveness and accuracy of the classification results.

[0005] In order to solve the above technical problems, the technical solution of the present invention is:

[0006] A two-stage fine-grained insect recognition method based on key parts collaborative positioning includes the following steps:

[0007] Step (1): Dataset acquisition and processing;

[0008] According to the recognition requirements, the insect data set is obtained from the Internet and on-site collection methods, and then the data set is labeled. The labeling method is to label the entire target object and key parts.

[0009] Step (2): construct a two-stage fine-grained recognition model;

[0010] The two-stage fine-grained recognition model consists of a target detection network and a classification network; the target detection network shown is used to extract images of key parts, and the classification network is used to classify images of key parts.

[0011] Step (3): First, the input insect image is detected through the target detection network to obtain the position information of the key parts of the insect to be identified. After obtaining the position information of each key part of the insect to be identified, the image is cropped according to the position information to obtain a sub-image containing only a single key part. Then, the cropped images are preprocessed, including resizing and normalization. Then, HOG feature extraction is performed on the preprocessed images and the areas whose similarity exceeds the set threshold are filtered out to obtain the target image.

[0012] Step (4): Pass the target image obtained in step (3) to the classification network to identify the insect category.

[0013] First, the target image obtained in step (3) is processed by the classification network, and the output of the classification network is concatenated with the center point position information of the key parts of the insect to be identified. The concatenated feature vector is input into the fully connected layer, and through further feature processing, an enhanced feature vector containing spatial position information and category information is finally generated, that is, the classification result of the key parts is obtained. The classification network adjusts and fuses the global context information through the global context enhancement module. The adjusted context feature map is added element by element to the input feature map, thereby improving the model's understanding and response capabilities to the overall scene.

[0014] Step (5): Finally, the classification results of each part are integrated and calculated through a collaborative mechanism to generate the final overall classification output.

[0015] Preferably, the target detection method of the target detection network is:

[0016] The target detection network includes an image encoder and a decoder. The image encoder uses a ResNet50 model to extract high-level features from the original input image; first, features are obtained from the original input image through multi-layer convolution, and then the features are processed by the activation function ReLU to obtain an image feature map, and then the spatial size of the feature map is reduced by a pooling layer. Then, the target area RoI is cropped in the feature map extracted by the image encoder using a Gaussian noise box. Finally, the target area RoI and the Gaussian noise box are input into the decoder together to extract the multi-scale information of the target, and prediction is performed through a convolution layer with a convolution kernel size of 1*1. The convolution layer is divided into a regression layer and a classification layer. The function of the regression layer is to predict the position information of the bounding box of the target object, and the number of channels is 4. The function of the classification layer is to predict the category of the key part to which each bounding box belongs. For each bounding box, the classification layer outputs a vector whose length is equal to the number of categories in the data set, and each position corresponds to the probability distribution of a category. After being processed by the softmax function, the probability distribution of each category is obtained, and the number of channels is equal to the number of categories.

[0017] The method for obtaining the Gaussian noise frame is as follows: for any given input feature map C×H×W, the number of input channels is 3, and Gaussian noise controlled by variance scheduling is added to the input image according to the rule of adding noise in the diffusion detection model, thereby obtaining 500 Gaussian noise frames. The noise frame is obtained using the following formula:

[0018] x t ~q(x t |x0)(1)

[0019]

[0020] Where t is the time step of adding noise, x t represents the noise frame at time t, α t is a hyperparameter, ε t-1 represents the noise at time t-1, and x0 represents the real target detection box during training.

[0021] Preferably, the network structure of the image encoder of the target detection network is as follows: the input image first passes through a normal convolution layer with a convolution kernel size of 7*7, 64 channels and a step size of 2; a maximum pooling layer with a kernel size of 3*3 and a step size of 2; a BTNK1 module with a step size of 1 and 64 channels; and 2 BTNK2 modules with 256 channels; then passes through a BTNK1 module with a step size of 2, an input channel of 256 and an output channel of 128; 3 BTNK2 modules with a channel of 512; then outputs to a BTNK1 module with a step size of 2, an input channel of 512 and an output channel of 256; 5 BTNK2 modules with a channel of 1024; finally, it passes through a BTNK1 module with a step size of 2, an input channel of 1025 and an output channel of 512; and 2 BTNK2 modules with a channel of 2048, and outputs a feature map with a shape of (2048, 7, 7).

[0022] Preferably, in the image encoder, the BTNK1 module has four parameters: C, W, C1, S; wherein S: represents the step size in the convolution layer. When S is 1, the input size and the output size are the same, indicating that no downsampling is performed. C1: represents the number of feature maps output by the convolution layer, that is, the number of output channels. C: represents the number of input channels. W: represents the input size, that is, the length and width. The BTNK1 module divides the input image into two paths for processing. First, the first path processes the input image through a convolution layer with a convolution kernel size of 1*1, a channel number of C1, and a step size of S, and then enters the BN layer and the RELU activation function; then enters a convolution layer with a convolution kernel size of 3*3, a channel number of C1, and a step size of 1, and then enters the BN layer and the RELU activation function; finally enters a convolution layer with a convolution kernel size of 1*1, a channel number of C1*4, and a step size of S, and then enters the BN layer. The second path processes the input image through a convolution layer with a kernel size of 1*1, a channel number of C1*4, and a step size of S, and then enters the BN layer. The output results of the two paths are then added together and activated by the RELU function to output an image with a shape of (C1*4, W / S, W / S).

[0023] The BTNK2 module has two parameters: C and W; C: represents the number of input channels. W: represents the input size, i.e., length and width. The BTNK2 module processes the input image in two paths. First, the first path processes the input image through a convolution layer with a convolution kernel size of 1*1, a number of channels of C / 4, and a stride of 1, and then enters the BN layer and the RELU activation function; then enters a convolution layer with a convolution kernel size of 3*3, a number of channels of C / 4, and a stride of 1, and then enters the BN layer and the RELU activation function; finally, it enters a convolution layer with a convolution kernel size of 1*1, a number of channels of C, and a stride of 1, and then enters the BN layer. The second path directly outputs the input image. Then, the output results of the two paths are added together, and then output as an image with a shape of (C, W, W) through the RELU activation function.

[0024] Preferably, the network structure of the classification network is as follows: the global image and the local image are first input into the global context enhancement module to obtain a fused feature map, and then pass through a normal convolution layer with a convolution kernel size of 3*3, a number of channels of 3, and a step size of 2; an MBConv1 module with a step size of 1; an MBConv6 module with a convolution kernel size of 3*3, a number of channels of 3, and a step size of 2; an MBConv6 module with a convolution kernel size of 5*5, a number of channels of 3, and a step size of 2; an MBConv6 module with a convolution kernel size of 3*3, a number of channels of 3, and a step size of 3; an MBConv6 module with a convolution kernel size of 5*5, a number of channels of 3, and a step size of 3; an MBConv6 module with a convolution kernel size of 5*5, a number of channels of 3, and a step size of 3; an MBConv6 module with a convolution kernel size of 5*5, a number of channels of 3, and a step size of 4; an MBConv6 module with a convolution kernel size of 3*3, a number of channels of 3, and a step size of 1; finally, pass through a normal convolution layer with a convolution kernel size of 1*1, a pooling layer, and a fully connected layer.

[0025] Preferably, in the classification network, the specific structure of the global context enhancement module is as follows: first, the global feature map passes through a common convolution layer with a convolution kernel size of 1*1 and a ReLU activation function, and then is added element by element with the local feature map, and then passes through a common convolution layer with a convolution kernel size of 1*1 and a ReLU activation function to obtain an output fused feature map.

[0026] Preferably, in the classification network, the MBConv1 module with a step size of 1 specifically operates as follows: first, the input fusion feature map is subjected to a dimensionality-increasing convolution with a convolution kernel size of 1*1, a BN layer and a Swish activation layer, then a depth-separable convolution with a convolution kernel size of k*k and a step size of 1, a BN layer and a Swish activation layer, then a SE module and a dimensionality-reducing convolution with a convolution kernel size of 1*1 and a BN layer, then a Dropout layer, and finally a shortcut is performed with the feature map initially input by MBConv1 to obtain the module output;

[0027] The specific operation of the MBConv6 module with a step size of 1 is as follows: first, the input feature map is subjected to a dimensionality-increasing convolution with a convolution kernel size of 1*1, a BN layer and a Swish activation layer, then a depth-wise separable convolution with a convolution kernel size of k*k and a step size of 1, a BN layer and a Swish activation layer, then a SE module and a dimensionality-reducing convolution with a convolution kernel size of 1*1 and a BN layer, then a Dropout layer, and finally a shortcut with the feature map originally input to MBConv6 to obtain the module output;

[0028] The specific operation of the MBConv6 module with a stride of 2 is as follows: first, the input feature map passes through a dimensionality-increasing convolution with a convolution kernel size of 1*1, as well as a BN layer and SiLU, then passes through a depthwise separable convolution with a convolution kernel size of 3*3 and a stride of 2, as well as a BN layer and SiLU, then passes through the SE module, then passes through a dimensionality-reducing convolution with a convolution kernel size of 1*1 and a BN layer, and finally obtains the module output through the Dropout operation.

[0029] The SE module consists of a global average pooling and two fully connected layers, where the number of nodes in the first fully connected layer is 1 / 4 of the number of channels of the feature map of the corresponding MBConv module input, and the Swish activation function is used, and the number of nodes in the second fully connected layer is the number of channels of the feature map output by the depthwise separable convolution in the corresponding MBConv module, and the Sigmoid activation function is used.

[0030] As a preference, the final classification result is calculated through a collaborative mechanism. First, the two-stage fine-grained recognition model obtains the category information of the key parts of the insect to be identified and the insect as a whole and the corresponding classification scores. Next, a weight is randomly assigned to each part, and the comprehensive score of each category is calculated. Specifically, the comprehensive score formula of the category is as follows:

[0031]

[0032] Among them, s i,c represents the score of the ith part for category c, w i Represents the weight of the ith part. After calculating the comprehensive scores of all categories, the category with the highest score is selected as the final prediction category.

[0033] Preferably, during training, if the final predicted label obtained by training is inconsistent with the true label, it is necessary to calculate the loss value, using the cross entropy loss function, whose formula is:

[0034]

[0035] Where: C is the number of categories; y cis an indicator of the true category (i.e., y c =1, otherwise y c =0); is the probability that the model predicts class c.

[0036] Next, the gradient calculation is performed to adjust the weight. The gradient calculation formula is as follows:

[0037]

[0038] in, represents the score of the predicted category c on part i, y c Represents the score of the true category c on part i.

[0039] Finally, the weights are updated by gradient descent, and the process is repeated until the predicted category output by the model matches the true label. If the labels of all parts do not match the true labels, the output is classified incorrectly.

[0040] As a preferred method, in order to better evaluate the model, three levels of evaluation indicators were introduced: average detection accuracy, classification accuracy, and overall classification accuracy to comprehensively measure the performance of the model. Average detection accuracy evaluates the accuracy of the model in locating the key parts of insects, classification accuracy reflects the accuracy of the model in classifying each part, and overall classification accuracy shows the comprehensive performance of the model on all samples. These indicators combined provide a reliable basis for the optimization and improvement of the model.

[0041] The detection average accuracy Det_mAP represents the average accuracy of the model across all categories, and the formula is:

[0042]

[0043] Where C is the number of categories, AP i is the average precision of the i-th class.

[0044] Partial classification accuracy (Cls_pACC): indicates the classification accuracy of the classification model on all samples. The formula is:

[0045]

[0046] Where N is the number of samples, y i is the true label of the i-th sample, is the predicted label of the ith sample, 1 is the indicator function, which takes 1 when the predicted label is the same as the true label, otherwise it takes 0.

[0047] Overall classification accuracy (Cls_oACC): used to measure the overall classification performance of the model, indicating the classification accuracy of the model in the complete insect sample.

[0048] The present invention has the following characteristics and beneficial effects:

[0049] 1. This paper adopts a two-stage model to solve the problem of insect classification. In the first stage, the target detection network is used to locate the key parts of the insect (such as the head, thorax, abdomen) and the whole body, and then the classification network is used in the second stage to further analyze the characteristics of these parts. This two-stage framework can significantly improve the accuracy of insect image classification.

[0050] 2. During the classification process, we cropped various parts of the insect according to the position information provided by the detection model and input these cropped images into the classification network. Different from the traditional single classification method, we not only located the key parts of the whole, but also combined the center point position information of the insect head, chest, and abdomen with the output of the classification network through feature fusion. Finally, through the global context enhancement module, the global context information is dynamically adjusted and fused, so that the model can better capture the characteristic details of the insect when dealing with complex backgrounds, thereby further improving the robustness of the classification.

[0051] 3. In order to further improve the classification effect, this study introduced a collaborative mechanism. This mechanism combines the category information and classification accuracy of different parts to obtain the overall classification result. Through this collaborative calculation, the model finally outputs triple information of the insect, including the accuracy of detection, the characteristics of the parts obtained by classification, and the overall category result calculated by the collaborative mechanism. This mechanism ensures the comprehensiveness and accuracy of the classification and generates the structural feature map and category information of the insect. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0053] Figure 1 It is a flow chart for implementing the present invention.

[0054] Figure 2 Schematic diagram of the structure of the target detection network according to an embodiment of the present invention.

[0055] Figure 3 A schematic diagram of the structure of a classification network according to an embodiment of the present invention.

[0056] Figure 4 This is the implementation flow chart of the two-stage fine-grained recognition model.

[0057] Figure 5 This is a result graph output by the classification network according to an embodiment of the present invention.

[0058] Figure 6 This is a diagram of the final result output of the overall model of the embodiment of the present invention. DETAILED DESCRIPTION

[0059] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0060] The embodiment of the present invention provides a two-stage fine-grained insect recognition method based on key parts collaborative positioning, mainly taking red fire ants and similar ants as examples, including the following steps:

[0061] Step (1) In this embodiment, a red fire ant dataset named Ant7C is constructed. The main problems of this dataset are small intra-class differences and unbalanced data distribution, and can be used to verify the performance of the discovery model. The dataset is divided into a training image dataset, a verification image dataset, and a test image dataset in a ratio of 7:2:1. Image data enhancement on this basis will not cause data leakage, which will make the accuracy of the results more trustworthy. By marking the key parts in the red fire ant dataset, it can help extract the key features of the insects, and better classify and identify them, so that the model can learn and understand the characteristics of different insects more accurately. The annotation format is coco format, and the annotation information of each picture will be stored in a json file to form the Ant7C dataset.

[0062] Step (2): Build a target detection network, such as Figure 2 As shown in the figure, the target detection network includes an image encoder and a decoder. The backbone network uses ResNet50.

[0063] The target detection method of the target detection network is: for any given input feature map C×H×W, the number of input channels is 3, and Gaussian noise controlled by variance scheduling is added to the input image according to the rule of adding noise in the diffusion detection model, thereby obtaining 500 Gaussian noise boxes. The noise box is obtained using the following formula:

[0064] x t ~q(x t |x0)(8)

[0065]

[0066]

[0067] Where t is the time step of adding noise, x t represents the noise frame at time t, α t is a hyperparameter, εt-1 represents the noise at time t-1, and x0 represents the real target detection box during training.

[0068] The image encoder uses the ResNet model to extract high-level features from the original image. It first obtains features from the image through multiple layers of convolution, then processes the features with the activation function ReLU to obtain the image feature map, and then uses the pooling layer to reduce the spatial size of the feature map. Then, the obtained Gaussian noise box is used to crop the target area RoI in the feature map extracted by the image encoder. Finally, the target area RoI and the noise box are input into the decoder together to extract the multi-scale information of the target, and prediction is performed through a convolution layer or a transposed convolution layer with a convolution kernel size of 1*1. Its convolution layer is divided into a regression layer and a classification layer. The function of the regression layer is to predict the position information of the bounding box of the target object, and the number of channels is 4. The function of the classification layer is to predict the object category to which each bounding box belongs. For each bounding box, the classification layer outputs a vector whose length is equal to the number of categories in the data set. Each position corresponds to the probability distribution of a category. After being processed by the softmax function, the probability distribution of each category is obtained, and the number of channels is equal to the number of categories.

[0069] The network structure of the image encoder of the target detection network includes: a normal convolution layer with a convolution kernel size of 7*7, 64 channels and a stride of 2; a maximum pooling layer with a kernel size of 3*3 and a stride of 2; a BTNK1 module with a stride of 1 and 64 channels; 2 BTNK2 modules with 256 channels; then output to a BTNK1 module with a stride of 2, an input channel of 256 and an output channel of 128; 3 BTNK2 modules with 512 channels; then output to a BTNK1 module with a stride of 2, an input channel of 512 and an output channel of 256; 5 BTNK2 modules with 1024 channels; finally output to a BTNK1 module with a stride of 2, an input channel of 1025 and an output channel of 512; 2 BTNK2 modules with 2048 channels, and output a picture with a shape of (2048, 7, 7).

[0070] In the target detection network, the BTNK1 module has four parameters: C, W, C1, S. S: represents the step size in the convolution layer. When S is 1, the input size and the output size are the same, indicating that no downsampling is performed. C1: represents the number of feature maps output by the convolution layer, that is, the number of output channels. C: represents the number of input channels. W: represents the input size, that is, length and width. This module divides the image into two paths for processing. First, the first path first puts the input image into a convolution layer with a convolution kernel size of 1*1, a channel number of C1, and a step size of S, and then enters the BN layer and RELU activation function; then enters a convolution layer with a convolution kernel size of 3*3, a channel number of C1, and a step size of 1, and then enters the BN layer and RELU activation function; finally enters a convolution layer with a convolution kernel size of 1*1, a channel number of C1*4, and a step size of S, and then enters the BN layer. The second path puts the input image into a convolution layer with a kernel size of 1*1, a channel number of C1*4, and a stride of S, and then enters the BN layer. The output results of the two paths are then added together and output as an image with a shape of (C1*4, W / S, W / S) through the RELU activation function.

[0071] The BTNK2 module has two parameters: C and W. C: represents the number of input channels. W: represents the input size, i.e., length and width. This module processes the image in two paths. First, the first path puts the input image into a convolution layer with a convolution kernel size of 1*1, a number of channels of C / 4, and a stride of 1, and then enters the BN layer and the RELU activation function; then enters a convolution layer with a convolution kernel size of 3*3, a number of channels of C / 4, and a stride of 1, and then enters the BN layer and the RELU activation function; finally, it enters a convolution layer with a convolution kernel size of 1*1, a number of channels of C, and a stride of 1, and then enters the BN layer. The second path directly outputs the input image. Then, the output results of the two paths are added together, and then the RELU activation function is used to output it as an image with a shape of (C, W, W).

[0072] Step (3): Construct a classification network, such as Figure 3 shown.

[0073] The network structure of the classification network includes: 1 global context enhancement module (GCE); a common convolutional layer with a convolution kernel size of 3*3, 3 channels and a step size of 2; an MBConv1 module with a step size of 1; an MBConv6 module with a convolution kernel size of 3*3, 3 channels and a step size of 2; an MBConv6 module with a convolution kernel size of 5*5, 3 channels and a step size of 2; an MBConv6 module with a convolution kernel size of 3*3, 3 channels and a step size of 3; an MBConv6 module with a convolution kernel size of 5*5, 3 channels and a step size of 3; an MBConv6 module with a convolution kernel size of 5*5, 3 channels and a step size of 3; an MBConv6 module with a convolution kernel size of 5*5, 3 channels and a step size of 4; an MBConv6 module with a convolution kernel size of 3*3, 3 channels and a step size of 1; and finally, an ordinary convolutional layer with a convolution kernel size of 1*1, a pooling layer and a fully connected layer.

[0074] Among them, in the classification network, the GCE module first passes the global feature map through a common convolution layer with a convolution kernel size of 1*1 and a ReLU activation function, then performs element-by-element addition with the local feature map, and then passes through a common convolution layer with a convolution kernel size of 1*1 and a ReLU activation function to obtain the output fused feature map. MBConv1 with a step size of 1 passes the input fused feature map through a dimensionality-enhancing convolution with a convolution kernel size of 1*1, passes through a BN layer and a Swish activation layer, passes through a depth-separable convolution with a convolution kernel size of k*k and a step size of 1, then passes through a BN layer and a Swish activation layer, passes through a SE module and a dimensionality-reducing convolution with a convolution kernel size of 1*1, passes through a BN layer, and then passes through a Dropout layer, and finally shortcuts the feature map originally input by MBConv1; MBConv6 with a step size of 1 passes the input feature map through a dimensionality-enhancing convolution with a convolution kernel size of 1*1, passes through a BN layer and a Swish activation layer, passes through a convolution kernel size of k*k, and a step size of 1. The depthwise separable convolution with a length of 1 is then passed through the BN layer and the Swish activation layer, through the SE module and the dimensionality reduction convolution with a convolution kernel size of 1*1, through the BN layer, and then through the Dropout layer, and finally shortcut with the feature map originally input by MBConv1; MBConv6 with a stride of 2 is to subject the input feature map to a dimensionality increase convolution with a convolution kernel size of 1*1, through the BN layer and SiLU, through the depthwise separable convolution with a convolution kernel size of 3*3 and a stride of 2, through the BN layer and SiLU, through the SE module, then through the dimensionality reduction convolution with a convolution kernel size of 1*1, through the BN layer, and finally through the Dropout operation.

[0075] The SE module consists of a global average pooling and two fully connected layers, where the number of nodes in the first fully connected layer is 1 / 4 of the number of channels of the feature map input to the MBConv module, and the Swish activation function is used, and the number of nodes in the second fully connected layer is the number of channels of the feature map output by the depthwise separable convolution in the MBConv module, and the Sigmoid activation function is used.

[0076] Step (4): Construct a two-stage fine-grained recognition model, which consists of a target detection network and a classification network. First, the input image is detected by the target detection network to obtain the corresponding key part information, and then the image area of ​​the key part is cropped out and processed accordingly. Then, it is input into the classification network for processing to obtain the classification result. The classification result is integrated and calculated through the collaborative mechanism to generate the final overall classification output.

[0077] Specifically, Figure 4 As shown in the figure. First, the input ant image is detected by the target detection network to obtain the position information of the head, chest, abdomen and the whole body, so that the key parts of the target object in the image can be accurately located. After obtaining the key position information of each part, the image is cropped according to this position information to obtain a sub-image containing only a single key part. Next, the cropped image is first subjected to image preprocessing transformation, such as image normalization, resizing, rotation and other operations, and then the HOG feature is extracted from the preprocessed image to obtain the corresponding HOG feature. The current HOG feature is compared with the previously stored HOG feature, the cosine similarity is calculated, and the area with too high similarity (exceeding the set threshold) is filtered out to obtain the target image. Then the target image is input into the classification network for classification, and the preliminary classification result is output. Finally, the classification results obtained by each target object in the image are weighted by the weight coordination mechanism to obtain the final target classification result, that is, the classification result of the key part. The classification network adjusts and fuses the global context information through the global context enhancement module. The adjusted context feature map is added element-by-element to the input feature map, thereby improving the model's understanding and response capabilities to the overall scenario.

[0078] Step (5): Finally, the classification results of each part are integrated and calculated through a collaborative mechanism to generate the final overall classification output.

[0079] First, the cascade model obtains the category information and corresponding classification scores of the ant's head, chest, abdomen, and whole body. Next, a weight is randomly assigned to each part and the comprehensive score of each category is calculated. Specifically, the comprehensive score formula for a category is as follows:

[0080]

[0081] Among them, s i,c represents the score of the ith part for category c, w i Represents the weight of the ith part. After calculating the comprehensive scores of all categories, the category with the highest score is selected as the final prediction category.

[0082] Step (6): Based on the data processed in step 1, the two-stage fine-grained recognition model is trained, and insect recognition is completed through the trained two-stage fine-grained recognition model.

[0083] The input image is detected by the object detection network, and after generating the key parts image, it is classified using the classification network. After the loss is calculated, back propagation is performed and the model parameters are updated, while the training loss and accuracy of each epoch are recorded.

[0084] If the final predicted label obtained after training is inconsistent with the true label, it is necessary to calculate the loss value, using the cross entropy loss function, whose formula is:

[0085]

[0086] Where: C is the number of categories; y c is an indicator of the true category (i.e., y c =1, otherwise y c =0); is the probability that the model predicts class c.

[0087] Next, the gradient calculation is performed to adjust the weight. The gradient calculation formula is as follows:

[0088]

[0089] in, represents the score of the predicted category c on part i, y c Represents the score of the true category c on part i.

[0090] Finally, the weights are updated by gradient descent, and the process is repeated until the predicted category output by the model matches the true label. If the labels of all parts do not match the true labels, the output is classified incorrectly.

[0091] At the same time, three levels of evaluation indicators are introduced: average detection accuracy, classification accuracy, and overall classification accuracy to comprehensively measure the performance of the model. Average detection accuracy evaluates the accuracy of the model in locating the ant's part information, classification accuracy reflects the model's classification accuracy for each part, and overall classification accuracy shows the comprehensive performance of the model on all samples. These indicators combined provide a reliable basis for the optimization and improvement of the model.

[0092] The detection average accuracy Det_mAP represents the average accuracy of the model across all categories, and the formula is:

[0093]

[0094] Where C is the number of categories, AP i is the average precision of the i-th class.

[0095] Partial classification accuracy (Cls_pACC): indicates the classification accuracy of the classification model on all samples. The formula is:

[0096]

[0097] Where N is the number of samples, y i is the true label of the i-th sample, is the predicted label of the ith sample, 1 is the indicator function, which takes 1 when the predicted label is the same as the true label, otherwise it takes 0.

[0098] Overall classification accuracy (Cls_oACC): used to measure the overall classification performance of the model, indicating the classification accuracy of the model in the complete ant sample.

[0099] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions and variations of these embodiments including components are made without departing from the principles and spirit of the present invention, and still fall within the scope of protection of the present invention.

Claims

1. A two-stage fine-grained insect recognition method based on key parts collaborative positioning, characterized in that: The steps include: Step (1): Dataset acquisition and processing; According to the recognition requirements, insect data sets are obtained from the Internet and on-site collection methods, and then the data sets are annotated. The annotation method is to annotate the entire target object and key parts; Step (2): construct a two-stage fine-grained recognition model; The two-stage fine-grained recognition model consists of a target detection network and a classification network; the target detection network shown is used to extract images of key parts, and the classification network is used to classify images of key parts; Step (3): First, the input insect image is detected through the target detection network to obtain the position information of the key parts of the insect to be identified. After obtaining the position information of each key part of the insect to be identified, the image is cropped according to the position information to obtain a sub-image containing only a single key part. Then, the cropped images are preprocessed, including resizing and normalization. Then, HOG feature extraction is performed on the preprocessed images and the areas whose similarity exceeds the set threshold are filtered out to obtain the target image. Step (4): passing the target image obtained in step (3) to the classification network to identify the insect category; First, the target image obtained in step (3) is processed by the classification network, and the output of the classification network is concatenated with the central point position information of the key parts of the insect to be identified; the concatenated feature vector is input into the fully connected layer, and through further feature processing, an enhanced feature vector containing spatial position information and category information is finally generated, that is, the classification result of the key parts is obtained; the classification network adjusts and fuses the global context information through the global context enhancement module; the adjusted context feature map is added element by element to the input feature map, thereby improving the model's understanding and response capabilities to the overall scene; Step (5): Finally, the classification results of each part are integrated and calculated through a collaborative mechanism to generate the final overall classification output.

2. A two-stage fine-grained insect recognition method based on key parts collaborative positioning according to claim 1, characterized in that: The target detection method of the target detection network is: The target detection network includes an image encoder and a decoder; the image encoder uses a ResNet50 model to extract high-level features from the original input image; first, features are obtained from the original input image through multi-layer convolution, and then the features are processed by an activation function ReLU to obtain an image feature map, and then the spatial size of the feature map is reduced by a pooling layer; then, a Gaussian noise frame is used to crop the feature map extracted by the image encoder to obtain the target area RoI; finally, the target area RoI and the Gaussian noise frame are input into the decoder together to extract the multi-scale information of the target, and prediction is performed through a convolution layer with a convolution kernel size of 1*1; the convolution layer is divided into a regression layer and a classification layer, the function of the regression layer is to predict the position information of the bounding box of the target object, and the number of channels is 4, and the function of the classification layer is to predict the category of the key part to which each bounding box belongs; For each bounding box, the classification layer outputs a vector whose length is equal to the number of categories in the dataset. Each position corresponds to the probability distribution of a category. After processing with the softmax function, the probability distribution of each category is obtained. The number of channels is equal to the number of categories. The Gaussian noise frame is obtained as follows: for any given input feature map C×H×W, the number of input channels is 3, and Gaussian noise controlled by variance scheduling is added to the input image according to the rule of adding noise in the diffusion detection model, thereby obtaining 500 Gaussian noise frames; the noise frame is obtained using the following formula: x t ~q(x t |x0)(1) Where t is the time step of adding noise, x t represents the noise frame at time t, α t is a hyperparameter, ε t-1 represents the noise at time t-1, and x0 represents the real target detection box during training.

3. A two-stage fine-grained insect recognition method based on key parts collaborative positioning according to claim 2, characterized in that: The network structure of the image encoder of the target detection network is as follows: the input image first passes through a normal convolution layer with a convolution kernel size of 7*7, 64 channels, and a step size of 2; a maximum pooling layer with a kernel size of 3*3 and a step size of 2; a BTNK1 module with a step size of 1 and a channel of 64; and two BTNK2 modules with a channel of 256; then passes through a BTNK1 module with a step size of 2, an input channel of 256, and an output channel of 128; three BTNK2 modules with a channel of 512; then outputs to a BTNK1 module with a step size of 2, an input channel of 512, and an output channel of 256; five BTNK2 modules with a channel of 1024; finally, it passes through a BTNK1 module with a step size of 2, an input channel of 1025, and an output channel of 512; and two BTNK2 modules with a channel of 2048, and outputs a feature map with a shape of (2048, 7, 7).

4. A two-stage fine-grained insect recognition method based on key parts collaborative positioning according to claim 3, characterized in that: In the image encoder, the BTNK1 module has four parameters: C, W, C1, S; Where S: represents the step size in the convolution layer. When S is 1, the input size and output size are the same, indicating that no downsampling is performed; C1: represents the number of feature maps output by the convolution layer, that is, the number of output channels; C: represents the number of input channels; W: represents the input size, that is, length and width; The BTNK1 module divides the input image into two paths for processing. First, the first path processes the input image through a convolution layer with a convolution kernel size of 1*1, a channel number of C1, and a step size of S, and then enters the BN layer and RELU activation function; then enters a convolution kernel size of 3* 3. A convolutional layer with a channel number of C1 and a stride of 1, followed by a BN layer and a RELU activation function; finally, a convolutional layer with a convolution kernel size of 1*1, a channel number of C1*4, and a stride of S, followed by a BN layer; the second path processes the input image through a convolutional layer with a convolution kernel size of 1*1, a channel number of C1*4, and a stride of S, followed by a BN layer; the output results of the two paths are then added together, and then the RELU activation function is used to output an image with a shape of (C1*4, W / S, W / S); The BTNK2 module has two parameters: C and W; C represents the number of input channels; W represents the input size, i.e., length and width; the BTNK2 module divides the input image into two paths for processing. First, the first path processes the input image through a convolution layer with a convolution kernel size of 1*1, a number of channels of C / 4, and a step size of 1, and then enters the BN layer and the RELU activation function; then it enters a convolution layer with a convolution kernel size of 3*3, a number of channels of C / 4, and a step size of 1, and then enters the BN layer and the RELU activation function; finally, it enters a convolution layer with a convolution kernel size of 1*1, a number of channels of C, and a step size of 1, and then enters the BN layer; the second path directly outputs the input image; then the output results of the two paths are added, and then the RELU activation function is used to output it as an image with a shape of (C, W, W).

5. The two-stage fine-grained insect recognition method based on key parts collaborative positioning according to claim 1 is characterized in that: The network structure of the classification network is as follows: the global image and the local image are first input into the global context enhancement module to obtain a fused feature map, and then pass through a normal convolution layer with a convolution kernel size of 3*3, a number of channels of 3, and a step size of 2; an MBConv1 module with a step size of 1; an MBConv6 module with a convolution kernel size of 3*3, a number of channels of 3, and a step size of 2; an MBConv6 module with a convolution kernel size of 5*5, a number of channels of 3, and a step size of 2; an MBConv6 module with a convolution kernel size of 3*3, a number of channels of 3, and a step size of 3; an MBConv6 module with a convolution kernel size of 5*5, a number of channels of 3, and a step size of 3; an MBConv6 module with a convolution kernel size of 5*5, a number of channels of 3, and a step size of 3; an MBConv6 module with a convolution kernel size of 5*5, a number of channels of 3, and a step size of 4; an MBConv6 module with a convolution kernel size of 3*3, a number of channels of 3, and a step size of 1; finally, pass through a normal convolution layer with a convolution kernel size of 1*1, a pooling layer, and a fully connected layer.

6. A two-stage fine-grained insect recognition method based on key parts collaborative positioning according to claim 5, characterized in that: In the classification network, the specific structure of the global context enhancement module is as follows: first, the global feature map passes through a common convolution layer with a convolution kernel size of 1*1 and a ReLU activation function, and then is added element by element with the local feature map, and then passes through a common convolution layer with a convolution kernel size of 1*1 and a ReLU activation function to obtain an output fused feature map.

7. The two-stage fine-grained insect recognition method based on key parts collaborative positioning according to claim 5 is characterized in that: In the classification network, the specific operation of the MBConv1 module with a step size of 1 is as follows: first, the input fusion feature map is subjected to a dimensionality-increasing convolution with a convolution kernel size of 1*1, a BN layer and a Swish activation layer, then a depth-separable convolution with a convolution kernel size of k*k and a step size of 1, a BN layer and a Swish activation layer, then a SE module and a dimensionality-reducing convolution with a convolution kernel size of 1*1 and a BN layer, then a Dropout layer, and finally a shortcut is performed with the feature map initially input by MBConv1 to obtain the module output; The specific operation of the MBConv6 module with a step size of 1 is as follows: first, the input feature map is subjected to a dimensionality-increasing convolution with a convolution kernel size of 1*1, a BN layer and a Swish activation layer, then a depth-wise separable convolution with a convolution kernel size of k*k and a step size of 1, a BN layer and a Swish activation layer, then a SE module and a dimensionality-reducing convolution with a convolution kernel size of 1*1 and a BN layer, then a Dropout layer, and finally a shortcut with the feature map originally input to MBConv6 to obtain the module output; The specific operation of the MBConv6 module with a stride of 2 is as follows: first, the input feature map is subjected to a dimensionality-increasing convolution with a convolution kernel size of 1*1, a BN layer and SiLU, then a depth-wise separable convolution with a convolution kernel size of 3*3 and a stride of 2, a BN layer and SiLU, then a SE module, then a dimensionality-reducing convolution with a convolution kernel size of 1*1 and a BN layer, and finally a Dropout operation is performed to obtain the module output; The SE module consists of a global average pooling and two fully connected layers, where the number of nodes in the first fully connected layer is 1 / 4 of the number of channels of the feature map of the corresponding MBConv module input, and the Swish activation function is used, and the number of nodes in the second fully connected layer is the number of channels of the feature map output by the depthwise separable convolution in the corresponding MBConv module, and the Sigmoid activation function is used.

8. The two-stage fine-grained insect recognition method based on key parts collaborative positioning according to claim 1 is characterized in that: The final classification results are calculated through a collaborative mechanism, as follows: First, the two-stage fine-grained recognition model obtains the category information of the key parts of the insect to be identified and the insect as a whole, as well as the corresponding classification scores; next, a weight is randomly assigned to each part, and the comprehensive score of each category is calculated; specifically, the comprehensive score formula of the category is as follows: Among them, s i,c represents the score of the ith part for category c, w i Represents the weight of the ith part; after calculating the comprehensive scores of all categories, the category with the highest score is selected as the final prediction category.

9. A two-stage fine-grained insect recognition method based on key parts collaborative positioning according to any one of claims 1 to 8, characterized in that: During training, if the final predicted label obtained by training is inconsistent with the true label, it is necessary to calculate the loss value and use the cross entropy loss function, whose formula is: Where: C is the number of categories; y c is an indicator of the true category, that is, when category c is the true category, y c =1, otherwise y c =0; is the probability of the model predicting category c; Next, the gradient calculation is performed to adjust the weight. The gradient calculation formula is as follows: in, represents the score of the predicted category c on part i, y c Represents the score of the true category c on part i; Finally, the weights are updated using the gradient descent method, and the process is repeated until the predicted category output by the model matches the true label; if the labels of all parts are inconsistent with the true labels, the output classification is wrong.

10. The two-stage fine-grained insect recognition method based on key parts collaborative positioning according to claim 1, characterized in that: In order to better evaluate the model, three levels of evaluation indicators were introduced: average detection accuracy, classification accuracy and overall classification accuracy to comprehensively measure the performance of the model; average detection accuracy evaluates the accuracy of the model in locating the information of key parts of insects, classification accuracy reflects the classification accuracy of the model for each part, and overall classification accuracy shows the comprehensive performance of the model on all samples; these indicators combined provide a reliable basis for the optimization and improvement of the model; The detection average accuracy Det_mAP represents the average accuracy of the model across all categories, and the formula is: Where C is the number of categories, AP i is the average precision of the i-th class; Partial classification accuracy (Cls_pACC): represents the classification accuracy of the classification model on all samples; the formula is: Where N is the number of samples, y i is the true label of the i-th sample, is the predicted label of the i-th sample, 1 is the indicator function, which takes 1 when the predicted label is the same as the true label, otherwise it takes 0; Overall classification accuracy (Cls_oACC): used to measure the overall classification performance of the model, indicating the classification accuracy of the model in the complete insect sample.

Images