A two-stage fine-grained insect identification method based on key site co-localization

Abstract

This invention discloses a two-stage fine-grained insect identification method based on key region co-localization. The method utilizes a detection model to improve the relative size of key regions, enabling better identification of effective features of target insects across multiple scales and enhancing the ability to identify easily confused species. Simultaneously, a Global Context Enhancement (GCE) module dynamically adjusts and fuses global contextual information, allowing the model to better capture insect feature details when dealing with complex backgrounds, thereby further improving classification robustness. A collaborative mechanism is introduced to comprehensively calculate the results, ensuring the comprehensiveness and accuracy of the classification results.

Description

Technical Field

[0001] This invention belongs to the field of agricultural image detection and classification technology, specifically involving a two-stage fine-grained insect recognition method based on key part co-localization. Background Technology

[0002] With the increasing complexity of ecosystems and the acceleration of globalization, the accurate identification of insect species, as a key component of ecosystems, is of paramount importance. Precise insect classification not only contributes to a deeper understanding of the structure and function of biodiversity but also plays an indispensable role in controlling invasive alien species and maintaining ecological balance. Among these methods, image-based insect identification has gradually become mainstream due to its simplicity, speed, and ease of operation. With the development of deep learning models such as convolutional neural networks (CNNs), the efficiency and accuracy of classification have been significantly improved, while also significantly reducing labor costs and meeting the needs of large-scale data processing.

[0003] To more accurately identify images of similar insect species, fine-grained recognition models and methods have been introduced to improve insect identification accuracy (in 2021, Loris et al. integrated CNN networks with different topologies to identify small pest datasets; in 2024, Md et al. constructed a convolutional neural network based on the ResNet152V2 architecture for classifying field pest images; and in 2022, Zhu et al. added an attention mechanism to the YOLOv5 backbone network). However, these methods cannot fully and effectively utilize the feature characteristics of the target object, and when the image background is complex, they cannot capture the feature details of the target object well. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and propose a two-stage fine-grained insect identification method based on key part co-localization. This method improves the relative size of key parts by using a detection model (the fine-grained identification model of that stage), which can better identify the effective features of target insects at multiple scales and improve the identification ability of easily confused species. At the same time, a global context enhancement module is used to dynamically adjust and fuse global context information, enabling the model to better capture the feature details of insects when dealing with complex backgrounds, thereby further improving the robustness of classification. A collaborative mechanism is introduced to comprehensively calculate the results, ensuring the comprehensiveness and accuracy of the classification results.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0006] A two-stage fine-grained insect identification method based on key site co-localization includes the following steps:

[0007] Step (1): Dataset acquisition and processing;

[0008] Insect datasets were obtained from online and field collection methods based on identification requirements. The datasets were then labeled, with the labeling method involving labeling the entire target object as well as key parts.

[0009] Step (2): Construct a two-stage fine-grained recognition model;

[0010] The two-stage fine-grained recognition model consists of an object detection network and a classification network; the object detection network is used to extract images of key parts, and the classification network is used to classify the images of key parts.

[0011] Step (3): First, the input insect image is detected by the target detection network to obtain the location information of the key parts of the insect to be identified. After obtaining the location information of each key part of the insect to be identified, the image is cropped using this location information to obtain a sub-image containing only a single key part. Then, the cropped images are preprocessed, including resizing and normalization. Then, HOG features are extracted from the preprocessed images and regions with similarity exceeding the set threshold are filtered out to obtain the target image.

[0012] Step (4): The target image obtained in step (3) is transmitted to the classification network for insect category identification.

[0013] First, the target image obtained in step (3) is processed by a classification network. The output of the classification network is then concatenated with the center point location information of the key parts of the insect to be identified. The concatenated feature vector is input into a fully connected layer. Through further feature processing, an enhanced feature vector containing spatial location information and category information is finally generated, thus obtaining the classification result of the key parts. The classification network adjusts and fuses global context information through a global context enhancement module. The adjusted context feature map is added element-wise to the input feature map, thereby improving the model's understanding and response capabilities to the overall scene.

[0014] Step (5): Finally, the classification results of each part are integrated and calculated through the collaborative mechanism to generate the final overall classification output.

[0015] Preferably, the target detection method of the target detection network is as follows:

[0016] The object detection network comprises an image encoder and a decoder. The image encoder uses a ResNet50 model to extract high-level features from the original input image. First, features are extracted from the original input image through multiple convolutional layers. Then, the features are processed using the ReLU activation function to obtain an image feature map. Next, pooling layers reduce the spatial size of the feature map. Then, Gaussian noise boxes are used to crop the feature map extracted by the image encoder to obtain the target region (RoI). Finally, the target RoI and the Gaussian noise boxes are input into the decoder to extract multi-scale information of the target. Prediction is performed using convolutional layers with a kernel size of 1*1. The convolutional layers are divided into regression layers and classification layers. The regression layer predicts the bounding box location information of the target object, with 4 channels. The classification layer predicts the category of the key part to which each bounding box belongs. For each bounding box, the classification layer outputs a vector whose length equals the number of categories in the dataset. Each position corresponds to a probability distribution of a category. After processing with a softmax function, the probability distribution of each category is obtained, with the number of channels equal to the number of categories.

[0017] The method for obtaining Gaussian noise boxes is as follows: For any given input feature map C×H×W with 3 input channels, Gaussian noise controlled by variance scheduling is added to the input image according to the noise addition rules of the diffusion detection model, thus obtaining 500 Gaussian noise boxes. The noise boxes are obtained using the following formula:

[0018] x t ~q(x t |x0)(1)

[0019]

[0020] Where t is the time step for adding noise, and x t The noise box at time t, α t It is a hyperparameter, ε t-1 x represents the noise at time t-1, and x0 represents the actual object detection box during training.

[0021] Preferably, the network structure of the image encoder of the target detection network is as follows: The input image first passes through a regular convolutional layer with a kernel size of 7*7, 64 channels, and a stride of 2; a max pooling layer with a kernel size of 3*3 and a stride of 2; a BTNK1 module with a stride of 1 and 64 channels; and two BTNK2 modules with 256 channels each; then through one BTNK1 module with a stride of 2, 256 input channels, and 128 output channels; and three BTNK2 modules with 512 channels each; then outputs to one BTNK1 module with a stride of 2, 512 input channels, and 256 output channels; and five BTNK2 modules with 1024 channels each; finally, it passes through one BTNK1 module with a stride of 2, 1025 input channels, and 512 output channels; and two BTNK2 modules with 2048 channels each, outputting a feature map of shape (2048, 7, 7).

[0022] Preferably, the BTNK1 module in the image encoder has four parameters: C, W, C1, and S; where S represents the stride in the convolutional layer. When S is 1, the input and output sizes are the same, indicating that no downsampling is performed. C1 represents the number of feature maps output by the convolutional layer, i.e., the number of output channels. C represents the number of input channels. W represents the input size, i.e., the length and width. The BTNK1 module processes the input image in two paths. First, the input image is processed through a convolutional layer with a kernel size of 1*1, C1 channels, and a stride of S, followed by a BN layer and a ReLU activation function. Then, it is processed through a convolutional layer with a kernel size of 3*3, C1 channels, and a stride of 1, followed by a BN layer and a ReLU activation function. Finally, it is processed through a convolutional layer with a kernel size of 1*1, C1*4 channels, and a stride of S, followed by a BN layer. The second path processes the input image through a convolutional layer with a kernel size of 1*1, C1*4 channels, and a stride of S, followed by a batch normalization (BN) layer. The outputs of the two paths are then summed, and the image is activated by ReLU to produce an image of shape (C1*4, W / S, W / S).

[0023] The BTNK2 module has two parameters: C and W; where C represents the number of input channels, and W represents the input dimensions, i.e., length and width. The BTNK2 module processes the input image in two paths. First, the input image is processed through a convolutional layer with a kernel size of 1x1, C / 4 channels, and a stride of 1, followed by a batch normalization (BN) layer and a ReLU activation function. Next, it is processed through a convolutional layer with a kernel size of 3x3, C / 4 channels, and a stride of 1, followed by another BN layer and a ReLU activation function. Finally, it is processed through a convolutional layer with a kernel size of 1x1, C channels, and a stride of 1, followed by another BN layer. The second path directly outputs the input image. The outputs from the two paths are then summed, and the image is further processed through a ReLU activation function to produce an image of shape (C, W, W).

[0024] Preferably, the network structure of the classification network is as follows: The global image and local image are first input into the global context enhancement module to obtain a fused feature map, and then pass through a regular convolutional layer with a kernel size of 3*3, 3 channels, and a stride of 2; an MBConv1 module with a stride of 1; an MBConv6 module with a kernel size of 3*3, 3 channels, and a stride of 2; an MBConv6 module with a kernel size of 5*5, 3 channels, and a stride of 2; an MBConv6 module with a kernel size of 3*3, 3 channels, and a stride of 3; an MBConv6 module with a kernel size of 5*5, 3 channels, and a stride of 3; an MBConv6 module with a kernel size of 5*5, 3 channels, and a stride of 4; an MBConv6 module with a kernel size of 3*3, 3 channels, and a stride of 1; finally, it passes through a regular convolutional layer with a kernel size of 1*1, a pooling layer, and a fully connected layer.

[0025] Preferably, the global context enhancement module in the classification network has the following specific structure: First, the global feature map is passed through a regular convolutional layer with a kernel size of 1*1 and a ReLU activation function, then element-wise added to the local feature map, and then passed through a regular convolutional layer with a kernel size of 1*1 and a ReLU activation function to obtain the output fused feature map.

[0026] Preferably, in the classification network, the MBConv1 module with a stride of 1 operates as follows: First, the input fused feature map is passed through an up-dimensional convolution with a kernel size of 1*1, a BN layer, and a Swish activation layer. Then, it is passed through a depthwise separable convolution with a kernel size of k*k and a stride of 1, a BN layer, and a Swish activation layer. Next, it is passed through an SE module, a down-dimensional convolution with a kernel size of 1*1, and a BN layer. After that, it is passed through a Dropout layer. Finally, it is shortened with the feature map initially input to MBConv1 to obtain the module output.

[0027] The MBConv6 module with a stride of 1 operates as follows: First, the input feature map is passed through an up-dimensional convolution with a kernel size of 1*1, followed by a BN layer and a Swish activation layer. Then, it is passed through a depthwise separable convolution with a kernel size of k*k and a stride of 1, followed by a BN layer and a Swish activation layer. Next, it is passed through an SE module, a down-dimensional convolution with a kernel size of 1*1, and a BN layer. Finally, it is passed through a Dropout layer and then shortened with the initial input feature map of MBConv6 to obtain the module output.

[0028] The MBConv6 module with a stride of 2 operates as follows: First, the input feature map is processed by an up-dimensional convolution with a kernel size of 1*1, followed by a BN layer and SiLU. Then, it is processed by a depthwise separable convolution with a kernel size of 3*3 and a stride of 2, followed by a BN layer and SiLU. Next, it is processed by the SE module, followed by a down-dimensional convolution with a kernel size of 1*1 and a BN layer. Finally, the module output is obtained through the Dropout operation.

[0029] The SE module consists of a global average pooling layer and two fully connected layers. The number of nodes in the first fully connected layer is 1 / 4 of the number of channels in the feature map of the corresponding MBConv module, and it uses the Swish activation function. The number of nodes in the second fully connected layer is the number of channels in the output feature map of the depthwise separable convolution within the corresponding MBConv module, and it uses the Sigmoid activation function.

[0030] Preferably, a collaborative mechanism is used to calculate the final classification result. First, the two-stage fine-grained recognition model obtains the category information and corresponding classification scores for key parts of the insect to be identified and the insect as a whole. Next, weights are randomly assigned to each part, and a comprehensive score for each category is calculated. Specifically, the formula for the comprehensive score of each category is as follows:

[0031]

[0032] Among them, s i,c w represents the score of the i-th part for category c. i This represents the weight of the i-th part. After calculating the overall score for all categories, the category with the highest score is selected as the final predicted category.

[0033] Preferably, if the final predicted label obtained during training is inconsistent with the true label, it is necessary to calculate the loss value using the cross-entropy loss function, the formula of which is:

[0034]

[0035] Where: C is the number of categories; y cIt is an indicator of the true category (i.e., when category c is the true category, y is the true category). c =1, otherwise y c =0); It is the probability that the model predicts for category c.

[0036] Next, gradient calculation is performed to adjust the weights. The gradient calculation formula is as follows:

[0037]

[0038] in, y represents the predicted score of category c at location i. c This represents the score of the true category c at part i.

[0039] Finally, the weights are updated using gradient descent, and this process is repeated until the model's predicted class matches the true label. If the labels for all body parts do not match the true label, a classification error is output.

[0040] As a preferred approach, a three-tiered evaluation metric was introduced to better assess the model: average detection accuracy, classification accuracy, and overall classification accuracy, to comprehensively measure the model's performance. Average detection accuracy assesses the model's accuracy in locating key parts of insects, classification accuracy reflects the model's accuracy in classifying individual parts, and overall classification accuracy demonstrates the model's comprehensive performance across all samples. The combination of these metrics provides a reliable basis for model optimization and improvement.

[0041] The detection average accuracy, Det_mAP, represents the average accuracy of the model across all classes, and is expressed by the formula:

[0042]

[0043] Where C is the number of categories, AP i It is the average precision of the i-th class.

[0044] Partial classification accuracy (Cls_pACC): Represents the classification accuracy of the model across all samples. The formula is:

[0045]

[0046] Where N is the number of samples, y i It is the true label of the i-th sample. is the predicted label of the i-th sample, and 1 is the indicator function, which takes the value 1 when the predicted label is the same as the true label, and 0 otherwise.

[0047] Overall classification accuracy (Cls_oACC): This measures the model's overall classification performance, representing the model's classification accuracy in complete insect samples.

[0048] This invention has the following characteristics and beneficial effects:

[0049] 1. This invention employs a two-stage model to solve the insect classification problem. The first stage uses a target detection network to locate key parts of the insect (such as the head, thorax, and abdomen) and the insect as a whole. Then, in the second stage, a classification network is used to further analyze the features of these parts. This two-stage framework can significantly improve the accuracy of insect image classification.

[0050] 2. During the classification process, we crop various parts of the insect based on the location information provided by the detection model and input these cropped images into the classification network. Unlike traditional single classification methods, we not only locate the key parts of the whole insect but also combine the center point location information of the insect's head, thorax, and abdomen with the output of the classification network through feature fusion. Finally, through the global context enhancement module, we dynamically adjust and fuse global context information, enabling the model to better capture the feature details of the insect when dealing with complex backgrounds, thereby further improving the robustness of classification.

[0051] 3. To further improve classification accuracy, this study introduced a collaborative mechanism. This mechanism integrates category information from different body parts with classification accuracy to arrive at an overall classification result. Through this collaborative computation, the model ultimately outputs three pieces of information about the insect: detection accuracy, the body part features obtained from classification, and the overall category result calculated by the collaborative mechanism. This mechanism ensures the comprehensiveness and accuracy of classification, generating structural feature maps and category information for the insect. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart illustrating the implementation of the present invention.

[0054] Figure 2 This is a schematic diagram of the target detection network in an embodiment of the present invention.

[0055] Figure 3 This is a schematic diagram of the classification network structure in an embodiment of the present invention.

[0056] Figure 4 This is a flowchart illustrating the implementation of a two-stage fine-grained recognition model.

[0057] Figure 5 This is a diagram showing the results output by the classification network in an embodiment of the present invention.

[0058] Figure 6 This is the final result diagram output by the overall model in the embodiment of the present invention. Detailed Implementation

[0059] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0060] This invention provides a two-stage fine-grained insect identification method based on key site co-localization, taking the red imported fire ant and its similar ant species as examples, including the following steps:

[0061] Step (1) In this embodiment, a red imported fire ant dataset named Ant7C was constructed. This dataset mainly suffers from small intra-class differences and imbalanced data distribution, and can be used to verify the performance of the discovery model. The dataset is divided into a training image dataset, a validation image dataset, and a test image dataset in a ratio of 7:2:1. Image data augmentation on this basis will not cause data leakage, thus making the accuracy of the results more reliable. By labeling key parts in the red imported fire ant dataset, it is possible to extract key features of insects and better classify and identify them, enabling the model to learn and understand the characteristics of different insects more accurately. The annotation format is COCO format, and the annotation information of each image is stored in a JSON file to form the Ant7C dataset.

[0062] Step (2): Construct an object detection network, such as Figure 2 As shown, the object detection network includes an image encoder and a decoder. The backbone network uses ResNet50.

[0063] The target detection method of the described target detection network is as follows: For any given input feature map C×H×W with 3 input channels, Gaussian noise controlled by variance scheduling is added to the input image according to the noise addition rules of the diffusion detection model, thereby obtaining 500 Gaussian noise boxes. The noise boxes are obtained using the following formula:

[0064] x t ~q(x t |x0)(8)

[0065]

[0066]

[0067] Where t is the time step for adding noise, and x t The noise box at time t, α t It is a hyperparameter, εt-1 x represents the noise at time t-1, and x0 represents the actual object detection box during training.

[0068] The image encoder uses a ResNet model to extract high-level features from the original image. It first extracts features from the image through multiple convolutional layers, then processes these features using the ReLU activation function to obtain an image feature map. Next, pooling layers are used to reduce the spatial size of the feature map. Then, the target region (RoI) is cropped from the feature map extracted by the image encoder using the obtained Gaussian noise bounding boxes. Finally, the target RoI and the noise bounding boxes are input into the decoder to extract multi-scale information of the target. Prediction is performed using convolutional layers with a kernel size of 1*1 or transposed convolutional layers. Its convolutional layers consist of regression and classification layers. The regression layer predicts the bounding box location information of the target object, with 4 channels. The classification layer predicts the object category of each bounding box. For each bounding box, the classification layer outputs a vector with a length equal to the number of categories in the dataset. Each position corresponds to a probability distribution for a category, which is processed by a softmax function to obtain the probability distribution for each category. The number of channels equals the number of categories.

[0069] The network structure of the image encoder of the target detection network includes: a regular convolutional layer with a kernel size of 7*7, 64 channels, and a stride of 2; a max pooling layer with a kernel size of 3*3 and a stride of 2; a BTNK1 module with a stride of 1 and 64 channels; two BTNK2 modules with 256 channels each; the output is then fed to a BTNK1 module with a stride of 2, 256 input channels, and 128 output channels; three BTNK2 modules with 512 channels each; the output is then fed to a BTNK1 module with a stride of 2, 512 input channels, and 256 output channels; five BTNK2 modules with 1024 channels each; finally, the output is fed to a BTNK1 module with a stride of 2, 1025 input channels, and 512 output channels; and two BTNK2 modules with 2048 channels each, outputting an image of shape (2048, 7, 7).

[0070] In the object detection network, the BTNK1 module has four parameters: C, W, C1, and S. S: represents the stride in the convolutional layer. When S is 1, the input and output sizes are the same, indicating no downsampling. C1: represents the number of feature maps output by the convolutional layer, i.e., the number of output channels. C: represents the number of input channels. W: represents the input size, i.e., length and width. This module processes the image in two paths. First, the input image is fed into a convolutional layer with a kernel size of 1*1, C1 channels, and a stride of S, followed by a BN layer and a ReLU activation function. Next, it is fed into a convolutional layer with a kernel size of 3*3, C1 channels, and a stride of 1, followed by another BN layer and a ReLU activation function. Finally, it is fed into a convolutional layer with a kernel size of 1*1, C1*4 channels, and a stride of S, followed by another BN layer. The second path feeds the input image into a convolutional layer with a kernel size of 1*1, C1*4 channels, and a stride of S, followed by a batch normalization (BN) layer. The outputs of the two paths are then summed, and the image is activated by ReLU to produce an image of shape (C1*4, W / S, W / S).

[0071] The BTNK2 module has two parameters: C and W. C represents the number of input channels. W represents the input dimensions, i.e., length and width. This module processes the image in two paths. The first path feeds the input image into a convolutional layer with a 1x1 kernel, C / 4 channels, and a stride of 1, followed by a batch normalization (BN) layer and a ReLU activation function. Next, it feeds into a convolutional layer with a 3x3 kernel, C / 4 channels, and a stride of 1, again followed by a BN layer and a ReLU activation function. Finally, it feeds into a convolutional layer with a 1x1 kernel, C channels, and a stride of 1, followed by another BN layer. The second path directly outputs the input image. The outputs from both paths are then summed, and the image is further processed by the ReLU activation function to produce an image of shape (C, W, W).

[0072] Step (3): Construct a classification network, such as Figure 3 As shown.

[0073] The network structure of the classification network includes: a Global Context Enhancement (GCE) module; a regular convolutional layer with a kernel size of 3*3, 3 channels, and a stride of 2; an MBConv1 module with a stride of 1; an MBConv6 module with a kernel size of 3*3, 3 channels, and a stride of 2; an MBConv6 module with a kernel size of 5*5, 3 channels, and a stride of 2; an MBConv6 module with a kernel size of 3*3, 3 channels, and a stride of 3; an MBConv6 module with a kernel size of 5*5, 3 channels, and a stride of 3; an MBConv6 module with a kernel size of 5*5, 3 channels, and a stride of 4; an MBConv6 module with a kernel size of 3*3, 3 channels, and a stride of 1; and finally, a regular convolutional layer with a kernel size of 1*1, a pooling layer, and a fully connected layer.

[0074] In the classification network, the GCE module first processes the global feature map through a regular convolutional layer with a 1x1 kernel and a ReLU activation function, then performs element-wise addition with the local feature map, followed by another regular convolutional layer with a 1x1 kernel and a ReLU activation function to obtain the output fused feature map. MBConv1 with a stride of 1 processes the input fused feature map through a 1x1 kernel-upward convolution, a BN layer and a Swish activation layer, a kxk kernel-separable convolution with a stride of 1, another BN layer and a Swish activation layer, a SE module and a 1x1 kernel-downward convolution, another BN layer, a Dropout layer, and finally a Shortcut with the initial input feature map of MBConv1. MBConv6 with a stride of 1 processes the input feature map through a 1x1 kernel-upward convolution, a BN layer and a Swish activation layer, another kxk kernel-separable convolution, and a stride of 1. The first convolution with a depthwise separable convolution of length 1 passes through a BN layer and a Swish activation layer, then through an SE module and a 1x1 kernel-reduced convolution, followed by another BN layer, then a Dropout layer, and finally a Shortcut operation with the initial input feature map of MBConv1. The second convolution with a stride of 2 passes the input feature map through a 1x1 kernel-upgraded convolution, then through a BN layer and SiLU, then through a 3x3 kernel-separable convolution with a stride of 2, then through a BN layer and SiLU, then through an SE module, then through a 1x1 kernel-reduced convolution, then through another BN layer, and finally through a Dropout operation.

[0075] The SE module consists of a global average pooling layer and two fully connected layers. The number of nodes in the first fully connected layer is 1 / 4 of the number of channels in the input feature map of the MBConv module, and it uses the Swish activation function. The number of nodes in the second fully connected layer is the number of channels in the depthwise separable convolution output feature map within the MBConv module, and it uses the Sigmoid activation function.

[0076] Step (4): Construct a two-stage fine-grained recognition model. The two-stage fine-grained recognition model consists of an object detection network and a classification network. First, the object detection network detects the input image to obtain the corresponding key part information. Then, the image region of the key part is cropped out and processed accordingly. Then, it is input into the classification network for processing to obtain the classification result. After the collaborative mechanism integrates and calculates the classification result, the final overall classification output is generated.

[0077] Specifically, such as Figure 4 As shown, the process begins by detecting the input ant image using a target detection network to obtain the positional information of the head, thorax, abdomen, and overall shape. This allows for accurate localization of key parts of the target object within the image. After obtaining the key positional information for each part, the image is cropped to obtain a sub-image containing only a single key part. Next, the cropped image undergoes image preprocessing transformations, such as image normalization, resizing, and rotation. Then, HOG features are extracted from the preprocessed image to obtain the corresponding HOG features. The current HOG features are compared with previously stored HOG features, and cosine similarity is calculated. Regions with excessively high similarity (exceeding a set threshold) are filtered out to obtain the target image. The target image is then input into a classification network for classification, outputting preliminary classification results. Finally, the classification results obtained for each target object in the image are weighted using a weighted collaborative mechanism to obtain the final target classification result, i.e., the classification result for the key parts. The classification network uses a global context enhancement module to adjust and fuse global contextual information. The adjusted context feature map is added element-wise to the input feature map, thereby improving the model's understanding and ability to respond to the overall scene.

[0078] Step (5): Finally, the classification results of each part are integrated and calculated through the collaborative mechanism to generate the final overall classification output.

[0079] First, the cascaded model obtains the category information and corresponding classification scores for four parts of the ant: head, thorax, abdomen, and whole body. Next, weights are randomly assigned to each part, and a comprehensive score for each category is calculated. Specifically, the formula for the comprehensive score of a category is as follows:

[0080]

[0081] Among them, s i,c w represents the score of the i-th part for category c. i This represents the weight of the i-th part. After calculating the overall score for all categories, the category with the highest score is selected as the final predicted category.

[0082] Step (6): Based on the data processed in Step 1, train the two-stage fine-grained recognition model and complete the insect recognition through the trained two-stage fine-grained recognition model.

[0083] The input image is processed by an object detection network to generate images of key parts, which are then classified using a classification network. After loss calculation, backpropagation is performed to update the model parameters, and the training loss and accuracy for each epoch are recorded.

[0084] If the final predicted label obtained after training is inconsistent with the true label, it is necessary to calculate the loss value using the cross-entropy loss function, the formula of which is:

[0085]

[0086] Where: C is the number of categories; y c It is an indicator of the true category (i.e., when category c is the true category, y is the true category). c =1, otherwise y c =0); It is the probability that the model predicts for category c.

[0087] Next, gradient calculation is performed to adjust the weights. The gradient calculation formula is as follows:

[0088]

[0089] in, y represents the predicted score of category c at location i. c This represents the score of the true category c at part i.

[0090] Finally, the weights are updated using gradient descent, and this process is repeated until the model's predicted class matches the true label. If the labels for all body parts do not match the true label, a classification error is output.

[0091] A three-tiered evaluation system was also introduced: average detection accuracy, classification accuracy, and overall classification accuracy, to comprehensively measure the model's performance. Average detection accuracy assesses the model's accuracy in locating ant body parts, classification accuracy reflects the model's accuracy in classifying individual body parts, and overall classification accuracy demonstrates the model's comprehensive performance across all samples. These metrics, combined, provide a reliable basis for model optimization and improvement.

[0092] The detection average accuracy, Det_mAP, represents the average accuracy of the model across all classes, and is expressed by the formula:

[0093]

[0094] Where C is the number of categories, AP i It is the average precision of the i-th class.

[0095] Partial classification accuracy (Cls_pACC): Represents the classification accuracy of the model across all samples. The formula is:

[0096]

[0097] Where N is the number of samples, y i It is the true label of the i-th sample. is the predicted label of the i-th sample, and 1 is the indicator function, which takes the value 1 when the predicted label is the same as the true label, and 0 otherwise.

[0098] Overall classification accuracy (Cls_oACC): This measures the model's overall classification performance, representing the model's classification accuracy in complete ant samples.

[0099] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments, including components, without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. A two-stage fine-grained insect identification method based on key component co-localization, characterized in that, Includes the following steps: Step (1): Dataset acquisition and processing; Insect datasets were obtained from online and field collection methods based on identification requirements. The datasets were then labeled, with the labeling method involving labeling the entire target object as well as key parts. Step (2): Construct a two-stage fine-grained recognition model; The two-stage fine-grained recognition model consists of an object detection network and a classification network; the object detection network is used to extract images of key parts, and the classification network is used to classify the images of key parts. Step (3): First, the input insect image is detected by the target detection network to obtain the location information of the key parts of the insect to be identified. After obtaining the location information of each key part of the insect to be identified, the image is cropped using this location information to obtain a sub-image containing only a single key part. Then, the cropped images are preprocessed, including resizing and normalization. Then, HOG features are extracted from the preprocessed images and regions with similarity exceeding the set threshold are filtered out to obtain the target image. Step (4): Transmit the target image obtained in step (3) to the classification network to identify the insect category; First, the target image obtained in step (3) is processed by a classification network. The output of the classification network is combined with the center point location information of the key parts of the insect to be identified. The combined feature vector is input into a fully connected layer. Through further feature processing, an enhanced feature vector containing spatial location information and category information is finally generated, which is the classification result of the key parts. The classification network adjusts and fuses the global context information through the global context enhancement module. The adjusted context feature map is added to the input feature map element by element. The specific network structure of the classification network is as follows: The global image and local image are first input into the global context enhancement module to obtain a fused feature map. Then, they pass through a regular convolutional layer with a kernel size of 3*3, 3 channels, and a stride of 2; an MBConv1 module with a stride of 1; an MBConv6 module with a kernel size of 3*3, 3 channels, and a stride of 2; an MBConv6 module with a kernel size of 5*5, 3 channels, and a stride of 2; an MBConv6 module with a kernel size of 3*3, 3 channels, and a stride of 3; an MBConv6 module with a kernel size of 5*5, 3 channels, and a stride of 3; an MBConv6 module with a kernel size of 5*5, 3 channels, and a stride of 4; and an MBConv6 module with a kernel size of 3*3, 3 channels, and a stride of 1. Finally, they pass through a regular convolutional layer with a kernel size of 1*1, a pooling layer, and a fully connected layer. In the classification network, the global context enhancement module has the following specific structure: First, the global feature map is passed through a regular convolutional layer with a kernel size of 1*1 and a ReLU activation function, then element-wise added to the local feature map, and then passed through a regular convolutional layer with a kernel size of 1*1 and a ReLU activation function to obtain the output fused feature map. Step (5): Finally, the classification results of each part are integrated and calculated through the collaborative mechanism to generate the final overall classification output.

2. The two-stage fine-grained insect identification method based on key component co-localization according to claim 1, characterized in that, The target detection method of the target detection network is as follows: The object detection network includes an image encoder and a decoder. The image encoder uses a ResNet50 model to extract high-level features from the original input image. First, features are extracted from the original input image through multiple convolutional layers, and then the ReLU activation function is used to process the features to obtain an image feature map. Then, pooling layers are used to reduce the spatial size of the feature map. Next, Gaussian noise boxes are used to crop the feature map extracted by the image encoder to obtain the target region (RoI). Finally, the target region (RoI) and the Gaussian noise boxes are input into the decoder to extract the multi-scale information of the target. Prediction is performed through convolutional layers with a kernel size of 1*1. The convolutional layers are divided into regression layers and classification layers. The regression layer predicts the bounding box position information of the target object, with 4 channels. The classification layer predicts the category of the key part to which each bounding box belongs. For each bounding box, the classification layer outputs a vector whose length is equal to the number of classes in the dataset. Each position corresponds to the probability distribution of a class. After processing by the softmax function, the probability distribution of each class is obtained. The number of channels is equal to the number of classes. The Gaussian noise boxes are obtained as follows: For any given input feature map C×H×W with 3 input channels, Gaussian noise controlled by variance scheduling is added to the input image according to the noise addition rules of the diffusion detection model, thus obtaining 500 Gaussian noise boxes; the noise boxes are obtained using the following formula: （1） （2） Where t is the time step for adding noise. The noise box at time t is represented. It is a hyperparameter. This represents the noise at time t-1. This represents the actual object detection bounding box during training.

3. The two-stage fine-grained insect identification method based on key component co-localization according to claim 2, characterized in that, The image encoder of the target detection network has the following specific network structure: The input image first passes through a regular convolutional layer with a kernel size of 7*7, 64 channels, and a stride of 2; a max pooling layer with a kernel size of 3*3 and a stride of 2; a BTNK1 module with a stride of 1 and 64 channels; and two BTNK2 modules with 256 channels each; then through one BTNK1 module with a stride of 2, 256 input channels, and 128 output channels; and three BTNK2 modules with 512 channels each; then outputs to one BTNK1 module with a stride of 2, 512 input channels, and 256 output channels; and five BTNK2 modules with 1024 channels each; finally, it passes through one BTNK1 module with a stride of 2, 1025 input channels, and 512 output channels; and two BTNK2 modules with 2048 channels each, outputting a feature map of shape (2048, 7, 7).

4. The two-stage fine-grained insect identification method based on key component co-localization according to claim 3, characterized in that, In the image encoder, the BTNK1 module has four parameters: C, W, C1, S; Where S represents the stride in the convolutional layer; when S is 1, the input and output sizes are the same, indicating no downsampling is performed; C1 represents the number of feature maps output by the convolutional layer, i.e., the number of output channels; C represents the number of input channels; W represents the input size, i.e., length and width; the BTNK1 module processes the input image in two paths. First, the input image is processed through a convolutional layer with a kernel size of 1*1, C1 channels, and a stride of S, followed by a BN layer and a ReLU activation function; then it is processed through a convolutional layer with a kernel size of 3*1.

3. A convolutional layer with C1 channels and a stride of 1 is passed through the input image, followed by a batch normalization (BN) layer and a ReLU activation function. Finally, a convolutional layer with a kernel size of 1*1, C1*4 channels, and a stride of S is passed through the input image, followed by a BN layer. The second path processes the input image through a convolutional layer with a kernel size of 1*1, C1*4 channels, and a stride of S, followed by a BN layer. The outputs of the two paths are then added together, and the image is output as an image with a shape of (C1*4, W / S, W / S) after passing through the ReLU activation function. The BTNK2 module has two parameters: C and W; where C represents the number of input channels; and W represents the input dimensions, i.e., length and width. The BTNK2 module processes the input image in two paths. The first path processes the input image through a convolutional layer with a kernel size of 1*1, C / 4 channels, and a stride of 1, followed by a batch normalization (BN) layer and a ReLU activation function. Next, it passes through a convolutional layer with a kernel size of 3*3, C / 4 channels, and a stride of 1, followed by another BN layer and a ReLU activation function. Finally, it passes through a convolutional layer with a kernel size of 1*1, C channels, and a stride of 1, followed by another BN layer. The second path directly outputs the input image. The outputs from the two paths are then summed and passed through a ReLU activation function to output an image of shape (C, W, W).

5. The two-stage fine-grained insect identification method based on key component co-localization according to claim 1, characterized in that, In the classification network, the MBConv1 module with a stride of 1 operates as follows: First, the input fused feature map is passed through an up-dimensional convolution with a kernel size of 1*1, a BN layer, and a Swish activation layer. Then, it is passed through a depthwise separable convolution with a kernel size of k*k and a stride of 1, a BN layer, and a Swish activation layer. Next, it is passed through an SE module, a down-dimensional convolution with a kernel size of 1*1, and a BN layer. Finally, it is passed through a Dropout layer and then shortened with the initial input feature map of MBConv1 to obtain the module output. The MBConv6 module with a stride of 1 operates as follows: First, the input feature map is passed through an up-dimensional convolution with a kernel size of 1*1, followed by a BN layer and a Swish activation layer. Then, it is passed through a depthwise separable convolution with a kernel size of k*k and a stride of 1, followed by a BN layer and a Swish activation layer. Next, it is passed through an SE module, a down-dimensional convolution with a kernel size of 1*1, and a BN layer. Finally, it is passed through a Dropout layer and then shortened with the initial input feature map of MBConv6 to obtain the module output. The MBConv6 module with a stride of 2 operates as follows: First, the input feature map is processed by an up-dimensional convolution with a kernel size of 1*1, followed by a BN layer and SiLU. Then, it is processed by a depthwise separable convolution with a kernel size of 3*3 and a stride of 2, followed by a BN layer and SiLU. Next, it is processed by the SE module, followed by a down-dimensional convolution with a kernel size of 1*1 and a BN layer. Finally, the module output is obtained through the Dropout operation. The SE module consists of a global average pooling layer and two fully connected layers. The number of nodes in the first fully connected layer is 1 / 4 of the number of channels in the feature map of the corresponding MBConv module, and it uses the Swish activation function. The number of nodes in the second fully connected layer is the number of channels in the output feature map of the depthwise separable convolution within the corresponding MBConv module, and it uses the Sigmoid activation function.

6. The two-stage fine-grained insect identification method based on key component co-localization according to claim 1, characterized in that, The final classification result is calculated through a collaborative mechanism, as follows: First, the two-stage fine-grained recognition model obtains the category information and corresponding classification scores for the key parts of the insect to be identified and the insect as a whole. Next, weights are randomly assigned to each part, and a comprehensive score for each category is calculated. Specifically, the formula for the comprehensive score of each category is as follows: （3） in, Indicates the first Each part corresponds to a category The score, Indicates the first The weights of each part are determined; after calculating the comprehensive score of all categories, the category with the highest score is selected as the final predicted category.

7. A two-stage fine-grained insect identification method based on key site co-localization according to any one of claims 1-6, characterized in that, If the final predicted label obtained during training is inconsistent with the true label, it is necessary to calculate the loss value using the cross-entropy loss function, the formula of which is: （4） in: It is the number of categories; It is an indicator of the true category, i.e., the category. When it is a true category ,otherwise ; It is the model prediction category The probability of; Next, gradient calculation is performed to adjust the weights. The gradient calculation formula is as follows: （5） in, Indicates the prediction category In the location The score on Represents the true category In the location The score on; Finally, the weights are updated using gradient descent, and this process is repeated until the predicted class output by the model matches the true label; if the labels of all parts do not match the true labels, the output is a classification error.

8. A two-stage fine-grained insect identification method based on key component co-localization according to claim 1, characterized in that, To better evaluate the model, a three-level evaluation metric was introduced: average detection accuracy, classification accuracy, and overall classification accuracy, to comprehensively measure the model's performance. Average detection accuracy assesses the model's accuracy in locating key parts of insects, classification accuracy reflects the model's accuracy in classifying each part, and overall classification accuracy demonstrates the model's overall performance across all samples. The combination of these metrics provides a reliable basis for model optimization and improvement. Average accuracy of detection The average accuracy of the model across all categories is expressed by the formula: （6） Where C is the number of categories, It is the first The average precision of the class; Partial classification accuracy : Represents the classification accuracy of the classification model on all samples; the formula is: （7） in, It is the number of samples. It is the first one The true label of the sample It is the first Predicted label for each sample, It is an indicator function that takes the value 1 when the predicted label is the same as the true label, and 0 otherwise; Overall classification accuracy : Used to measure the model's overall classification performance, representing the model's classification accuracy in complete insect samples.

Images