Insect fine-grained picture classification method based on attention mechanism

Abstract

The application discloses an insect fine-grained picture classification method based on an attention mechanism, collects fine-grained pictures of different morphologies of different kinds of insects, deletes low-quality pictures with repetition, blurring and overexposure through an artificial screening mode, so that the remaining pictures meet the characteristics of large intra-class difference and small inter-class difference, then insect main body enhancement processing is carried out, and a data set with a category label is established; then a classification network based on the attention mechanism for insect classification is built and trained, finally, the trained neural network model is used for classifying and detecting fine-grained insect pictures to be detected, so that the category to which the insect in the picture belongs is directly output.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image classification under the computer field, and more specifically relates to an insect fine-grained picture classification method based on an attention mechanism. BACKGROUND

[0002] With the vigorous development of artificial intelligence technology, computer vision technology based on convolutional neural networks has become a fast lane for development. Today, image classification technology has been widely used in transportation, finance, medicine, Internet and other fields. Traditional image classification focuses on classifying images with large differences, such as cats, dogs, fish, and flowers. The images of the above four categories have significant differences, and such problems can achieve a recognition rate not inferior to humans under the existing computer vision technology based on convolutional neural networks. In the fine-grained image classification task, different sub-classes need to be identified, such as distinguishing between Alaskan dogs and husky dogs, British short-haired cats and American short-haired cats, etc. Compared with the traditional image classification scene, the differences between different categories are smaller, and higher learning ability of the network is required.

[0003] Several main ideas of fine-grained image classification algorithm are as follows: fine-tuning method based on general image classification network, positioning and recognition combined method based on strong supervision or weak supervision attention mechanism, and bilinear pooling method based on high-order feature fusion.

[0004] Among them, the weakly supervised attention mechanism method does not need to add extra data on the dataset, and the training process is simple and direct, with higher portability and universality. However, when the classic Squeeze-and-Excitation network or CBAM network is trained through channel or mixed attention mechanism, the calculation process of attention weight only considers the feature information of the input sample, and cannot calculate from the more global information of the category, which has great limitations. At the same time, the data enhancement used in the training process is only for picture classification in ordinary scenes, and does not process the characteristics in the insect scene.

[0005] Identifying insects presents a greater challenge, primarily due to the following factors: 1. Tiny individual size: Compared to mammals and plants, insects are significantly smaller, resulting in even more minute individual characteristics; 2. Vibrant colors: Insects often possess vibrant colors and dense patterns, and in some cases, fine-grained features reside within these subtle differences; 3. Significant background interference: Image classification of insects is significantly affected by background interference, mainly due to two factors. First, the tiny size of insects means that when capturing a clear image of an individual, a clear background is often included, and the distinct patterns of flowers, leaves, etc., in the background can strongly confuse the image; second, many insects in nature have evolved mimicry abilities, blending almost seamlessly into the environment in images, making them extremely difficult to identify. All these factors contribute to the increased difficulty in classifying insects. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a fine-grained insect image classification method based on an attention mechanism. By adding image enhancement processing and an attention mechanism, accurate insect image classification can be achieved quickly.

[0007] To achieve the above-mentioned objectives, this invention provides a fine-grained image classification method for insects based on an attention mechanism, characterized by comprising the following steps:

[0008] (1) Image acquisition and preprocessing;

[0009] Collect fine-grained images of different morphologies of different insect species, and manually screen and remove low-quality images that are duplicate, blurry, or overexposed, so that the remaining images meet the characteristics of large intraclass differences and small interclass differences.

[0010] (2) Image enhancement processing;

[0011] (2.1) Standard image enhancements include: image scaling, random image rotation, random horizontal or vertical image flipping, and random image cropping;

[0012] (2.2) Enhancement of the insect's main body;

[0013] (2.2.1) Generate a two-dimensional class activation graph;

[0014] Using the trained ResNet-101 network, each image P after standard augmentation is processed. i Image features are extracted, and then the gradient of these features is calculated inversely to obtain the image features in the corresponding image P. i The responses at different spatial locations are used to generate a two-dimensional class activation map C. i i = 1, 2, ... represents the image number;

[0015] (2.2.2) Generate a two-dimensional subject mask image based on the class activation graph;

[0016] Set the threshold β; traverse the class activation graph C. i For each pixel in the main mask image M, if the pixel value is less than the threshold β, then that pixel is not present in the main mask image M. i The pixel values ​​in the image are set to 0 if they are not otherwise set to 1, thus obtaining the main body mask image M. i

[0017] (2.2.3) Enhance the main body;

[0018] Edit the image i With the main body mask image M i Multiplying the corresponding pixel values ​​yields the image H representing the main region of the insect. i ;

[0019] Based on insect main region image H i For image P i The background in the image is Gaussian blurred and grayscale processed to achieve P-value for each image. i The main body is enhanced;

[0020] (3) Create a dataset;

[0021] The enhanced images were classified one by one, and insects of the same category were placed in the same folder. The folder was named with the scientific name of the insect and assigned the corresponding category label. Then, the save path of all sample images was written into a text file line by line.

[0022] (4) Build and train an attention-based classification network;

[0023] (4.1) The classification network based on the attention mechanism is based on the neural network ResNet-101. An RS channel attention module is added to the output of ResNet-101 to build a component network based on the attention mechanism.

[0024] (4.2) In each round of training, by reading the save path of the text file, the average sampling strategy is first used to sample all the images. Each time, n different categories are randomly obtained, and m sample images are randomly selected from each category, for a total of n×m sample images as single batch training data.

[0025] (4.3) The training data is first input into the convolutional layer of the ResNet-101 network. The convolutional layer encodes each sample image into a three-dimensional tensor X of size w×h×c. k Where k = 1, 2, ..., n × m, w and h are the length and width of the sample image, and c is the number of channels; the three-dimensional tensor Xk Compressed into a tensor of size 1×c through a pooling layer. Will After passing through a fully connected layer, the softmax function is used to process the data to obtain the predicted probability vectors res of insects belonging to each category in the sample image. Input to the RS channel attention module;

[0026] (4.4) In the RS channel attention module, n×m tensors are first completed using intra-class features or inter-class features. The pairing process yields n×m pairs of tensors; then, the channel weight vector W for each pair of tensors is calculated using a multilayer perceptron (MLP). k :

[0027]

[0028] in, Let k be the k-th tensor pair, k1, k2 ∈ [1, n×m] and k1 ≠ k2;

[0029] Calculate channel attention weights that include global information

[0030]

[0031] Among them, f k X represents k After input to the fully connected layer, according to X k The label classification number is used to find the one-dimensional vector composed of the parameters in the row corresponding to the number in the fully connected layer;

[0032] Channel attention weights tensor We perform weighting to obtain the weighted feature tensor of the k-th sample image.

[0033] Finally, the feature tensor After passing through a fully connected layer, the input is fed into a softmax function to calculate the predicted probability vector of insects belonging to each category in the sample image.

[0034] (4.5) Calculate multiple losses using various loss functions;

[0035] (4.5.1) Calculate the triplet loss value Loss1;

[0036]

[0037] in, Indicates Tensors that are successfully matched using intra-class feature pairing as a baseline Indicates Tensors that were successfully matched using inter-class feature pairing as a baseline;

[0038] (4.5.2) Calculate the sorting loss value Loss2;

[0039]

[0040] Here, y is an indicator vector consisting of 1s or -1s. When an element in the vector is 1, it indicates that res is greater than 1. Conversely, it is -1;

[0041] (4.5.3) Calculate the loss value Loss3;

[0042] Loss3=-(1-p k ) γ lgp k

[0043] Where, p k Representing vectors middle The predicted probability value for the corresponding category; γ is the scaling factor;

[0044] (4.5.4) Calculate the total loss value, Loss;

[0045] Loss = Loss1 + Loss2 + Loss3

[0046] (4.6) Obtain a classification network based on the attention mechanism;

[0047] The algorithm determines whether the total loss value (Loss) of the network has converged or whether the current iteration count has reached a preset maximum value. If both conditions are met, the iteration stops, and the trained attention-based classification network is obtained. Otherwise, the loss function is used to... fin Perform backpropagation and update the network parameters, then return to step (4.2) for the next round of training;

[0048] (5) Real-time classification of insect images;

[0049] Fine-grained images of a certain type of insect are collected, and the images are preprocessed and enhanced according to steps (1) and (2). Then, they are input into an attention-based classification network to output the insect category.

[0050] The objective of this invention is achieved as follows:

[0051] This invention presents an attention-based fine-grained image classification method for insects. It involves collecting fine-grained images of different insect species and their various forms, manually removing duplicate, blurry, and overexposed low-quality images to ensure the remaining images exhibit high intra-class variation and low inter-class variation. Insect subject enhancement processing is then performed, and a dataset with category labels is created. Next, an attention-based classification network for insect classification is built and trained. Finally, the trained neural network model classifies and detects the fine-grained insect images to be tested, directly outputting the category to which the insect in the image belongs.

[0052] Furthermore, the insect fine-grained image classification method based on the attention mechanism of this invention also has the following beneficial effects:

[0053] (1) The present invention can improve the problem of difficult feature extraction caused by the small size of insects by enhancing the insect body;

[0054] (2) When training the classification network, the present invention first controls the composition of the input to be different classes through a sampling strategy and ensures that different classes contain an average number of samples. This can alleviate the imbalance problem between the number of samples of different classes when the dataset is made during the sampling stage.

[0055] (3) The present invention designs a rich semantic channel attention module, which can effectively improve the recognition effect of the network, enrich the semantics of the extracted features, and has good versatility and can be applied to a variety of different scenarios. Attached Figure Description

[0056] Figure 1 This is a flowchart of the insect fine-grained image classification method based on the attention mechanism of the present invention;

[0057] Figure 2 It is an illustration of an insect;

[0058] Figure 3 This is a diagram of a classification network structure based on the attention mechanism;

[0059] Figure 4 This is a structural diagram of the RS channel attention module;

[0060] Figure 5 This is a comparison chart of results from different backbone networks in Insects252;

[0061] Figure 6 This is a comparison chart of the enhancement results of different β pairs on the insect body;

[0062] Figure 7 This is a comparison chart of the enhancement results of different α pairs on the insect body;

[0063] Figure 8The figures show the experimental results of ResNet-101 on different datasets under different sampling strategies;

[0064] Figure 9 This is a performance comparison chart of different attention structures;

[0065] Figure 10 This is a comparison chart showing the impact of different combinations of loss functions on classification;

[0066] Figure 11 This is a comparison chart of RSAML experimental results combined with insect subject enhancement. Detailed Implementation

[0067] The specific embodiments of the present invention will now be described with reference to the accompanying drawings to enable those skilled in the art to better understand the invention. It should be particularly noted that in the following description, detailed descriptions of known functions and designs that might obscure the main content of the invention will be omitted here.

[0068] Example

[0069] Figure 1 This is a flowchart of the insect fine-grained image classification method based on the attention mechanism of the present invention.

[0070] In this embodiment, as Figure 1 As shown, the present invention provides a fine-grained insect image classification method based on an attention mechanism, comprising the following steps:

[0071] S1. Image acquisition and preprocessing;

[0072] Collect fine-grained images of different morphologies of different insect species, and manually screen and remove low-quality images that are duplicate, blurry, or overexposed, so that the remaining images meet the characteristics of large intraclass differences and small interclass differences.

[0073] In this embodiment, we can construct the required private fine-grained insect dataset using both offline and online collection methods. Offline collection emphasizes sampling the same insect from different angles and distinguishes between different body phases to ensure the collection of more meaningful samples and as many different body phases of different insect species as possible, thus enabling the network to learn more accurate features. Similarly, different instars of different insects are collected separately, assigning different category labels to different instars of insects with metamorphosis characteristics. Online collection utilizes manual downloading and web crawling to download images from image websites such as Baidu Image Library, Google, and other official image websites. Finally, all the initially collected insect images need to be manually processed to filter and ensure image quality.

[0074] Ultimately, the preprocessed dataset comprised 9251 insect images, covering 252 different insect species. Figure 2 Some insects were displayed;

[0075] S2. Image enhancement processing;

[0076] S2.1 Standard image enhancements include: image scaling, random image rotation, random horizontal or vertical image flipping, and random image cropping;

[0077] In this embodiment, the various processes of standard enhancement are all conventional techniques and will not be described in detail here;

[0078] S2.2, Enhancement of the insect's main body;

[0079] S2.2.1 Generate a two-dimensional class activation graph;

[0080] Using the trained ResNet-101 network, each image P after standard augmentation is processed. i Image features are extracted, and then the gradient of these features is calculated inversely to obtain the image features in the corresponding image P. i The responses at different spatial locations are used to generate a two-dimensional class activation map C. i i = 1, 2, ... represents the image number;

[0081] S2.2.2 Generating a two-dimensional subject mask image based on class activation graph;

[0082] Set the threshold β; traverse the class activation graph C. i For each pixel in the main mask image M, if the pixel value is less than the threshold β, then that pixel is not present in the main mask image M. i The pixel values ​​in the image are set to 0 if they are not otherwise set to 1, thus obtaining the main body mask image M. i

[0083] S2.2.3, Enhance the main body;

[0084] Edit the image i With the main body mask image M i Multiplying the corresponding pixel values ​​yields the image H representing the main region of the insect. i ;

[0085] Based on insect main region image H i For image P i The background in the image is Gaussian blurred and grayscale processed to achieve P-value for each image. i The main body is enhanced;

[0086] In this embodiment, to preserve the main insect area in the image, the background is first subjected to Gaussian blurring, preserving the background color while reducing the interference of background texture on the main features. Next, the background is processed into grayscale, preserving the background texture while further reducing the interference of background color on the subject features. Both methods weaken the background, visually emphasizing the insect and reducing background interference. This improves upon the problem of difficulty in extracting features due to the small size of insects.

[0087] S3. Create a dataset;

[0088] The enhanced images were classified one by one, and insects of the same category were placed in the same folder. The folder was named with the scientific name of the insect and assigned the corresponding category label. Then, the save path of all sample images was written into a text file line by line.

[0089] S4. Build and train an attention-based classification network;

[0090] S4.1, such as Figure 3 As shown, the classification network based on the attention mechanism is based on the ResNet-101 neural network. An RS channel attention module is added to the output of the ResNet-101 to build a component network based on the attention mechanism.

[0091] S4.2 In each round of training, in order to alleviate the imbalance between the number of samples of different classes when the dataset was created during the sampling phase, a sampling strategy is used to control the composition of the input into different classes and ensure that different classes contain an average number of samples.

[0092] In this embodiment, the sampling strategy includes an average sampling strategy and a weighted sampling strategy, wherein the sampling process of the average sampling strategy is as follows:

[0093] The average sampling strategy does not directly randomly select samples. Instead, it first randomly selects n different categories from all images by reading the save path of the text file, and randomly selects m sample images from each category. If there are fewer than m sample images in a certain category, the same sample will be repeatedly selected until the number of samples m is met, thus selecting a total of n×m sample images.

[0094] The sampling process of the weighted sampling strategy is as follows:

[0095] The weighted sampling strategy biases the sampling process towards extracting more samples from categories that are more difficult to classify, aiming to improve the overall recognition performance. To determine the classification difficulty of each category, a ResNet-101 network is first trained using an existing insect dataset, and the average recognition accuracy (acc) for different insect categories is recorded.j j = 1, 2, ..., M, where M is the total number of insect categories in the insect dataset;

[0096] Then, by reading the save path of the text file, n different categories are randomly selected from all the images, and for each m... j There are m sample images, of which m are drawn from the j-th category. j satisfy:

[0097] m j =2+((batch) size -2*n)*P j )

[0098]

[0099] Among them, batch size This represents the number of samples input into the ResNet-101 network each time it is trained; thus, categories that are more difficult to classify have a higher probability P. j Therefore, it can occupy a larger proportion in each category. By manually controlling the proportion of different categories in a set of training data, the network can pay more attention to those categories that are more difficult to classify, thereby optimizing the training effect.

[0100] Thus far, the weighted sampling strategy has extracted a total of n×m samples. j Sample image.

[0101] Below, we will use the average sampling strategy as an example to illustrate this in detail:

[0102] In this embodiment, by reading the save path of the text file, the average sampling strategy is first used to sample all images. Each time, n different categories are randomly obtained, and m sample images are randomly selected from each category, for a total of n×m sample images as single batch training data.

[0103] S4.3. The training data is first input into the convolutional layer of the ResNet-101 network. The convolutional layer encodes each sample image into a three-dimensional tensor X of size w×h×c. k Where k = 1, 2, ..., n × m, w and h are the length and width of the sample image, and c is the number of channels; the three-dimensional tensor X k Then, it is compressed into a tensor of size 1×c through a pooling layer. Will After passing through a fully connected layer, the softmax function is used to process the data to obtain the predicted probability vectors res of insects belonging to each category in the sample image. Input to the RS channel attention module;

[0104] S4.4 In the RS channel attention module, first complete the n×m tensors using intra-class or inter-class features. The pairing yields n×m pairs of tensors;

[0105] In this embodiment, the structure of the RS channel attention module is as follows: Figure 4 As shown;

[0106] Below, we will explain in detail the process of pairing intra-class or inter-class features. The pairing process for intra-class feature pairs is as follows:

[0107] by Using this as a benchmark, find the relationship with the following formula. The nearest tensor is denoted as

[0108]

[0109] Among them, Label k =Label p express Category Label k and Category Label p They belong to the same category.

[0110] The pairing process for inter-class feature pairs is as follows:

[0111] by Using this as a benchmark, find the relationship with the following formula. The tensor with the furthest distance is denoted as

[0112]

[0113] Among them, Label k ≠Label q express Category Label k and Category Label q They do not belong to the same category.

[0114] Then, the channel weight vector W for each pair of tensors is calculated using a multilayer perceptron (MLP). k :

[0115]

[0116] in, Let k be the k-th tensor pair, k1, k2 ∈ [1, n×m] and k1 ≠ k2;

[0117] Calculate channel attention weights that include global information

[0118]

[0119] Among them, f k X represents k After input to the fully connected layer, according to X k The label classification number is used to find the one-dimensional vector composed of the parameters in the row corresponding to the number in the fully connected layer;

[0120] Channel attention weights tensor We perform weighting to obtain the weighted feature tensor of the k-th sample image.

[0121] Finally, the feature tensor After passing through a fully connected layer, the input is fed into a softmax function to calculate the predicted probability vector of insects belonging to each category in the sample image.

[0122] S4.5 Calculate multiple losses using various loss functions;

[0123] S4.5.1 Calculate the triplet loss value Loss1;

[0124]

[0125] in, Indicates Tensors that are successfully matched using intra-class feature pairing as a baseline Indicates Tensors that were successfully matched using inter-class feature pairing as a baseline;

[0126] S4.5.2 Calculate the sorting loss value Loss2;

[0127]

[0128] Here, y is an indicator vector consisting of 1s or -1s. When an element in the vector is 1, it indicates that res is greater than 1. Conversely, it is -1;

[0129] S4.5.3 Calculate the loss value Loss3;

[0130] Loss3=-(1-p k ) γ lgp k

[0131] Where, p k Representing vectors middle The predicted probability value for the corresponding category; γ is the scaling factor;

[0132] S4.5.4 Calculate the total loss value (Loss);

[0133] Loss = Loss1 + Loss2 + Loss3

[0134] S4.6 Obtain the classification network based on the attention mechanism;

[0135] The algorithm determines whether the total loss value (Loss) of the network has converged or whether the current iteration count has reached a preset maximum value. If both conditions are met, the iteration stops, and the trained attention-based classification network is obtained. Otherwise, the loss function is used to... fin Perform backpropagation and update the network parameters, then return to step S4.2 for the next round of training;

[0136] S5. Real-time classification of insect images;

[0137] Fine-grained images of a certain type of insect are collected, and the images are preprocessed and enhanced according to steps S1 and S2. Then, the images are input into an attention-based classification network to output the insect category.

[0138] experiment

[0139] This paper conducts extensive comparative and ablation experiments to ensure the effectiveness of each part of the network.

[0140] Backbone network selection: by Figure 5 As can be seen, ResNet101 achieved better performance than InceptionV3 on the task described in this paper by using recognition accuracy as the evaluation metric, and ResNet101 was ultimately chosen as the backbone network.

[0141] Effectiveness experiment of insect subject enhancement: The subject segmentation threshold for insect subject enhancement was controlled by two hyperparameters, α and β, respectively, and the hit probability of enhanced samples when using insect subject enhancement was measured. For example... Figure 6 As shown, the optimal performance improvement is achieved when β is 0.25. Figure 7 When β is optimal, it is finally determined that the enhancement effect on the insect body is the best when α is 0.35, at which point the improvement is the highest.

[0142] Sampling strategy experiment: such as Figure 8As shown, the average sampling strategy performs best when n×m=10×4. Meanwhile, when the sampled data is 40, the traditional random sampling strategy performs the worst under different sampling strategies, while the weighted sampling strategy performs the best, slightly better than the average sampling strategy. Furthermore, the training method using the sampling strategy improves performance by 0.7-1.2 percentage points compared to the default sampling method. The experimental results in the table demonstrate that a reasonable sampling strategy can effectively alleviate the problem caused by imbalanced samples across classes. It also shows that considering the classification difficulty of different classes during sampling allows the network to focus more on training these classes during the training process, thereby improving the overall network performance. The effectiveness of the sampling strategy has been experimentally verified.

[0143] RSA module experiment: such as Figure 9 As shown in the experiment, by selecting from various attention generation methods, it can be found that using intra-class feature pairs achieves the best results. At the same time, it can be found that using intra-class and inter-class feature pairs can improve the backbone network by 91.8%.

[0144] Multiple Loss Function Experiments: To verify the effectiveness of fusing multiple loss functions, ablation experiments were designed for different loss functions. 1. CE Loss: In this scenario, only CE loss is used for learning. 2. CE Loss + Triplet Loss: In this scenario, CE Loss and triplet loss are fused simultaneously for training. 3. CE Loss + Triplet Loss + Rank Loss: This represents the simultaneous fusion of three loss functions for training. 4. Focal Loss + Triplet Loss + Rank Loss: The loss composition is the same as 3, but Focal Loss replaces CE Loss. The sampling strategy used is a weighted sampling strategy, with 10 categories and 4 images sampled from each category, resulting in 40 sample images per round. The results show that compared to the traditional training method using only CE Loss, fusing multiple loss functions allows the network to learn more accurate features and improves the model's encoding ability. By fully utilizing the sample information provided by the sampling strategy, multiple loss functions can effectively improve the network's learning ability. It can be observed that the classification accuracy was improved the most, which is precisely the advantage brought by the sampling strategy of dividing the sampled samples into samples with the same label and samples with different labels. The results are as follows. Figure 10 As shown, the results are all optimal.

[0145] Comparative Experiment: The Rich Semantics Attention (RSA) structure proposed in this embodiment incorporates multiple loss functions (Multi-Loss, ML) for training. It is trained using the Balanced Strategy and Weighted Strategy proposed in this chapter. The networks are referred to as RSAML-B and RSAML-W, respectively. Figure 11 As shown, a comparative experiment was conducted on the RSAML-B and RSAML-W structures, combining them with an insect subject enhancement scheme. In different control groups, it was found that the networks achieved improved performance after applying the insect subject enhancement scheme.

[0146] Although the illustrative specific embodiments of the present invention have been described above to enable those skilled in the art to understand the invention, it should be understood that the invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of the present invention are protected.

Claims

1. A fine-grained image classification method for insects based on an attention mechanism, characterized in that, Includes the following steps: (1) Image acquisition and preprocessing; Collect fine-grained images of different morphologies of different insect species, and manually screen and remove low-quality images that are duplicate, blurry, or overexposed, so that the remaining images meet the characteristics of large intraclass differences and small interclass differences. (2) Image enhancement processing; (2.1) Standard image enhancements include: image scaling, random image rotation, random horizontal or vertical image flipping, and random image cropping; (2.2) Enhancement of the insect's main body; (2.2.1) Generate a two-dimensional class activation graph; Each image after standard augmentation is augmented using the trained ResNet-101 network. Image features are extracted, and then the gradient of these features is calculated inversely to obtain the image features in the corresponding image. The responses at different spatial locations are used to generate a two-dimensional class activation map. , Indicates the image number; (2.2.2) Generate a two-dimensional subject mask image based on the class activation graph; Set threshold Traversing the activation graph For each pixel value, if the pixel value is less than the threshold... Then the pixel in the main mask image The pixel values ​​in the image are set to 0 if they are not otherwise set to 1, thus obtaining the main body mask image. ; (2.2.3) Enhance the main body; Image With the main body mask image Multiplying the corresponding pixel values ​​yields an image representing the main region of the insect. ; Based on images of the main body region of insects For the image The background in each image is subjected to Gaussian blur and grayscale processing to achieve the desired effect. The main body is enhanced; (3) Create a dataset; The enhanced images were classified one by one, and insects of the same category were placed in the same folder. The folder was named with the scientific name of the insect and assigned the corresponding category label. Then, the save path of all sample images was written into a text file line by line. (4) Construct and train a classification network based on the attention mechanism; (4.1) The classification network based on the attention mechanism is based on ResNet-101. An RS channel attention module is added to the output end of ResNet-101 to build a component network based on the attention mechanism. (4.2) In each round of training, by reading the save path of the text file, all images are first sampled using an average sampling strategy, and then randomly selected each time. There are several different categories, and each category is randomly selected. Sample images, totaling [number] images. One sample image is used as training data for a single batch; (4.3) The training data is first input into the convolutional layer of the ResNet-101 network. The convolutional layer encodes each sample image into a format of size . 3D tensor ,in, , The length and width of the sample image. Number of channels; 3D tensor Then, it is compressed into a size of [size] through the pooling layer. tensor ,Will After passing through a fully connected layer, the softmax function is used to process the data to obtain the predicted probability vectors of insects belonging to each category in the sample images. Then Input to the RS channel attention module; (4.4) In the RS channel attention module, first complete the process using intra-class features or inter-class features. Tensor The pairings yielded For tensors; then through a multilayer perceptron Calculate the channel weight vector for each pair of tensors : ； in, Indicates the first For tensors, and ; Calculate channel attention weights that include global information : ； in, express After input to the fully connected layer, according to The label classification number is used to find the one-dimensional vector composed of the parameters in the row corresponding to the number in the fully connected layer; Channel attention weights tensor Weighting is performed to obtain the first... Weighted feature tensor of sample images ; Finally, the feature tensor After passing through a fully connected layer, the input is fed into a softmax function to calculate the predicted probability vector of insects belonging to each category in the sample image. ; (4.5) Calculate multiple losses using various loss functions; (4.5.1) Calculate the triplet loss value ; ； in, Indicates Tensors that are successfully matched using intra-class feature pairing as a baseline Indicates Tensors that were successfully matched using inter-class feature pairing as a baseline; (4.5.2) Calculate the ranking loss value ; ； in, It is an indicator vector consisting of 1s or -1s. When an element in the vector is 1, it indicates... Greater than Conversely, it is -1; (4.5.3) Calculate the loss value ; ； in, Representing vectors middle The predicted probability value for the corresponding category; This is the scaling factor; (4.5.4) Calculate the total loss value ; ； (4.6) Obtain a classification network based on the attention mechanism; Determine the total loss of the network If convergence or the current iteration count reaches a preset maximum value, the iteration stops, resulting in a trained attention-based classification network; otherwise, it continues using... Perform backpropagation and update the network parameters, then return to step (4.2) for the next round of training; (5) Real-time classification of insect images; Collect fine-grained images of a certain type of insect, perform image preprocessing and enhancement processing according to steps (1) and (2), and then input them into an attention-based classification network to output the insect category.

2. The fine-grained classification method based on attention mechanism according to claim 1, characterized in that, The sampling process of the average sampling strategy is as follows: By reading the save path of the text file, images are randomly selected from all images. There are several different categories, and each category is randomly selected. Zhang sample images, for a certain category, the number of sample images is less than [number]. Zhang's method involves repeatedly drawing the same sample until the required number of samples is met. Thus, a total of Sample image.

3. The fine-grained classification method based on attention mechanism according to claim 1, characterized in that, The average sampling strategy can also be replaced by a weighted sampling strategy, the specific sampling process of which is as follows: 1) Train a ResNet-101 network using existing insect datasets and record the average recognition accuracy for different insect categories. , , This represents the total number of insect categories in the insect dataset. 2) Randomly select images from all images by reading the save path of the text file. There are 10 different categories, and each category The sample image, of which, the first Number of samples drawn from each category satisfy: ； ； in, This represents the number of samples input into the ResNet-101 network each time it is trained. At this point, the weighted sampling strategy has extracted a total of [number] samples. Sample image.

4. The fine-grained classification method based on attention mechanism according to claim 1, characterized in that, The intra-class feature pairing method is as follows: by Using this as a benchmark, find the relationship with the following formula. The nearest tensor is denoted as ; ； in, express Category tags and Category tags They belong to the same category.

5. The fine-grained classification method based on attention mechanism according to claim 1, characterized in that, The inter-class feature pairing method is as follows: by Using this as a benchmark, find the relationship with the following formula. The tensor with the furthest distance is denoted as ; ； in, express Category tags and Category tags They do not belong to the same category.

Images