Agricultural pest image detection and classification method and system

Abstract

The application discloses an agricultural pest image detection and classification method and system, and the method comprises the following steps: S1, pre-processing the obtained agricultural pest image to be classified, and dividing the pre-processed agricultural pest image to be classified into a training set and a test set; S2, taking an image classification model as an agricultural pest image classification model, wherein the image classification model is an improved YoloX model, the YoloX model is taken as a basic architecture, a darknet53 feature extraction module is replaced by a Swin-Transformer module, a DG module and a CLFM module are added to a FPN feature fusion module of a neck network, and a Focalloss loss function is set; S3, training the image classification model by using the training set; S4, detecting the test set by using the trained image classification model, and outputting an agricultural pest image detection and classification result. The application improves the accuracy of agricultural pest image detection and classification, improves the YoloX model, introduces a Focalloss target loss function, and improves the detection and classification effect and adaptability of the agricultural pest image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image detection, in particular to an agricultural pest image detection and classification method and system. BACKGROUND

[0002] China is a big agricultural country, and pest integrated control as an important strategy of agricultural production plays a crucial role in sustainable agricultural development. Different types of pests invade the crops every year during the cultivation process, which causes the crops to decrease in yield and quality to varying degrees. In severe cases, it may even lead to large-scale crop failure. Accurate and effective classification and identification of insects is an important prerequisite for timely pest control and avoiding significant economic losses to crops.

[0003] Traditional pest identification and detection mainly rely on insect experts or insect classification personnel to identify and identify based on professional knowledge and research experience or reference literature. With the continuous development of the Internet, the use of information technology provides a new method and idea for crop pest identification.

[0004] The key to whether the mainstream machine learning image classification model can achieve good recognition effect lies in the extraction of image features. The mainstream method in the industry is to extract the edge contour and other features of the object in the image. However, this method has poor independence of classification results, which affects the accuracy of agricultural pest image detection and classification. SUMMARY

[0005] The present application provides an agricultural pest image detection and classification method and system to solve the problems of low accuracy and large classification precision fluctuation in agricultural pest image detection and classification.

[0006] To achieve the above effects, the technical scheme of the present application is as follows:

[0007] An agricultural pest image detection and classification method, comprising the following steps:

[0008] S1: Preprocessing the obtained agricultural pest image to be classified, and dividing the preprocessed agricultural pest image to be classified into a training set and a test set;

[0009] S2: Taking the image classification model as the classification model of the agricultural pest image, the image classification model being an improved YoloX model, taking the YoloX model as the basic architecture, replacing the darknet53 feature extraction module with the Swin-Transformer module, adding the DG module and the CLFM module to the FPN feature fusion module of the neck network, and setting the Focalloss loss function;

[0010] The DG module comprises a channel attention mechanism and a spatial attention mechanism, which are respectively a DWConv convolution and a GAM attention mechanism; the DG module is used for up-sampling and down-sampling operations on the preprocessed agricultural pest image.

[0011] S3: training the image classification model by using the training set;

[0012] S4: detecting the test set by using the trained image classification model, and outputting an agricultural pest image detection classification result.

[0013] In the above scheme, the CLFM module considers the shallow information of the original framework in the neck part, and further introduces the initially extracted feature information dark2. The CLFM stands for Cross-Layer Fusion Module, and its purpose is to fuse and extract feature information of different output sizes.

[0014] Further, the preprocessing in step S1 is specifically: the acquired agricultural pest image is spliced in a Mosaic random scaling, random cropping and random arrangement manner, data augmentation is performed on the agricultural pest image, and a mixup operation is added on the basis of Mosaic.

[0015] Further, the darknet53 feature extraction layer in step S2 is replaced by a Swin-Transformer module, which is specifically:

[0016] The original YoloX model CSPDarknet-53 feature extraction module is replaced by a Swin-Transformer module with the same number of channels, the imported agricultural pest image to be classified is subjected to feature extraction, and first, second, third and fourth feature maps are obtained in turn, the FPN feature fusion module is subjected to secondary up-sampling processing on the fourth feature map, and in the up-sampling process, the secondary up-sampling result is subjected to feature fusion with the third feature map and the second feature map in turn in a top-down order, and second and third fusion feature maps are obtained.

[0017] The first, second, third and fourth feature layers respectively comprise different numbers of Swin-Transformer modules, and the numbers are 3, 6, 6 and 9 in turn.

[0018] Further, the neck network in step S2 comprises a first convolutional layer, a first up-sampling layer, a first GAM block, a first fusion layer, a first C3 layer, a second convolutional layer, a second up-sampling layer, a first GAM block, a second fusion layer, a second C3 layer, a first DG module, a third fusion layer, a third C3 layer, a second DG module, a fourth fusion layer, and a fourth C3 layer connected in sequence.

[0019] Further, the GAM attention mechanism in step S2 comprises a channel attention submodule and a spatial attention submodule; the channel attention submodule uses a three-dimensional arrangement to retain information in three dimensions, and amplifies cross-dimension channel-space dependencies through a two-layer multilayer perceptron (MLP); the spatial attention submodule uses two convolutional layers to fuse spatial information and delete a max-pooling operation.

[0020] Further, the operation process of the DG module is specifically as follows:

[0021] A1: input the C3 layer feature value obtained by the FPN feature fusion module into the DG module;

[0022] A2: the C3 layer feature value is input into DWConv convolution for 1x1 dimension increasing convolution operation, to obtain two one-dimensional vectors, the convolution kernel and the channel number are consistent, and the output channel number is equal to the input channel number;

[0023] A3: the two one-dimensional vectors after the dimension increasing convolution operation are input into the shared fully connected layer in the channel attention submodule for convolution operation and then added, to generate a one-dimensional channel attention feature map after Sigmoid activation;

[0024] A4: multiply the one-dimensional channel attention feature map after Sigmoid activation and the C3 layer feature value to obtain a feature map F' after channel attention adjustment;

[0025] A5: input the feature map F' after channel attention adjustment into the spatial attention submodule, reduce the channel number through a convolution operation with a convolution kernel of 7, reduce the calculation amount, then perform a convolution operation with a convolution kernel of 7 to increase the channel number, and keep the channel number consistent; finally, output a spatial attention feature map through Sigmoid;

[0026] A6: multiply the spatial attention feature map and the feature map F' after channel attention adjustment to obtain an output feature map after the DG module is improved.

[0027] Further, the operation process of the CFLM module in step S2 is specifically as follows:

[0028] P1, the first feature map generated by the Swin-Transformer module is represented as image feature value f1; the second C3 layer feature value in the FPN feature extraction module is represented as image feature value f2; image feature value f1 and image feature value f2 are simultaneously input into the CLFM module as inputs for fusion and extraction of feature information of different output sizes;

[0029] P2, the image feature value f1 is subjected to group convolution operation to obtain a static feature value k1;

[0030] The group convolution includes a plurality of filter groups and is configured to receive a plurality of input channels, divide the plurality of input channels into a plurality of channel groups, and convolve each respective channel group with a respective filter group, wherein each channel group includes one or more channels; the group convolution operation is to group-convolve all adjacent key values within a 2*2 area size in space;

[0031] P3, the static feature value k1 is connected with the image feature value f1, and after the connection operation, a 1*1 convolution operation is performed to obtain an attention matrix A;

[0032] The attention matrix is calculated by multiplying two matrices to obtain attention weights, and then acting on another matrix to obtain overall weights and output;

[0033] P4, the input image feature value f2 is subjected to convolution operation to obtain an image feature value V1;

[0034] P5, the image feature value V1 and the attention matrix A are multiplied to obtain a dynamic feature value k2, and the formula is: ; in the formula, represents an element-wise fusion operation;

[0035] The softmax function formula is: , wherein is the output value of the i-th node, and C is the number of output nodes, i.e., the number of categories; the dynamic feature value k2 is converted into a probability distribution in the range [0, 1] through the softmax function;

[0036] P6, in order to match the size of the dynamic feature value k2, the static feature value k1 is subjected to convolution operation, and the CLT module is the splicing of the static feature value k1 and the dynamic feature value k2.

[0037] Further, S2 sets the Focalloss loss function, specifically:

[0038] The loss function of the YoloX model adopts the Focalloss loss function; the Focalloss loss function FL (p tAs shown below:

[0039]

[0040] In the formula, p t = , p represents the classification confidence of the output, y = 1 represents that the true value confidence is 1, is a modulation factor; γ is a trained modulation coefficient, ranging from [0, 5];

[0041] The loss function of the improved YoloX model is target positioning loss loss_iou , target confidence loss loss_obj , and target category loss FocalLoss respectively; the total loss of the improved YoloX model is as follows: Loss

[0042] Loss = α 1 loss_iou + α 2 loss_obj + α 3 FocalLoss

[0043] In the formula, α 1、 α 2、 α 3 are weight coefficients corresponding to the loss function.

[0044] Further, step S3 specifically comprises: performing transfer learning training using a training set, inputting the training set into the image classification model, and outputting a detection classification result corresponding to the at least one agricultural pest image; inputting the detection classification result corresponding to the at least one agricultural pest image and the category information of the agricultural pest image into a loss function, and outputting a loss result; adjusting the network parameters of the image classification model according to the loss result until the loss function or the number of iterations meets a preset condition; and taking the model obtained when the loss function or the number of iterations meets the preset condition as the trained image classification model.

[0045] Further, in step S4, the evaluation indexes of recall rate, accuracy, average accuracy, average precision, and F1 score are used to evaluate the agricultural pest image detection classification result; and the agricultural pest image whose evaluation indexes all meet the threshold values is output as the pest detection result image, and each evaluation index is as follows:

[0046] ;

[0047] ;

[0048] ​ ;

[0049] ;

[0050] ;

[0051] wherein Recall represents a recall rate, TP represents a number of correctly detected samples, FN represents a number of missed samples; Precision represents an accuracy rate, FP represents a number of false detected samples; AP represents an average accuracy rate; mAP represents an average precision mean, AP ( i ) represents an AP value of the i pest, N C represents a number of pest categories; F 1 represents F a score.

[0052] An agricultural pest image detection and classification system comprises:

[0053] a preprocessing module configured to preprocess an obtained agricultural pest image and divide the preprocessed agricultural pest image into a training set and a test set;

[0054] a YoloX model improvement module configured to use an image classification model as a classification model of the agricultural pest image, wherein the image classification model is an improved YoloX model, the YoloX model is used as a basic architecture, a darknet53 feature extraction module is replaced by a Swin-Transformer module, a DG module and a CLFM module are added to a FPN feature fusion module of a neck network, and a Focalloss loss function is set;

[0055] the DG module comprises a channel attention mechanism and a spatial attention mechanism, the channel attention mechanism and the spatial attention mechanism are respectively a DWConv convolution and a GAM attention mechanism; and the DG module is configured to perform up-sampling and down-sampling operations on the preprocessed agricultural pest image.

[0056] a training module configured to train the image classification model by using the training set;

[0057] a test module configured to detect the test set by using the trained image classification model and output an agricultural pest image detection and classification result.

[0058] Compared with the prior art, the technical scheme of the present application has the following beneficial effects:

[0059] The application improves the accuracy and stability of agricultural pest image detection and classification, while maintaining a high processing rate of the image classification model. The Swin-Transformer module is added to make the feature information extraction more comprehensive. The GAM attention mechanism is added to improve the generalization ability of the image classification model. The DWConv convolution is used to reduce the convolution operation parameter amount. The Focalloss target loss function is introduced to improve the detection and classification effect of agricultural pest images and adaptability. BRIEF DESCRIPTION OF DRAWINGS

[0060] The drawings are only used for illustrative description, and cannot be understood as a limitation on the application; in order to better illustrate the embodiments, some components of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; for those skilled in the art, some well-known structures and their descriptions in the drawings may be omitted for understanding.

[0061] Figure 1 The agricultural pest image detection and classification method flowchart provided by the embodiment of the application is shown in the figure;

[0062] Figure 2 The preprocessed image provided by the embodiment of the application is shown in the figure;

[0063] Figure 3 The CLFM module flowchart provided by the embodiment of the application is shown in the figure;

[0064] Figure 4 The improved YoloX model structure provided by the embodiment of the application is shown in the figure;

[0065] Figure 5a And Figure 5b The agricultural pest image detection and classification result images provided by the embodiments of the application are shown in the figures;

[0066] Figure 6 The agricultural pest image detection and classification result provided by the embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical scheme and advantages of the embodiments of the application clearer, the technical scheme in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0068] EMBODIMENT

[0069] For better understanding, please refer to Figure 1 , Figure 2The application provides an embodiment of an agricultural pest image detection and classification method, comprising the following steps:

[0070] S1: preprocessing the obtained agricultural pest image to be classified, and dividing the preprocessed agricultural pest image to be classified into a training set and a test set;

[0071] S2: taking the image classification model as the classification model of the agricultural pest image, wherein the image classification model is an improved YoloX model, the YoloX model is taken as a basic architecture, a darknet53 feature extraction module is replaced with a Swin-Transformer module, a DG module and a CLFM module are added to a FPN feature fusion module of a neck network, and a Focalloss loss function is set;

[0072] The DG module comprises a channel attention mechanism and a spatial attention mechanism, wherein the channel attention mechanism and the spatial attention mechanism are respectively DWConv convolution and GAM attention mechanism; the DG module is used for up-sampling and down-sampling operations on the preprocessed agricultural pest image;

[0073] S3: training the image classification model by using the training set;

[0074] S4: detecting the test set by using the trained image classification model, and outputting an agricultural pest image detection and classification result.

[0075] It can be understood that when the detection image is classified, multiple prediction results containing confidence values are given for the same picture, the YoloX model determines that the confidence value is the highest, and the final category of the agricultural pest image to be classified is determined; in the ip102 data set, the pest image of different growth stages containing the same category, the YoloX model accurately determines which category the pest belongs to according to the learned features, the YoloX model counts the output pest category of each agricultural pest image to be classified, and saves the result in a txt file; finally, when the detection image is output, the result contains the category of the agricultural pest image to be classified and the matching result and the confidence value.

[0076] The Swin-Transformer module can better capture global features, increase the receptive field, maximize the preservation of global and local features, and effectively reduce the calculation time during model training. Using DWConv convolution and GAM attention mechanism, the parameter amount is reduced, the multi-receptive field fusion is improved, the learning ability of the model for deep features is further improved, and the problem of gradient disappearance during training of the model is prevented; help the model better complete the detection target. The Head part of the original YoloX model introduces a Focalloss loss function in the classification loss to replace the cross-entropy loss function of the original YoloX model, improves the model performance caused by data imbalance, and further improves the learning ability of the model for deep features. One prediction head in the Head structure of the original YoloX model is output by three branches, which realizes the decoupling of the confidence (conf), the frame offset (x, y, w, h) and the class information, and makes the classification and regression more accurate.

[0077] As a preferred technical solution, in the embodiment, the pretreatment in step S1 is specifically: the agricultural pest image obtained is spliced in a manner of Mosaic random scaling, random cropping and random arrangement, data enhancement is performed on the agricultural pest image, and a mixup operation is added on the basis of Mosaic.

[0078] As a preferred technical solution, in the embodiment, the darknet53 feature extraction layer in step S2 is replaced by a Swin-Transformer module, specifically: the original YoloX model CSPDarknet-53 feature extraction module is replaced by a Swin-Transformer module with the same number of channels, the imported agricultural pest image to be classified is subjected to feature extraction, and first, second, third and fourth feature maps are obtained in turn, the FPN feature fusion module performs twice up-sampling processing on the fourth feature map, and in the up-sampling process, the twice up-sampling results are sequentially subjected to feature fusion with the third feature map and the second feature map in a top-down order, and second and third fusion feature maps are obtained.

[0079] The first, second, third and fourth feature layers respectively include different numbers of Swin-Transformer modules, and the numbers are 3, 6, 6 and 9 in turn.

[0080] As a preferred technical solution, in the embodiment, the DWConv convolution in step S2 is as follows:

[0081]

[0082] In the formula, D Kis the length (width, such as 3) of the convolution kernel, M is the number of input channels, N is the number of output channels, D F is the input feature map resolution.

[0083] It should be noted that the pictures of the training set of the present application are uniformly normalized to pictures with a size of 640*640, N very large, 1 / N very small, so the operation amount of 3*3 DW convolution is 1 / 9 of the standard convolution operation amount.

[0084] As shown in Figure 4 As a preferred technical solution, in the embodiment, the neck network in step S2 includes a first convolution layer, a first upsampling layer, a first GAM block, a first fusion layer, a first C3 layer, a second convolution layer, a second upsampling layer, a first GAM block, a second fusion layer, a second C3 layer, a first DG module, a third fusion layer, a third C3 layer, a second DG module, a fourth fusion layer, and a fourth C3 layer connected in sequence.

[0085] As a preferred technical solution, in the embodiment, the GAM attention mechanism in step S2 includes a channel attention sub-module and a spatial attention sub-module; the channel attention sub-module uses a three-dimensional arrangement to retain information in three dimensions, and amplifies cross-dimension channel-space dependencies through a two-layer multilayer perceptron (MLP); and the spatial attention sub-module uses two convolution layers for spatial information fusion and deletes a max-pooling operation.

[0086] It can be understood that the multilayer perceptron (MLP) is an encoder-decoder structure, which is the same as the BAM, and the compression ratio thereof is r; two convolution layers are used in the spatial attention sub-module for spatial information fusion; the spatial attention sub-module uses the same reduction ratio r as the channel attention module, while deleting the pooling operation to further retain feature mapping, and therefore, the spatial attention module sometimes significantly increases the number of parameters.

[0087] It can be understood that the process of adding GAM attention mechanism in the FPN feature pyramid module is that: the agricultural pest image passes through the channel attention submodule to use three-dimensional arrangement to retain information in three dimensions, use a two-layer multilayer perceptron (MLP) to amplify the cross-dimensional channel-space dependency, and perform feature fusion operation with x itself to enter the spatial attention submodule. In order to pay attention to spatial information, two convolution layers are used in the spatial attention submodule to fuse spatial information and delete the max pooling operation; the same reduction ratio r as BAM is used from the channel attention submodule, in order to prevent a significant increase in parameters, a Group convolution with Channel Shuffle is adopted, and then the feature fusion is performed again to obtain the output feature map; the DG module is input into the DWConv convolution after the feature dimension splicing, and then the GAM attention mechanism operation is performed, which reduces the calculation parameter amount while retaining the information to amplify the "global" cross-dimensional interaction.

[0088] As a preferred technical solution, in the embodiment, the operation process of the DG module is specifically:

[0089] A1: input the C3 layer feature value obtained by the FPN feature fusion module into the DG module;

[0090] A2: the C3 layer feature value enters the DWConv convolution to perform 1x1 dimension increasing convolution operation, obtains two one-dimensional vectors, the convolution kernel is consistent with the channel number, and the output channel number is equal to the input channel number;

[0091] A3: the two one-dimensional vectors after the dimension increasing convolution operation are sent to the shared fully connected layer in the channel attention submodule to perform convolution operation and then added, generate a one-dimensional channel attention feature map, and then pass through Sigmoid activation;

[0092] A4: multiply the one-dimensional channel attention feature map after Sigmoid activation and the C3 layer feature value to obtain the channel attention adjusted feature map F';

[0093] A5: input the channel attention adjusted feature map F' into the spatial attention submodule, reduce the channel number through convolution kernel 7, reduce the calculation amount, and then perform a convolution operation with convolution kernel 7, increase the channel number, and keep the channel number consistent; finally, output the spatial attention feature map through Sigmoid;

[0094] A6: multiply the spatial attention feature map and the channel attention adjusted feature map F' to obtain the output feature map of the improved DG module.

[0095] It should be noted that the construction process of the Swin-Transformer module is:

[0096] The incoming input is a sequence Z(l-1) of format [H / 4n, W / 4n, nC];

[0097] After the incoming LayerNorm layer is normalized, a window self-attention W-MSA operation is performed;

[0098] The W-MSA output is added to Z(l-1), and the output is Z'l;

[0099] After the incoming LayerNorm layer is normalized, an MLP operation is performed;

[0100] The MLP output is added to Z'l, and the output is Zl;

[0101] The incoming input is a sequence Zl of format [H / 4n, W / 4n, nC];

[0102] After the incoming LayerNorm layer is normalized, a sliding window self-attention SW-MSA operation is performed;

[0103] The SW-MSA output is added to Zl, and the output is Z'l+1;

[0104] After the incoming LayerNorm layer is normalized, an MLP operation is performed;

[0105] The MLP output is added to Z'l+1, and the output is Zl+1.

[0106] As a preferred technical solution, in the embodiment, the GAM attention mechanism can put all global interaction features and give feature mapping, which is represented as input F1 e RCxHxW, wherein the intermediate state F2 and the output F3 are defined as follows:

[0107]

[0108] Wherein, F1 represents the input state (Input features, agricultural pest image to be classified), F2 represents the intermediate state, and F3 represents the output state (Output features, agricultural pest image detection classification result), Mc and Ms are channel attention feature maps and spatial attention feature maps, respectively, C, H, and W represent the number of channels, image height, and image width, respectively, represents an element-wise fusion operation.

[0109] As a preferred technical solution, in the embodiment, as shown in Figure 3 The operation process of the CFLM module in step S2 is specifically as follows:

[0110] P1, the first feature map generated by the Swin-Transformer module is represented as image feature value f1; the second C3 layer feature value in the FPN feature extraction module is represented as image feature value f2; image feature value f1 and image feature value f2 are simultaneously input into the CLFM module as inputs for fusion and extraction of feature information of different output sizes;

[0111] P2, group convolution operation is performed on image feature value f1 to obtain static feature value k1;

[0112] The group convolution includes a plurality of filter groups and is configured to receive a plurality of input channels, divide the plurality of input channels into a plurality of channel groups, and convolve each respective channel group with a respective filter group, wherein each channel group includes one or more channels; the group convolution operation is to perform group convolution on all adjacent key values within a 2*2 area size in space;

[0113] P3, connecting operation is performed on static feature value k1 and image feature value f1, and after the connecting operation, 1*1 convolution operation is performed to obtain attention matrix A; wherein the 1*1 convolution operation is twice different 1*1 convolution operation, and the first convolution has an activation function and the other does not have an activation function;

[0114] The attention matrix is calculated by multiplying two matrices to obtain attention weights, and then acting on another matrix to obtain overall weights and output;

[0115] P4, 1x1 convolution operation is performed on input image feature value f2 to obtain image feature value V1;

[0116] P5, image feature value V1 and attention matrix A are multiplied to obtain dynamic feature value k2, and the formula is: ; in the formula, represents element-wise fusion operation;

[0117] The formula of the softmax function is: , wherein is the output value of the i-th node, and C is the number of output nodes, i.e. the number of categories; the dynamic feature value k2 is converted into a probability distribution with a range of [0, 1] through the softmax function;

[0118] The formula of the softmax function converts all feature values into 0 to 1 size, and the sum of all feature values is 1;

[0119] P6, in order to match the size of dynamic feature value k2, convolution operation is performed on static feature value k1, and the CLT module is the splicing of static feature value k1 and dynamic feature value k2.

[0120] As a preferred technical solution, in the embodiment, the Focalloss loss function is set, specifically:

[0121] The loss function of the YoloX model adopts the Focalloss loss function to reduce the imbalance problem of positive and negative samples, and increases the weight of inaccurate classification samples in the loss function; the Focalloss loss function FL (p t ) is as follows:

[0122]

[0123] In the formula, p t = , p represents the output classification confidence, y =1 represents that the true value confidence is 1, is a modulation factor; γ is a modulation coefficient for training, and the range is [0, 5];

[0124] When γ is 0, the Focalloss loss function is the CE loss function; The loss contribution of easy-to-classify samples is reduced, thereby increasing the loss proportion of difficult-to-classify samples, and when P t tends to 1, that is, the classification inaccurate sample is an easy-to-classify sample, at this time, the modulation factor tends to 0, and the contribution to the loss is small, that is, the loss proportion of easy-to-classify samples is reduced. When p t is very small, that is, a sample is classified into a positive sample, but the probability that the classification inaccurate sample is a foreground is very small, that is, it is misclassified into a positive sample, at this time, the modulation factor tends to 1, and the total loss FocalLoss is not affected. Therefore, the FocalLoss loss function effectively reduces the influence of simple samples on model training, and focuses on complex samples for training.

[0125] The loss function of the improved YoloX model is respectively a target positioning loss loss_iou , a target confidence loss loss_obj , and a target category loss FocalLoss ; the total loss Loss of the improved YoloX model is as follows:

[0126] Loss = α 1 loss_iou + α 2 loss_obj + α 3 FocalLoss

[0127] In the formula, α 1、 α 2、 α3 are the weight coefficients corresponding to the loss function. loss_iou is the overlap loss of the prediction box and the real box of the agricultural pest image, loss_obj is the cross entropy loss according to the positive and negative samples and whether the feature points contain the prediction result of the detected agricultural pest image, FocalLoss is the category prediction result loss of the real box of the agricultural pest image and the category of the feature points; the target positioning loss value is calculated, and the accuracy of the pest position is evaluated through the recognized and positioned agricultural pest image.

[0128] As a preferred technical solution, in the embodiment, step S3 is specifically: using the training set to perform transfer learning training, so as to solve the problem that the generalization performance of the trained model is not strong due to insufficient pest data; inputting the training set into the image classification model, outputting the detection classification result corresponding to the at least one agricultural pest image; inputting the detection classification result corresponding to the at least one agricultural pest image and the category information of the agricultural pest image into the loss function, and outputting the loss result; adjusting the network parameters of the image classification model according to the loss result until the loss function or the number of iterations meets the preset condition; and taking the model obtained when the loss function or the number of iterations meets the preset condition as the trained image classification model.

[0129] It should be noted that in the embodiment, the size of the agricultural pest image in the input training set is 640x640, the sgd optimizer is used, the learning rate adjustment method is cosine annealing decay, the preset number of iterations is 150, the training batch size is 32, the initial learning rate is 0.01, the minimum learning rate is 0.0001, the freeze training method is adopted to improve the training efficiency and accelerate the convergence, and the first 50 rounds are freeze training and the last 100 rounds are unfreeze training.

[0130] Among them, the first 70% of the training set is subjected to data enhancement using the Mosaic algorithm and data enhancement using the Mixup algorithm, and the last 30% of the training set is not subjected to data enhancement using the Mosaic algorithm and data enhancement using the Mixup algorithm; under the training strategy, not only the images synthesized by data enhancement can enhance the learning ability of the model to local features, thereby improving the generalization ability of the entire model, but also the data set can be more focused on the original picture, so that the model can learn the overall features of the target well.

[0131] As a preferred technical solution, in the embodiment, in step S4, the evaluation indexes of recall rate, accuracy rate, average accuracy rate, average precision mean and F1 score are used to evaluate the agricultural pest image detection classification result; the evaluation indexes for evaluating the pest detection are calculated: recall rate (Recall), accuracy rate (Precision), average accuracy rate (Average precision, AP), average precision mean (Mean of average precision, mAP) and F1 score, and finally the agricultural pest images that meet the threshold values of the evaluation indexes are output as the pest detection result images, and the evaluation indexes are as follows:

[0132] ;

[0133] ;

[0134] ;

[0135] ;

[0136] ;

[0137] wherein, Recall represents the recall rate, TP the number of correctly detected samples, FN the number of missed samples; Precision represents the accuracy rate, FP the number of misdetected samples; AP represents the average accuracy rate; mAP represents the average precision mean, AP ( i ) represents the AP value of the i-th i pest category, N C the number of pest categories; F 1 represents F F1 score.

[0138] As a preferred technical solution, the embodiment further provides an agricultural pest image detection classification system, comprising:

[0139] a preprocessing module, configured to preprocess the obtained agricultural pest image, and divide the preprocessed agricultural pest image into a training set and a test set;

[0140] The YoloX model improvement module is used as an image classification model as an agricultural pest image classification model, the image classification model is an improved YoloX model, the YoloX model is used as a basic architecture, the darknet53 feature extraction module is replaced by a Swin-Transformer module, a DG module and a CLFM module are added to the FPN feature fusion module of the neck network, and a Focalloss loss function is set;

[0141] The DG module comprises a channel attention mechanism and a spatial attention mechanism, and the channel attention mechanism and the spatial attention mechanism are respectively a DWConv convolution and a GAM attention mechanism; the DG module is used for up-sampling and down-sampling operations on the pretreated agricultural pest image;

[0142] The training module is used for training the image classification model by using a training set;

[0143] The testing module is used for detecting a test set by using the trained image classification model, and outputting an agricultural pest image detection classification result.

[0144] The improved YoloX model is used as an agricultural pest image classification model, the image classification model is an improved YoloX model, the YoloX model is used as a basic architecture, the darknet53 feature extraction module is replaced by a Swin-Transformer module, a DG module and a CLFM module are added to the FPN feature fusion module of the neck network, and a Focalloss loss function is set. The Focalloss is introduced to strengthen the image features submitted after the main processing of the image classification model, further improve the detection performance of the improved YoloX model, the improved YoloX model can effectively detect pests in crops, give the type of pest detection, and provide timely and accurate data information for subsequent targeted prevention and control scheme. The improved YoloX model mAP50 index is improved, and the module parameter quantity is lower, and the convergence speed is improved, so that more timely and effective pest control work is realized.

[0145] Figure 5a and Figure 5b 、 Figure 6 The result of the model of the application is shown in the figure, wherein, Figure 5b The parameters in the figure are as follows:

[0146] Calss: represents the type of pest, this figure is the first type in 97 types, and the name is rice leaf roller;

[0147] IOU: indicates the overlapping area size of two detection boxes in the figure, and the figure is 92.96%;

[0148] Confidence: The figure shows that there is a 75.04% probability of being the first class of pests, rice leaf roller;

[0149] Result: The figure shows the comparison between the detection result and the true result of the picture, and MATCH indicates a successful match.

[0150] Figure 6 Parameter diagram:

[0151] Iou=0.50::0.95 area=all indicates that the overlapping area size of the picture real frame and the detection frame is taken, and the AP value is calculated every 0.05 as an interval from 0.5 to 0.95, and then the average value is calculated;

[0152] IOU=0.50 area=all indicates that the ap value calculated by taking the overlapping area size of the picture real frame and the detection frame greater than 0.5 is taken;

[0153] IOU=0.75 area=all indicates that the ap value calculated by taking the overlapping area size of the picture real frame and the detection frame greater than 0.75 is taken;

[0154] Iou=0.50::0.95 area=small On the basis of the original calculation method, the area of the pest needs to be less than 32*32 pixels, and the result represents the ap value of the small object;

[0155] IOU=0.50 area=medium On the basis of the original calculation method, the area of the pest needs to be greater than 32*32 pixels and less than 96*96 pixels, and the result represents the ap value of the medium object;

[0156] IOU=0.75 area=large On the basis of the original calculation method, the area of the pest needs to be greater than 96*96 pixels, and the result represents the ap value of the large object.

[0157] The image classification model of the present application improves the detection confidence of the agricultural pest image to be classified whether for multiple targets or single target, and the mAP index is increased, based on the public agricultural pest data set Ip102, multiple evaluation indexes are used to evaluate the detection result image of the pest and disease, and the results show that the improved YoloX model of the present application has improved AP50 index and model convergence speed under the same training parameter configuration.

[0158] Obviously, the above embodiments of the present application are merely exemplary and are not intended to limit the embodiments of the present application. Based on the above description, any other different forms of changes or variations can be made by those skilled in the art. Here, it is not necessary and impossible to exhaust all the embodiments. Any modification, equivalent replacement and improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the claims of the present application.

Claims

1. An agricultural pest image detection classification method, characterized in that, The method comprises the following steps: S1: preprocessing the obtained agricultural pest image to be classified, and dividing the preprocessed agricultural pest image to be classified into a training set and a test set; S2: taking an image classification model as the classification model of the agricultural pest image, wherein the image classification model is an improved YoloX model, the YoloX model is taken as a basic architecture, a darknet53 feature extraction module is replaced with a Swin-Transformer module, a DG module and a CLFM module are added to an FPN feature fusion module of a neck network, and a Focalloss loss function is set; The DG module comprises a channel attention mechanism and a spatial attention mechanism, and the channel attention mechanism and the spatial attention mechanism are respectively a DWConv convolution and a GAM attention mechanism; The DG module is used for up-sampling and down-sampling operations on the preprocessed agricultural pest image; The operation process of the CLFM module in step S2 is as follows: P1: the first feature map generated by the Swin-Transformer module is represented as an image feature value f1, and the second C3 layer feature value in the FPN feature extraction module is represented as an image feature value f2; the image feature value f1 and the image feature value f2 are simultaneously input into the CLFM module as inputs for fusion and extraction of feature information of different output sizes; P2: the image feature value f1 is subjected to group convolution operation to obtain a static feature value k1; The group convolution includes a plurality of filter groups and is configured to receive a plurality of input channels, divide the plurality of input channels into a plurality of channel groups, and convolve each respective channel group with a respective filter group, wherein each channel group includes one or more channels; the group convolution operation is spatially applied to all adjacent key values within an area size of 2 2. P3: the static feature value k1 is connected with the image feature value f1, and after the connection operation, 1x1 convolution operation is performed to obtain an attention matrix A; The attention matrix is obtained by multiplying two matrices to calculate attention weights, and then acting on another matrix to obtain overall weights and output; P4: the input image feature value f2 is subjected to 1x1 convolution operation to obtain an image feature value V1; P5, multiply the image feature value V1 and the attention matrix A to obtain a dynamic feature value k2, the formula is: ; in the formula, indicates that the fusion operation is performed by elements; P6: in order to match the size of the dynamic feature value k2, the static feature value k1 is subjected to convolution operation, and the CLT module is the splicing of the static feature value k1 and the dynamic feature value k2; S3: training the image classification model by using the training set; S4: detecting the test set by using the trained image classification model, and outputting an agricultural pest image detection classification result.

2. The agricultural pest image detection and classification method according to claim 1, wherein, The preprocessing in step S1 is specifically: splicing the obtained agricultural pest image by means of Mosaic random scaling, random cropping and random arrangement, data augmentation is performed on the agricultural pest image, and a mixup operation is added on the basis of Mosaic.

3. The method of claim 1, wherein the method further comprises: The darknet53 feature extraction layer in step S2 is replaced with a Swin-Transformer module, and the Swin-Transformer module is specifically: The CSPDarknet-53 feature extraction module of the original YoloX model is replaced with a Swin-Transformer module with the same number of channels, and the imported agricultural pest image to be classified is subjected to feature extraction to obtain a first feature map, a second feature map, a third feature map and a fourth feature map in sequence; the FPN feature fusion module sequentially performs twice upsampling processing on the fourth feature map, and in the upsampling process, the twice upsampling results are sequentially subjected to feature fusion with the third feature map and the second feature map in a top-down order to obtain a second fused feature map and a third fused feature map, respectively. The first feature extraction layer, the second feature extraction layer, the third feature extraction layer and the fourth feature layer respectively include different numbers of Swin-Transformer modules, and the numbers are 3, 6, 6 and 9 in sequence.

4. The agricultural pest image detection and classification method of claim 1, wherein, The neck network in step S2 includes a first convolutional layer, a first upsampling layer, a first GAM block, a first fusion layer, a first C3 layer, a second convolutional layer, a second upsampling layer, a first GAM block, a second fusion layer, a second C3 layer, a first DG module, a third fusion layer, a third C3 layer, a second DG module, a fourth fusion layer and a fourth C3 layer connected in sequence.

5. The agricultural pest image detection and classification method of claim 4, wherein, The GAM attention mechanism in step S2 includes a channel attention submodule and a spatial attention submodule; the channel attention submodule uses a three-dimensional arrangement to retain information in three dimensions, and amplifies the cross-dimension channel-space dependency through a two-layer multilayer perceptron (MLP); The spatial attention submodule uses two convolutional layers to fuse spatial information and removes the max pooling operation.

6. The agricultural pest image detection and classification method of claim 5, wherein, The operation process of the DG module is as follows: A1: input the C3 layer feature value obtained by the FPN feature fusion module into the DG module; A2: the C3 layer feature value enters the DWConv convolution for 1x1 dimension lifting convolution operation to obtain two one-dimensional vectors, the convolution kernel and the channel number are consistent, and the output channel number is equal to the input channel number; A3: add the two one-dimensional vectors after the dimension lifting convolution operation to the shared fully connected layer in the channel attention submodule after convolution operation, generate a one-dimensional channel attention feature map after Sigmoid activation; A4: multiply the one-dimensional channel attention feature map after Sigmoid activation by the C3 layer feature value to obtain a feature map F' after channel attention adjustment; A5: input the feature map F' after channel attention adjustment into the spatial attention submodule, reduce the channel number through a convolution kernel of 7, reduce the calculation amount, then perform a convolution operation with a convolution kernel of 7, increase the channel number, and keep the channel number consistent; finally, output a spatial attention feature map through Sigmoid; A6: multiply the spatial attention feature map by the feature map F' after channel attention adjustment to obtain the output feature map of the improved DG module.

7. The method of claim 6, wherein the method further comprises: The formula of the softmax function is: , wherein is the output value of the i-th node, and C is the number of output nodes, i.e., the number of categories; the dynamic feature value k2 is converted into a probability distribution in the range [0, 1] through the softmax function.

8. The method of claim 7, wherein the method further comprises: The Focalloss loss function is set as follows: The loss function of the YoloX model adopts a Focalloss loss function; the Focalloss loss function FL (p t ) is as follows: where p t = , p represents the classification confidence of the output, y = 1 represents that the true value confidence is 1, is a modulation factor; γ is a trained modulation coefficient, ranging in [0, 5]; The loss function of the improved YoloX model is as follows: target positioning loss loss_iou , target confidence loss loss_obj , target category loss FocalLoss ; and total loss of the improved YoloX model Loss : Loss = α 1 loss_iou + α 2 loss_obj + α 3 FocalLoss In the formula, α 1、 α 2、 α 3 are weight coefficients corresponding to the loss function.

9. The method of claim 8, wherein, Step S3 is specifically: using the training set for transfer learning training, inputting the training set into the image classification model, and outputting detection classification results corresponding to at least one agricultural pest image; inputting the detection classification results corresponding to the at least one agricultural pest image and the category information of the agricultural pest image into a loss function, and outputting a loss result; adjusting the network parameters of the image classification model according to the loss result until the loss function or the number of iterations meets a preset condition; and taking the model obtained when the loss function or the number of iterations meets the preset condition as a trained image classification model.

10. An agricultural pest image detection and classification system, comprising: Comprise: A preprocessing module for preprocessing the acquired agricultural pest image, and dividing the preprocessed agricultural pest image into a training set and a test set; A YoloX model improvement module for taking an image classification model as a classification model of the agricultural pest image, wherein the image classification model is an improved YoloX model, the YoloX model is taken as a basic architecture, a darknet53 feature extraction module is replaced by a Swin-Transformer module, a DG module is added to a FPN feature fusion module of a neck network, a CLFM module is set, and a Focalloss loss function is set; The DG module comprises a channel attention mechanism and a spatial attention mechanism, and the channel attention mechanism and the spatial attention mechanism are respectively a DWConv convolution and a GAM attention mechanism; The DG module is used for upsampling and downsampling operations on the preprocessed agricultural pest image; The operation process of the CLFM module in step S2 is specifically: P1, the first feature map generated by the Swin-Transformer module is represented as an image feature value f1, and the second C3 layer feature value in the FPN feature extraction module is represented as an image feature value f2; the image feature value f1 and the image feature value f2 are simultaneously input into the CLFM module as inputs for fusion and extraction of feature information of different output sizes; P2, group convolution operation is performed on the image feature value f1 to obtain a static feature value k1; The group convolution comprises a plurality of filter groups and is configured to receive a plurality of input channels, divide the plurality of input channels into a plurality of channel groups, and convolve each corresponding channel group with a corresponding filter group, wherein each channel group comprises one or more channels; the group convolution operation is to perform group convolution on all adjacent key values within an area size of 2*2 in space; P3, the static feature value k1 is connected with the image feature value f1, and after the connection operation, a 1*1 convolution operation is performed to obtain an attention matrix A; The attention matrix is obtained by multiplying two matrices to calculate the attention weight, and then acting on another matrix to obtain the overall weight and output; P4, 1x1 convolution operation is performed on the input image feature value f2 to obtain an image feature value V1; P5, multiply the image feature value V1 and the attention matrix A to obtain a dynamic feature value k2, the formula is: ; wherein, represents the fusion operation by element; P6, in order to match the size of the dynamic feature value k2, the static feature value k1 is convolved, and the CLT module is the splicing of the static feature value k1 and the dynamic feature value k2. The training module is configured to train the image classification model by using the training set. The testing module is configured to detect the test set by using the trained image classification model, and output an agricultural pest image detection and classification result.

Images