Small target pest detection method based on YOLOv7 algorithm

The improved YOLOv7 algorithm with a transformer encoder and SIoU loss function enhances small target pest detection by capturing contextual information and optimizing anchor boxes, achieving superior accuracy in pest detection tasks.

CN119091119APending Publication Date: 2024-12-06WUHAN POLYTECHNIC UNIVERSITY
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411145192.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Existing YOLO series models face challenges in detecting small target pests due to limitations in capturing global contextual information, the impact of complex backgrounds, and the inefficiency of the CIoU loss function, which affects detection accuracy and anchor box suitability.

Method used

A small target pest detection method using an improved YOLOv7 algorithm with a context information extraction block based on a transformer encoder, a modified loss function (SIoU), and optimized anchor boxes obtained through K-means clustering, enhancing feature extraction and detection accuracy.

Benefits of technology

The method significantly improves detection accuracy for small target pests, achieving a mean average precision of 0.943, surpassing other models, by effectively addressing the limitations of the original YOLOv7 model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091119A_ABST
    Figure CN119091119A_ABST
Patent Text Reader

Abstract

The invention belongs to the field of small target pest detection, and discloses a small target pest detection method based on a YOLOv7 algorithm, and the technical scheme comprises the steps: 1, obtaining a small target pest picture data set needed by a training model; 2, manually marking to identify the positions and types of small target pests in the image, and splitting the data set into a training set, a verification set and a test set; 3, constructing an improved target detection model based on an original YOLOv7 algorithm; 4, constructing a loss function of the improved YOLOv7 model; 5, using a K-means algorithm to obtain the anchors suitable for the small target pests; 6, setting a plurality of training parameters to train the improved YOLOv7 target detection model; compared with an existing detection method, the method has the advantages that the relative precision is improved, and the method has higher recognition capability in a small target pest detection task.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of small target pest detection, and in particular relates to a small target pest detection method based on the YOLOv7 algorithm. Background Art

[0002] While using pesticides appropriately to limit pests, it is also very important to monitor crop pests in order to promptly remind agricultural production personnel to take appropriate protective measures and ensure the smooth progress of agricultural production. With the advancement of science and technology, the way of monitoring pests has changed from manual identification by experts in the past to the current pest detection by computers. At present, the use of computers to detect pests is mainly achieved through target detection models based on deep learning.

[0003] Currently in the field of target detection, the YOLO series of models have attracted widespread attention due to their excellent detection performance. The YOLO series of models currently have multiple versions, such as YOLOv5, YOLOv7, and YOLOv8. YOLOv5 was released by Ultralytics in 2020. It is known for its high speed and high precision, and can perform well in real-time applications. YOLOv7 was released in 2022, which uses more advanced feature extraction and fusion methods to improve the model's expression ability and detection effect. YOLOv8 was released in 2023, introducing a new network architecture and an improved training mechanism, further improving the model's robustness and generalization ability. However, small target pests are relatively complex in background, small in size, and difficult to extract features. Some researchers have conducted relevant research on the detection of small target pests. In 2018, Shen Y et al. improved the Faster R-CNN to solve the problem of detecting small-sized stored-grain pests. They improved the detection accuracy by fusing the features extracted from different layers. The experimental results showed that the mAP of small-sized stored-grain pest detection reached 0.88. Compared with the original model, the improved model has better detection effect on small targets. However, this method did not further explore how to reduce the impact of complex background on the accuracy of detecting small-target pests, so that the model can focus more on the small-target pests themselves and improve the feature extraction ability of small-target pests.

[0004] The shortcomings of the existing YOLO series models: (1) The YOLO series models use convolutional neural networks to extract target features. However, for deep convolutional networks, the convolution operation is essentially a local operation that can only extract features within a fixed receptive field. This locality limits the ability of the convolutional network to capture global contextual information and long-distance dependencies. In complex scenes, the detection task of small target pests is often affected by background, occlusion and other factors, which poses a huge challenge to target detection models based on convolution operations. (2) The CIoU loss function used by the original YOLOv7 does not take into account the direction in which the predicted box and the true box do not match, which is not conducive to regressing the predicted box toward the direction of the true box, which will affect the detection accuracy of small target pests. (3) The default anchor box parameters of the original YOLOv7 are trained using the COCO dataset and cannot reflect the object characteristics of small target pests. Summary of the invention

[0005] In order to solve the deficiencies in the above-mentioned prior art, the object of the present invention is to provide a small target pest detection method based on the YOLOv7 algorithm. The method proposes a context information extraction block based on a transformer encoder, which is introduced into the improved YOLOv7 target detection model based on a convolutional neural network proposed by us, thereby improving the detection accuracy of small target pests.

[0006] The technical solution adopted by this small target pest detection method based on the YOLOv7 algorithm to solve its technical problems is:

[0007] A small target pest detection method based on the YOLOv7 algorithm is provided, including:

[0008] The first step is to obtain the small target pest image dataset required for training the model;

[0009] In the second step, manual labeling was performed to identify the location and type of small target pests present in the image, and the dataset was split into training, validation, and test sets.

[0010] The third step is to build an improved target detection model based on the original YOLOv7 algorithm;

[0011] The fourth step is to construct the loss function of the improved YOLOv7 model;

[0012] Step 5: Use K-means algorithm to obtain anchors suitable for small target pests;

[0013] Step 6: Set multiple training parameters to train the improved YOLOv7 target detection model;

[0014] The seventh step is to take pictures of small target pests to be detected, upload the taken pictures, and infer the uploaded pictures of pests through the trained improved target detection model. During the inference process of the improved target detection model, pests of different sizes are detected through multiple feature extraction layers and multiple detection layers contained in it, and then the position, category and confidence of each target are finally determined through non-maximum suppression steps and confidence screening to obtain the pest recognition results.

[0015] Furthermore, the small target pest image dataset in the first step is a publicly available pest dataset, which contains 284 images, including 5807 whiteflies (WF), 1619 macrolophus (MR) and 688 nesidiocoris (NC).

[0016] Furthermore, the data set in the second step is randomly split into a training set, a validation set and a test set in a ratio of 8:1:1, wherein the splitting of the data set includes: organizing the collected data set; using the labeling software Labelimg to label various pest image samples in the collected pictures; saving the obtained pest labeling information as an XML file, wherein the XML file includes the category of the target object and the coordinate information of the target box; and converting the XML file into a TXT file format that is convenient for YOLOv7 to use.

[0017] Furthermore, the improved YOLOv7 model in the third step includes a backbone network (Backbone), a context information extraction block (Context information extraction block), a neck network (Neck), and a detection head (Head). The backbone network includes a CBS module, an ELAN module, and an MP module. The context information extraction block includes a CBS module, a Flatten operation, a positional encoding operation (Positional encoding), a Transformer Encoder module, and an Unflatten operation. The neck network includes a CBS module, a Cat operation, an ELAN-W module, and an UpSample module. The detection head includes a Detect module, wherein the backbone network, the neck network, and the detection head in the improved YOLOv7 model are consistent with the original YOLOv7 model.

[0018] Furthermore, the process of processing the feature map by the context information extraction block includes:

[0019] Step 1. The input feature map P5 with a shape of 20×20×1024 is first processed by the CBS module to a feature map S1 with a shape of 20×20×256 (the function of this module is to reduce the dimension of the input feature map P5 to save the computational overhead of subsequent operations). The CBS module consists of a Conv layer, a BN layer (batch normalization layer), and a SiLU activation function. The Conv convolution kernel is 1×1, the step size is 1, and the SiLU activation function has the characteristics of no upper bound, lower bound, and non-monotonicity (these characteristics make it more advantageous in the network, especially in deeper networks); the BN layer normalizes the input of the layer by rescaling and centering to make the network training faster and more stable;

[0020] Step 2: Flatten the input feature map S1 to obtain a feature map with a shape of 400×256 (this can be regarded as converting multi-channel two-dimensional data into multi-channel sequence data);

[0021] Step 3: Perform position encoding on the feature map with a shape of 400×256 to obtain a new feature map PE with a shape of 400×256;

[0022] The mathematical expression of the position encoding operation is:

[0023]

[0024] where pos is the position index, ranging from 0 to 399 (for sequences of length 400); i is the index of the embedding dimension, ranging from 0 to 255 (for embeddings of dimension 256), and d model is the dimension of the embedding vector, which is 256;

[0025] Step 4: Perform element wise sum operation on the feature map PE with a shape of 400×256 obtained in step 3 and the feature map with a shape of 400×256 obtained in step 2 to obtain the final feature map X with a shape of 400×256. The position encoding is calculated by sine and cosine functions and added element by element to the input to ensure that the model can recognize the order of each position in the input sequence.

[0026] Step 5: The feature map X with a shape of 400×256 is processed by the Tranformer encoder module; specifically, it is divided into the following steps:

[0027] Step 5.1, calculate multi-head attention; for the input feature map X, obtain the query Q, key K and value V through three linear transformations:

[0028] Q=XW Q (0.25)

[0029] K=XW K (0.26)

[0030] V=XW V (0.27)

[0031] Where W Q , W K and W V are the weight matrices learned during training, and their dimensions are 256×256; subsequently, self-attention is calculated, the similarity between the query and the key is calculated by dot product, and a scaling factor and a softmax function are applied:

[0032]

[0033] where d k is the dimension of the key, is a scaling factor to prevent the dot product result from being too large;

[0034] The self-attention mechanism is expanded to multiple "heads", each head learns a different attention mode; by splicing the outputs of each head and passing through a linear transformation, the model can integrate information from multiple perspectives to better understand the input data; then the outputs of each head are spliced ​​together and passed through a linear transformation to multi-head attention:

[0035] MultiHead(Q,K,V)=Concat(head1,head2,...,head h )W O (0.29)

[0036] Where W O is a learnable linear transformation matrix with a shape of 256×256, head i is the i-th head; each head i The calculation process is as follows:

[0037]

[0038] Here we set the number of heads to 8; then d head Then it is 32; The dimension of is 256×32; the shape of the feature map obtained after multi-head attention calculation is still 400×256, which is consistent with that before multi-head attention calculation; multi-head attention can better capture the semantic relationship of different positions through parallel calculation, thereby improving model performance;

[0039] Step 5.2: Add the output of step 5.1 and the input of the Transformer encoder module, that is, add them element by element to obtain a feature map with a shape of 400×256; then perform a Norm operation on the output of the Add operation, that is, layer normalization, to obtain a feature map with a shape of 400×256; the Add operation here can be regarded as a residual operation, which adds the two parts of the input element by element without changing their shape and size, which can help alleviate the gradient vanishing problem in the deep network and accelerate the training process of the model; the Norm operation refers to layer normalization, which normalizes the output of each layer so that the output of each position has the same distribution, thereby stabilizing the training process and improving the generalization ability of the model;

[0040] Step 5.3: Use the MLP layer to further process the output of step 5.2 to obtain a feature map with a shape of 400×256. The MLP layer consists of two fully connected layers to capture the nonlinear relationship between features and increase the expressiveness of the model.

[0041] Step 5.4: Perform an Add operation on the output of step 5.2 and the output of step 5.3, that is, add them element by element to obtain a feature map with a shape of 400×256; then perform a Norm operation on the output of the Add operation, that is, layer normalization, to obtain a feature map with a shape of 400×256;

[0042] Step 6: Unflatten the feature map with a shape of 400×256 in step 5 to expand it into a feature map S2 with a shape of 20×20×256. Feature map S2 contains rich contextual information. Using it as the input of the Neck part of the model will effectively improve the model's ability to recognize small targets.

[0043] Furthermore, the loss function of the improved YOLOv7 model in the fourth step is divided into three parts: bounding box regression loss, objectness loss and class loss; the confidence loss and class loss in the loss function of the improved YOLOv7 model are consistent with the original YOLOv7; and in the bounding box regression loss part, the SIoU loss function is used to replace the CIoU loss function used in the original YOLOv7 model; the advantage of the SIoU loss function is that it takes into account the direction in which the predicted box and the true box do not match, which helps the predicted box to regress toward the direction of the true box; in the SIoU loss function diagram, rectangle B represents the predicted box, rectangle B represents the true box. GT represents the real box, σ is the rectangle B and rectangle B GT The line connecting the midpoints, c h and cw is the height and width of the rectangle with σ as the diagonal, α and β are the angles between the diagonal and the width and height of the rectangle, W and H are the angles between rectangle B and rectangle B respectively. GT The width and height of the smallest rectangle enclosed; it consists of the following parts:

[0044] ① Angle loss: In order to measure whether the orientation between the predicted box and the true box matches, it is necessary to define the angle loss, which helps to make the predicted box regress to the orientation of the true box during training; the angle loss is defined as:

[0045]

[0046] in,

[0047]

[0048]

[0049] in, Represents rectangle B and rectangle B respectively GT The horizontal coordinate of the midpoint of Represents rectangle B and rectangle B respectively GT The ordinate of the midpoint of

[0050] ②Distance loss: Its mathematical expression is as follows:

[0051]

[0052] in,

[0053]

[0054] γ=2-cost angle (0.38)

[0055] ③ Shape loss: Its mathematical expression is:

[0056]

[0057] Among them, w and h are the width and height of the prediction box respectively, w gt and h gt are the width and height of the real box respectively, and θ is the control cost shape The weight parameter, when θ is 1, cost shape The weight is the largest; therefore, here we take θ to be 4 to avoid cost shape Too much weight;

[0058] ④Intersection over Union loss (IoU loss), calculated as:

[0059]

[0060] That is, the ratio of the area of ​​the intersection of the true box and the predicted box to the area of ​​the union of the true box and the predicted box;

[0061] ⑤The calculation formula of SIoU is:

[0062]

[0063] Furthermore, the anchors in the fifth step can be called anchor boxes, which are a set of predefined bounding boxes with different sizes and aspect ratios, used to represent objects that may exist in the image; these anchor boxes are evenly distributed in the image, and each grid unit contains multiple anchor boxes, representing objects of different sizes and shapes that may appear at that location; through the K-means clustering algorithm, a set of more suitable anchor boxes can be obtained based on the real bounding boxes of the real objects in the data set; this set of anchor boxes is closer to the object features in the data set, and can significantly improve the detection accuracy and efficiency of the improved YOLOv7 model; the original YOLOv7 model generally needs to set 9 anchor boxes;

[0064] (1) First, we collect the ground truth bounding boxes of a total of 8114 objects in the dataset, i.e., the width w and height h of each object. It is worth noting that the width and height here range from 0 to 1, which is the ratio of the width and height of the ground truth bounding box to the width and height of the image.

[0065] (2) In order to use K-means for clustering, we first need to define a suitable distance metric. A commonly used distance metric is the Euclidean distance, which measures the degree of overlap between two bounding boxes. The calculation process is as follows:

[0066]

[0067] , where b i is the i-th true box in the dataset, w i is the width of the i-th ground-truth box, h i is the height of the i-th real box, a k represents the kth anchor box, is the width of the kth anchor box, is the height of the kth anchor box; thus, the goal of the K-means clustering algorithm is to minimize the Euclidean distance between each true bounding box and its nearest anchor box;

[0068] (3) The width and height data of all objects in the dataset are used as input and the K-means clustering algorithm is applied. By specifying the K value (i.e., the number of anchors required) as 9, the algorithm automatically clusters the bounding boxes into 9 classes. The center (i.e., centroid) of each class is an anchor, which represents the typical aspect ratio of the objects in that class.

[0069] (4) The obtained 9 anchor boxes (i.e., the width and height values ​​of the 9 centroids) are applied to the target detection model to replace the original predefined anchor boxes of YOLOv7; when training the model, the model will predict the offset and bounding box based on these optimized anchor boxes, thereby improving the detection accuracy.

[0070] Furthermore, in the training process of the sixth step model, SGD is used as the optimizer, the momentum is set to 0.937, and the weight decay is set to 5×10 -4 The whole training process is 500 epochs, and the batch size is set to 2. In addition, the learning rate is set to 0.0025. The input image size is set to 1280×1280.

[0071] Compared with the prior art, the present invention has the following beneficial effects:

[0072] A small target pest detection method based on the YOLOv7 algorithm exemplified in the present invention improves relative accuracy compared to existing detection methods and has stronger recognition ability in small target pest detection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0074] Figure 1 It is a flow chart of the small target pest detection method based on the YOLOv7 algorithm of the present invention;

[0075] Figure 2 This is a structural diagram of the improved YOLOv7 model of the present invention;

[0076] Figure 3 A block structure diagram for extracting context information of the present invention;

[0077] Figure 4 Schematic diagram of the SIoU loss function of the present invention. DETAILED DESCRIPTION

[0078] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the invention are shown in the accompanying drawings.

[0079] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0080] Embodiment 1:

[0081] like Figure 1 As shown, this embodiment provides a small target pest detection method based on the YOLOv7 algorithm, including:

[0082] The first step is to obtain the small target pest image dataset required for training the model;

[0083] In the second step, manual labeling was performed to identify the location and type of small target pests present in the image, and the dataset was split into training, validation, and test sets.

[0084] The third step is to build an improved target detection model based on the original YOLOv7 algorithm;

[0085] The fourth step is to construct the loss function of the improved YOLOv7 model;

[0086] Step 5: Use K-means algorithm to obtain anchors suitable for small target pests;

[0087] Step 6: Set multiple training parameters to train the improved YOLOv7 target detection model;

[0088] The seventh step is to take pictures of small target pests to be detected, upload the taken pictures, and infer the uploaded pictures of pests through the trained improved target detection model. During the inference process of the improved target detection model, pests of different sizes are detected through multiple feature extraction layers and multiple detection layers contained in it, and then the position, category and confidence of each target are finally determined through non-maximum suppression steps and confidence screening to obtain the pest recognition results.

[0089] In this embodiment, the small target pest image dataset in the first step is a publicly available pest dataset [2], which contains 284 images, including 5807 whiteflies (WF), 1619 macrolophus (MR) and 688 nesidiocoris (NC).

[0090] In this embodiment, the data set in the second step is randomly split into a training set, a validation set and a test set in a ratio of 8:1:1, wherein the splitting of the data set includes: arranging the collected data set; using the labeling software Labelimg to label various pest image samples in the collected pictures; saving the obtained pest labeling information as an XML file, wherein the XML file includes the category of the target object and the coordinate information of the target box; and converting the XML file into a TXT file format that is convenient for YOLOv7 to use.

[0091] In this embodiment, the improved YOLOv7 model in the third step includes a backbone network (Backbone), a context information extraction block (Context information extraction block), a neck network (Neck), and a detection head (Head). The backbone network includes a CBS module, an ELAN module, and an MP module. The context information extraction block includes a CBS module, a Flatten operation, a positional encoding operation (Positional encoding), a Transformer Encoder module, and an Unflatten operation. The neck network includes a CBS module, a Cat operation, an ELAN-W module, and an UpSample module. The detection head includes a Detect module, wherein the backbone network, the neck network, and the detection head in the improved YOLOv7 model are consistent with the original YOLOv7 model.

[0092] The process of processing the feature map by the context information extraction block includes:

[0093] Step 1. The input feature map P5 with a shape of 20×20×1024 is first processed by the CBS module to a feature map S1 with a shape of 20×20×256 (the function of this module is to reduce the dimension of the input feature map P5 to save the computational overhead of subsequent operations). The CBS module consists of a Conv layer, a BN layer (batch normalization layer), and a SiLU activation function. The Conv convolution kernel is 1×1, the step size is 1, and the SiLU activation function has the characteristics of no upper bound, lower bound, and non-monotonicity (these characteristics make it more advantageous in the network, especially in deeper networks); the BN layer normalizes the input of the layer by rescaling and centering to make the network training faster and more stable;

[0094] Step 2: Flatten the input feature map S1 to obtain a feature map with a shape of 400×256 (this can be regarded as converting multi-channel two-dimensional data into multi-channel sequence data);

[0095] Step 3: Perform position encoding on the feature map with a shape of 400×256 to obtain a new feature map PE with a shape of 400×256;

[0096] The mathematical expression of the position encoding operation is:

[0097]

[0098] where pos is the position index, ranging from 0 to 399 (for sequences of length 400); i is the index of the embedding dimension, ranging from 0 to 255 (for embeddings of dimension 256), and d model is the dimension of the embedding vector, which is 256;

[0099] Step 4: Perform element wise sum operation on the feature map PE with a shape of 400×256 obtained in step 3 and the feature map with a shape of 400×256 obtained in step 2 to obtain the final feature map X with a shape of 400×256. The position encoding is calculated by sine and cosine functions and added element by element to the input to ensure that the model can recognize the order of each position in the input sequence.

[0100] Step 5: The feature map X with a shape of 400×256 is processed by the Tranformer encoder module; specifically, it is divided into the following steps:

[0101] Step 5.1, calculate multi-head attention; for the input feature map X, obtain the query Q, key K and value V through three linear transformations:

[0102] Q=XW Q (0.47)

[0103] K=XW K (0.48)

[0104] V=XW V (0.49)

[0105] Where W Q , W K and W V are the weight matrices learned during training, and their dimensions are 256×256; subsequently, self-attention is calculated, the similarity between the query and the key is calculated by dot product, and a scaling factor and a softmax function are applied:

[0106]

[0107] where d k is the dimension of the key, is a scaling factor to prevent the dot product result from being too large;

[0108] The self-attention mechanism is expanded to multiple "heads", each head learns a different attention mode; by splicing the outputs of each head and passing through a linear transformation, the model can integrate information from multiple perspectives to better understand the input data; then the outputs of each head are spliced ​​together and passed through a linear transformation to multi-head attention:

[0109] MultiHead(Q,K,V)=Concat(head1,head2,...,head h )W O (0.51)

[0110] Where W O is a learnable linear transformation matrix with a shape of 256×256, head i is the i-th head; each head i The calculation process is as follows:

[0111]

[0112] Here we set the number of heads to 8; then d head Then it is 32; The dimension of is 256×32; the shape of the feature map obtained after multi-head attention calculation is still 400×256, which is consistent with that before multi-head attention calculation; multi-head attention can better capture the semantic relationship of different positions through parallel calculation, thereby improving model performance;

[0113] Step 5.2: Add the output of step 5.1 and the input of the Transformer encoder module, that is, add them element by element to obtain a feature map with a shape of 400×256; then perform a Norm operation on the output of the Add operation, that is, layer normalization, to obtain a feature map with a shape of 400×256; the Add operation here can be regarded as a residual operation, which adds the two parts of the input element by element without changing their shape and size, which can help alleviate the gradient vanishing problem in the deep network and accelerate the training process of the model; the Norm operation refers to layer normalization, which normalizes the output of each layer so that the output of each position has the same distribution, thereby stabilizing the training process and improving the generalization ability of the model;

[0114] Step 5.3: Use the MLP layer to further process the output of step 5.2 to obtain a feature map with a shape of 400×256. The MLP layer consists of two fully connected layers to capture the nonlinear relationship between features and increase the expressiveness of the model.

[0115] Step 5.4: Perform an Add operation on the output of step 5.2 and the output of step 5.3, that is, add them element by element to obtain a feature map with a shape of 400×256; then perform a Norm operation on the output of the Add operation, that is, layer normalization, to obtain a feature map with a shape of 400×256;

[0116] Step 6: Unflatten the feature map with a shape of 400×256 in step 5 to expand it into a feature map S2 with a shape of 20×20×256. Feature map S2 contains rich contextual information. Using it as the input of the Neck part of the model will effectively improve the model's ability to recognize small targets.

[0117] In this embodiment, the loss function of the improved YOLOv7 model in the fourth step is divided into three parts: bounding box regression loss (Bounding Box Regression Loss), confidence loss (Objectness Loss) and category loss (ClassLoss); wherein, the confidence loss and category loss in the loss function of the improved YOLOv7 model are consistent with the original YOLOv7; and in the bounding box regression loss part, the SIoU loss function is used to replace the CIoU loss function used in the original YOLOv7 model; the advantage of the SIoU loss function is that it takes into account the direction in which the predicted box and the true box do not match, which helps the predicted box to regress toward the direction where the true box is located; the schematic diagram of the SIoU loss function is as shown in Figure 4 As shown, rectangle B represents the prediction box, rectangle B GT represents the real box, σ is the rectangle B and rectangle B GT The line connecting the midpoints, c h and c w is the height and width of the rectangle with σ as the diagonal, α and β are the angles between the diagonal and the width and height of the rectangle, W and H are the angles between rectangle B and rectangle B respectively. GT The width and height of the smallest rectangle enclosed; it consists of the following parts:

[0118] ① Angle loss: In order to measure whether the orientation between the predicted box and the true box matches, it is necessary to define the angle loss, which helps to make the predicted box regress to the orientation of the true box during training; the angle loss is defined as:

[0119]

[0120] in,

[0121]

[0122] in, Represents rectangle B and rectangle B respectively GT The horizontal coordinate of the midpoint of Represents rectangle B and rectangle B respectively GT The ordinate of the midpoint of

[0123] ②Distance loss: Its mathematical expression is as follows:

[0124]

[0125] in,

[0126]

[0127] γ=2-cost angle (0.60)

[0128] ③ Shape loss: Its mathematical expression is:

[0129]

[0130] Among them, w and h are the width and height of the prediction box respectively, w gt and h gt are the width and height of the real box respectively, and θ is the control cost shape The weight parameter, when θ is 1, cost shape The weight is the largest; therefore, here we take θ to be 4 to avoid cost shape Too much weight;

[0131] ④Intersection over Union loss (IoU loss), calculated as:

[0132]

[0133] That is, the ratio of the area of ​​the intersection of the true box and the predicted box to the area of ​​the union of the true box and the predicted box;

[0134] ⑤The calculation formula of SIoU is:

[0135]

[0136] In this embodiment, the anchors in the fifth step can be called anchor boxes, which are a set of predefined bounding boxes with different sizes and aspect ratios, used to represent objects that may exist in the image; these anchor boxes are evenly distributed in the image, and each grid unit contains multiple anchor boxes, representing objects of different sizes and shapes that may appear at this location; through the K-means clustering algorithm, a set of more suitable anchor boxes can be obtained according to the real bounding boxes of the real objects in the data set; this set of anchor boxes is closer to the object features in the data set, and can significantly improve the detection accuracy and efficiency of the improved YOLOv7 model; the original YOLOv7 model generally needs to set 9 anchor boxes;

[0137] (1) First, we collect the ground truth bounding boxes of a total of 8114 objects in the dataset, i.e., the width w and height h of each object. It is worth noting that the width and height here range from 0 to 1, which is the ratio of the width and height of the ground truth bounding box to the width and height of the image.

[0138] (2) In order to use K-means for clustering, we first need to define a suitable distance metric. A commonly used distance metric is the Euclidean distance, which measures the degree of overlap between two bounding boxes. The calculation process is as follows:

[0139]

[0140] , where b i is the i-th true box in the dataset, w i is the width of the i-th ground-truth box, h i is the height of the i-th real box, a k represents the kth anchor box, is the width of the kth anchor box, is the height of the kth anchor box; thus, the goal of the K-means clustering algorithm is to minimize the Euclidean distance between each true bounding box and its nearest anchor box;

[0141] (3) The width and height data of all objects in the dataset are used as input and the K-means clustering algorithm is applied. By specifying the K value (i.e., the number of anchors required) as 9, the algorithm automatically clusters the bounding boxes into 9 classes. The center (i.e., centroid) of each class is an anchor, which represents the typical aspect ratio of the objects in that class.

[0142] (4) The obtained 9 anchor boxes (i.e., the width and height values ​​of the 9 centroids) are applied to the target detection model to replace the original predefined anchor boxes of YOLOv7; when training the model, the model will predict the offset and bounding box based on these optimized anchor boxes, thereby improving the detection accuracy.

[0143] In this embodiment, during the training process of the sixth step model, SGD is used as the optimizer, the momentum is set to 0.937, and the weight decay is set to 5×10 -4 The whole training process is 500 epochs, and the batch size is set to 2. In addition, the learning rate is set to 0.0025. The input image size is set to 1280×1280.

[0144] The comparative experimental results of the method proposed in the present invention and other methods are shown in the following table:

[0145] Model Mean average precision YOLOv5 0.921 YOLOv7 0.923 YOLOv8 0.932 The method proposed by the present invention 0.943

[0146] In this experiment, we compared the detection performance of different target detection models on the small target pest dataset, including YOLOv5, YOLOv7, YOLOv8, and our proposed method; the experimental results show that our method performs well in accuracy, and its mean average precision (mAP) reaches 0.943, which is significantly higher than other models. Among them, the mAP of YOLOv5 is 0.921, the mAP of YOLOv7 is 0.923, and the mAP of YOLOv8 is 0.932; in comparison, our method improves the relative accuracy, indicating that it has stronger recognition ability in the small target pest detection task; this result verifies the effectiveness of our proposed improvement strategy in practical applications.

[0147] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the present application.

[0148] Except for the technical features described in the specification, the remaining technical features are known technologies to those skilled in the art. In order to highlight the innovative features of the present invention, the remaining technical features will not be described here in detail.

Claims

1. A small target pest detection method based on the YOLOv7 algorithm, characterized in that: include: The first step is to obtain the small target pest image dataset required for training the model; In the second step, manual labeling was performed to identify the location and type of small target pests present in the image, and the dataset was split into training, validation, and test sets. The third step is to build an improved target detection model based on the original YOLOv7 algorithm; The fourth step is to construct the loss function of the improved YOLOv7 model; Step 5: Use K-means algorithm to obtain anchors suitable for small target pests; Step 6: Set multiple training parameters to train the improved YOLOv7 target detection model; The seventh step is to take pictures of small target pests to be detected, upload the taken pictures, and infer the uploaded pictures of pests through the trained improved target detection model. During the inference process of the improved target detection model, pests of different sizes are detected through multiple feature extraction layers and multiple detection layers contained in it, and then the position, category and confidence of each target are finally determined through non-maximum suppression steps and confidence screening to obtain the pest recognition results.

2. A small target pest detection method based on YOLOv7 algorithm according to claim 1, characterized in that: The small target pest image dataset in the first step contains 284 images, including 5807 whiteflies, 1619 short-necked long-necked stink bugs and 688 tobacco stink bugs.

3. The small target pest detection method based on the YOLOv7 algorithm according to claim 1 is characterized in that: The data set in the second step is randomly split into a training set, a validation set and a test set in a ratio of 8:1:1, wherein the splitting of the data set includes: organizing the collected data set; using the labeling software Labelimg to label various pest image samples in the collected pictures; saving the obtained pest labeling information as an XML file, wherein the XML file includes the category of the target object and the coordinate information of the target box; and converting the XML file into a TXT file format that is convenient for YOLOv7 to use.

4. The small target pest detection method based on the YOLOv7 algorithm according to claim 1 is characterized in that: The improved YOLOv7 model in the third step includes a backbone network, a context information extraction block, a neck network, and a detection head. The backbone network includes a CBS module, an ELAN module, and an MP module. The context information extraction block includes a CBS module, a Flatten operation, a position encoding operation, a Transformer Encoder module, and an Unflatten operation. The neck network includes a CBS module, a Cat operation, an ELAN-W module, and an UpSample module. The detection head includes a Detect module. The backbone network, the neck network, and the detection head in the improved YOLOv7 model are consistent with the original YOLOv7 model.

5. The small target pest detection method based on the YOLOv7 algorithm according to claim 4 is characterized in that: The process of processing the feature map by the context information extraction block includes: Step 1. The input feature map P5 with a shape of 20×20×1024 is first processed by the CBS module to a feature map S1 with a shape of 20×20×256. The CBS module consists of a Conv layer, a BN layer, and a SiLU activation function. The Conv convolution kernel is 1×1, the step size is 1, and the SiLU activation function has the characteristics of no upper bound, lower bound, and non-monotonic. The BN layer normalizes the input of the layer by rescaling and centering to make the network training faster and more stable. Step 2: Flatten the input feature map S1 to obtain a feature map with a shape of 400×256. Step 3: Perform position encoding on the feature map with a shape of 400×256 to obtain a new feature map PE with a shape of 400×256. The mathematical expression of the position encoding operation is: where pos is the position index, ranging from 0 to 399 for a sequence of length 400; i is the index of the embedding dimension, ranging from 0 to 255 for an embedding of dimension 256, and d model is the dimension of the embedding vector, which is 256; Step 4: Perform element-wise addition operation on the feature map PE with a shape of 400×256 obtained in step 3 and the feature map with a shape of 400×256 obtained in step 2 to obtain the final feature map X with a shape of 400×256. The position encoding is calculated by sine and cosine functions and added to the input element-wise to ensure that the model can recognize the order of each position in the input sequence. Step 5: The feature map X with a shape of 400×256 is processed by the Tranformer encoder module; specifically, it is divided into the following steps: Step 5.1, calculate multi-head attention; for the input feature map X, obtain the query Q, key K and value V through three linear transformations: Q=XW Q (0.3) K=XW K (0.4) V=XW V (0.5) Where W Q , W K and W V are the weight matrices learned during training, and their dimensions are 256×256; subsequently, self-attention is calculated, the similarity between the query and the key is calculated by dot product, and a scaling factor and a softmax function are applied: where d k is the dimension of the key, is a scaling factor to prevent the dot product result from being too large; The self-attention mechanism is expanded to multiple "heads", each head learns a different attention mode; by splicing the outputs of each head and passing through a linear transformation, the model can integrate information from multiple perspectives to better understand the input data; then the outputs of each head are spliced ​​together and passed through a linear transformation to multi-head attention: MultiHead(Q,K,V)=Concat(head1,head2,...,head h )W O (0.7) Where W O is a learnable linear transformation matrix with a shape of 256×256, head i is the i-th head; each head i The calculation process is as follows: Here we set the number of heads to 8; then d head Then it is 32; and The dimension of is 256×32; the shape of the feature map obtained after multi-head attention calculation is still 400×256, which is consistent with that before multi-head attention calculation; multi-head attention can better capture the semantic relationship of different positions through parallel calculation, thereby improving model performance; Step 5.2: Add the output of step 5.1 to the input of the Transformer encoder module, that is, add them element by element to obtain a feature map with a shape of 400×256; then perform a Norm operation on the output of the Add operation, that is, layer normalization, to obtain a feature map with a shape of 400×256; the Add operation here can be regarded as a residual operation, which adds the two parts of the input element by element without changing their shape and size, which can help alleviate the gradient vanishing problem in the deep network and accelerate the training process of the model; the Norm operation refers to layer normalization, which normalizes the output of each layer so that the output of each position has the same distribution, thereby stabilizing the training process and improving the generalization ability of the model; Step 5.3: Use the MLP layer to further process the output of step 5.2 to obtain a feature map with a shape of 400×256. The MLP layer consists of two fully connected layers to capture the nonlinear relationship between features and increase the expressiveness of the model. Step 5.4: Perform an Add operation on the output of step 5.2 and the output of step 5.3, that is, add them element by element to obtain a feature map with a shape of 400×256; then perform a Norm operation on the output of the Add operation, that is, layer normalization, to obtain a feature map with a shape of 400×256; Step 6: Unflatten the feature map with a shape of 400×256 in step 5 to expand it into a feature map S2 with a shape of 20×20×256.

6. The small target pest detection method based on the YOLOv7 algorithm according to claim, characterized in that: The loss function of the improved YOLOv7 model in the fourth step is divided into three parts: bounding box regression loss, confidence loss and category loss; among them, the confidence loss and category loss in the loss function of the improved YOLOv7 model are consistent with the original YOLOv7; and in the bounding box regression loss part, the SIoU loss function is used to replace the CIoU loss function used by the original YOLOv7 model; in the SIoU loss function, rectangle B represents the prediction box, rectangle B GT represents the real box, σ is the rectangle B and rectangle B GT The line connecting the midpoints, c h and c w is the height and width of the rectangle with σ as the diagonal, α and β are the angles between the diagonal and the width and height of the rectangle, W and H are the angles between rectangle B and rectangle B respectively. GT The width and height of the smallest rectangle enclosed; it consists of the following parts: ① Angle loss: Angle loss is defined as: in, in, Represents rectangle B and rectangle B respectively GT The horizontal coordinate of the midpoint of Represents rectangle B and rectangle B respectively GT The ordinate of the midpoint of ②Distance loss: Its mathematical expression is as follows: in, γ=2-cost angle (0.16) ③ Shape loss: Its mathematical expression is: Among them, w and h are the width and height of the prediction box respectively, w gt and h gt are the width and height of the real box respectively, and θ is the control cost shape The weight parameter, when θ is 1, cost shape The weight is the largest; therefore, here we take θ to be 4 to avoid cost shape Too much weight; ④Intersection over Union loss (IoU loss), calculated as: That is, the ratio of the area of ​​the intersection of the true box and the predicted box to the area of ​​the union of the true box and the predicted box; ⑤The calculation formula of SIoU is:

7. The small target pest detection method based on the YOLOv7 algorithm according to claim 1 is characterized in that: The anchors in the fifth step are called anchor boxes, which are a set of predefined bounding boxes with different sizes and aspect ratios, used to represent objects that may exist in the image; these anchor boxes are evenly distributed in the image, and each grid cell contains multiple anchor boxes, representing objects of different sizes and shapes that may appear at that location; through the K-means clustering algorithm, a set of more suitable anchor boxes can be obtained based on the real bounding boxes of the real objects in the data set; The original YOLOv7 model sets 9 anchor boxes; (1) First, we collect the true bounding boxes of the 8114 objects in the dataset, i.e., the width w and height h of each object. The width and height here range from 0 to 1, which is the ratio of the width and height of the true bounding box to the width and height of the image. (2) Define a Euclidean distance, the calculation process is as follows: where b i is the i-th true box in the dataset, w i is the width of the i-th ground-truth box, h i is the height of the i-th real box, a k represents the kth anchor box, is the width of the kth anchor box, is the height of the kth anchor box; the goal of the K-means clustering algorithm is to minimize the Euclidean distance between each true bounding box and its nearest anchor box; (3) The width and height data of all objects in the dataset are used as input and the K-means clustering algorithm is applied. By specifying the K value as 9, the algorithm automatically clusters the bounding boxes into 9 classes. The center of each class is an anchor, which represents the typical aspect ratio of the objects in that class. (4) The obtained 9 anchor boxes are applied to the target detection model to replace the original predefined anchor boxes of YOLOv7; when training the model, the model will predict the offset and bounding box based on these optimized anchor boxes, thereby improving the detection accuracy.

8. The small target pest detection method based on the YOLOv7 algorithm according to claim 1 is characterized in that: In the training process of the sixth step model, SGD is used as the optimizer, the momentum is set to 0.937, and the weight decay is set to 5×10 -4 The whole training process is 500 epochs, and the batch size is set to 2. In addition, the learning rate is set to 0.0025. The input image size is set to 1280×1280.

Citation Information

Cited By

  • Lightweight Yolov10 unmanned aerial vehicle remote sensing wheat ear real-time detection method

    CN119418212A

  • Timing sequence picture recognition method and device for hanging archive bulk materials and medium

    CN120182730A