A two-stage image detection method for rice leaf roller damage based on anchor-free frames
A two-stage, frameless image detection method for rice leaf folder damage was developed. By utilizing the CSPNEXt network and dynamic sample allocation, the problem of accurate localization of rice leaf folder damage images in complex backgrounds was solved, achieving efficient detection results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-08
- Publication Date
- 2026-03-13
AI Technical Summary
In existing technologies, images of rice leaf roller damage are difficult to locate accurately in complex field backgrounds, and directional detection methods require manually setting a large number of predefined anchor frames, resulting in low detection efficiency.
A two-stage image detection method for rice leaf roller damage without anchor boxes is adopted. The method dynamically generates oriented initial spare boxes through a CSPNEXt basic feature acquisition network, a CSPNEXtPAFPN feature enhancement network, and an anchor box-free oriented initial spare box generation network. The detection accuracy is improved by dynamic sample allocation and loss calculation.
It enables precise location of rice leaf roller infestation areas under complex backgrounds, improves detection accuracy and efficiency, reduces reliance on predefined anchor frames, and enhances detection results.
Smart Images
Figure CN117036960B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of pest damage image recognition technology, specifically a two-stage rice leaf roller damage image detection method based on an anchor-frame-free method. Background Technology
[0002] Rice leaf roller is a common pest in paddy fields, seriously damaging rice yields. To effectively manage and control this pest, various technologies and methods have been adopted in agriculture for monitoring its damage. Rice leaf roller damage detection is a method that uses computer vision and image processing technology to automatically detect and identify pest damage in paddy fields, reducing the workload of agricultural workers. Intelligent detection and identification of rice leaf roller damage in paddy fields is more time-saving and labor-saving compared to manual observation and recording.
[0003] In images of pest damage, the rice leaf roller infestation is characterized by dense, overlapping patterns, arbitrary tilting, and varying lengths. The complex field background further complicates damage detection. Current horizontal detection methods often include a large portion of the infested background within a single bounding box, failing to accurately pinpoint the affected area and hindering effective detection in real-world environments. In contrast, directional detection methods can detect rice leaf roller damage against complex backgrounds, as the directional bounding box precisely encloses the affected area.
[0004] Given that the angle, scale, and aspect ratio of damage in the field vary greatly, the existing directional detection methods all rely on predefined anchor frames to locate damage in the field. This requires setting a large number of predefined anchor frames in advance to obtain more positive samples. Furthermore, the size, aspect ratio, and angle of the predefined anchor frames all need to be manually set to the optimal parameters through a large number of experiments, which is not practical for real-world applications. Summary of the Invention
[0005] The purpose of this invention is to address the shortcomings of existing technologies in detecting rice leaf roller damage images under complex field background conditions, and to provide a two-stage rice leaf roller damage image detection method based on an anchor-free frame to solve the above problems.
[0006] To achieve the above objectives, the technical solution of the present invention is as follows:
[0007] A two-stage image detection method for rice leaf roller damage based on anchor-free frames includes the following steps:
[0008] 11) Acquisition and preprocessing of images of rice leaf folder damage: Acquire the original images of rice leaf folder damage and perform preprocessing such as scale normalization, random flipping, and random rotation.
[0009] 12) Construction of a rice leaf roller damage detection model: A rice leaf roller damage detection model was constructed based on an anchorless oriented initial spare frame generation network.
[0010] 13) Training of the rice leaf folder damage detection model: Input the preprocessed rice leaf folder damage images into the rice leaf folder damage detection model for training.
[0011] 14) Acquisition and preprocessing of images to be detected: Acquire images of rice leaf roller damage to be detected and perform preprocessing;
[0012] 15) Obtaining the image detection results of rice leaf roller damage: Input the preprocessed image of rice leaf roller damage to be detected into the trained rice leaf roller damage detection model to obtain the image detection results of rice leaf roller damage.
[0013] The construction of the rice leaf roller damage detection model includes the following steps:
[0014] 21) The rice leaf roller damage detection model is set up to include the CSPNEXt basic feature acquisition network, the CSPNEXtPAFPN feature enhancement network, the anchorless box orientation initial spare box generation network, and the damage detection box generation network.
[0015] 22) Configure the CSPNEXt basic feature acquisition network;
[0016] 23) Configure the CSPNEXtPAFPN feature enhancement network;
[0017] 24) Set up an initial spare frame network for anchorless orientation;
[0018] The network for generating initial spare bounding boxes without anchor boxes is set as a detection head network, including a convolutional module and three convolutional layers with a kernel size of 1×1 and a stride of 1. The convolutional module includes a convolutional layer with a kernel size of 3×3, a stride of 1, and a padding size of 1, a normalization layer, and an activation function.
[0019] The three-layer feature output of the CSPNeXtPAFPN feature enhancement network shares the same anchorless oriented initial spare box generation network. That is, the three layers of features use the same detection head network, except for the normalization layer in the convolutional module. The final oriented initial spare box prediction results of the three layers of features are as follows:
[0020] The first layer of features, after passing through the anchorless bounding box generation network, yields the results of class convolution, regression convolution, and angular convolution: (BS,1,w1,h1), (BS,4,w1,h1), and (BS,1,w1,h1), respectively. Decoding these convolution results yields the first layer's oriented prediction bounding box P1, where BS represents the number of images processed in the batch.
[0021] The second layer features, after passing through the anchorless oriented initial spare box generation network, yield the results of class convolution, regression convolution, and angular convolution: (BS,1,w2,h2), (BS,4,w2,h2), and (BS,1,w2,h2). These are then decoded to obtain the oriented prediction box P2 for the second layer.
[0022] The third layer features, after passing through the anchorless oriented initial spare box generation network, yield the results of class convolution, regression convolution, and angular convolution: (BS,1,w3,h3), (BS,4,w3,h3), and (BS,1,w3,h3), respectively. Decoding these results yields the oriented predicted bounding box P3 for the third layer.
[0023] This yields the oriented initial spare frames for the anchorless oriented initial spare frame generation network;
[0024] 25) Set up the network for generating the final detection frame of the damage:
[0025] The symptom detection bounding box acquisition module is set to include a feature alignment module and a final detection module. The oriented initial spare boxes generated by the network without anchor boxes are subjected to maximum suppression at different layers, and at most the top K boxes are selected. The feature alignment module makes the feature regions of all the oriented initial spare boxes into feature maps ks*ks*256 with uniform size and number of channels. Then, the two fully connected layers in the final detection module are used to perform classification and regression for fine-tuning to obtain the final oriented detection bounding box.
[0026] The training of the rice leaf roller damage detection model includes the following steps:
[0027] 31) Set up backpropagation of the total loss of the two-stage rice leaf roller damage image detection model. Use 36 epochs for training. Use AdamW optimizer for training. The base learning rate is 0.0005 and the base weight decay factor is 0.05.
[0028] 32) Loss calculation for the network generated by the initial spare frame without anchor frame orientation;
[0029] 33) Calculation of the loss of the network generated by the final detection frame of the damage.
[0030] Setting up the CSPNEXt basic feature acquisition network includes the following steps:
[0031] 41) The CSPNEXt basic feature acquisition network is set to a 5-layer stacked structure, consisting of 1 backbone layer and 4 stage layers;
[0032] 42) The backbone layer is set to consist of three convolutional modules with a kernel size of 3x3 and a padding size of 1 stacked together. The stride of the first convolutional module is 2, and the stride of the next two convolutional modules is 1.
[0033] The convolutional module consists of a stacked convolutional layer, a normalization layer, and an activation function layer. The preprocessed image I = (BS, 3, W, H) is transformed into X through the basal layer.
[0034] 43) Each of the three stages is set to consist of one convolutional module and one cross-stage local layer. The cross-stage local layer consists of three convolutional modules with a kernel size of 1×1, a stride size of 1, and a padding size of 0, and n CSPNEXtBlocks with residual connections.
[0035] CSPNeXtBlock consists of a convolutional module with a kernel size of 3x3, a stride size of 1, and a padding size of 1, and a depthwise separable convolutional module with a kernel size of 5x5, a stride size of 1, and a padding size of 2, with residual connections; X passes through stage layer 1 and outputs X1, X1 is input to stage layer 2 and outputs X2, X2 is input to stage layer 3 and outputs X3;
[0036] 44) The stage layer 4 is set to include one convolutional module, one spatial pyramid pooling module and one cross-stage local layer in sequence. X3 is input to stage layer 4 and the output is X4. The spatial pyramid pooling module passes the input sequentially through multiple 5x5 pooling layers to obtain features of a larger scale range.
[0037] 45) Set n for stage layers 1-4 to be 3, 6, 6, and 3 respectively.
[0038] Setting up the CSPNEXtFPN feature enhancement network includes the following steps:
[0039] 51) Configure the CSPNEXtFPN feature enhancement network as a bidirectional spatial pyramid network.
[0040] One of them is a top-down network, which includes two convolutional modules, two upsampling layers and two top-down layers. The convolutional modules are used to unify the number of channels, the upsampling layers are used to unify the image resolution, and each top-down layer consists of a cross-stage local layer.
[0041] Another is the bottom-up network, which includes two convolutional modules and two bottom-up layers. The two convolutional modules are used for downsampling with a stride of 2. Each bottom-up layer consists of a cross-stage local layer.
[0042] 52) In the top-down network, the output X4 = (BS,1024,20,20) of the CSPNEXt basic feature acquisition network is changed to Y3 = (BS,512,20,20) through the convolution module, and then upsampled to (BS,512,40,40) and stacked with the output X3 = (BS,512,40,40) of the CSPNEXt basic feature acquisition network to form (BS,1024,40,40), which is used as the input of the top-down layer 2;
[0043] The output of layer 2 from top to bottom changes the number of channels through the convolution module to become Y2 = (BS,256,40,40), and after upsampling, it becomes (BS,256,80,80). This is then stacked with the output X2 = (BS,256,80,80) of the CSPNEXt basic feature acquisition network to form (BS,512,80,80), which serves as the input of layer 1 from top to bottom. Its output is denoted as Y1, and Z1 = Y1.
[0044] 53) In the bottom-up network, Z1 is downsampled by the convolution module and stacked with Y2 to form (BS, 512, 40, 40) as the input of the bottom-up layer 1, and its output is denoted as Z2;
[0045] Z2 is downsampled by the convolution module and stacked with Y3 to form (BS, 1024, 20, 20) as the input from the lower layer to the upper layer 2, and its output is denoted as Z3;
[0046] 54) Finally, Z1, Z2 and Z3 are used as the inputs to generate the network as the initial spare frames for anchorless orientation.
[0047] The loss calculation of the anchorless frame-oriented initial spare frame generation network includes the following steps:
[0048] 61) Forward computation of the network generated by the initial spare frame without anchor frame orientation:
[0049] 611) Input the preprocessed image I = (BS,3,W,H) into the CSPNEXt basic feature acquisition network, and sequentially pass through the backbone layer and 4 stage layers. After stage layers 2-4, the output feature maps X2-4 are of size (BS,256,w1,h1), (BS,512,w2,h2) and (BS,1024,w3,h3).
[0050] 612) Feature map X2-4 is input into the CSPNEXtPAFPN feature enhancement network.
[0051] In the top-down network, X4 = (BS,1024,20,20) changes the number of channels through the convolution module to become Y3 = (BS,512,20,20), and then upsamples it to become (BS,512,40,40), which is stacked with X3 = (BS,512,40,40) to form (BS,1024,40,40), which serves as the input to the second layer from the top down.
[0052] The output of layer 2 from top to bottom changes the number of channels through the convolution module to become Y2 = (BS, 256, 40, 40). After upsampling, it becomes (BS, 256, 80, 80) and is stacked with X2 = (BS, 256, 80, 80) to form (BS, 512, 80, 80), which serves as the input of layer 1 from top to bottom. Its output is denoted as Y1, and Z1 = Y1 is set.
[0053] In the bottom-up network, Z1 is downsampled by the convolution module and stacked with Y2 to form (BS, 512, 40, 40), which serves as the input to the bottom-up layer 1. Its output is denoted as Z2.
[0054] Z2 is downsampled by the convolution module and stacked with Y3 to form (BS, 1024, 20, 20) as the input from the lower layer to the upper layer 2, and its output is denoted as Z3;
[0055] Ultimately, Z1, Z2, and Z3 are used as inputs to generate the network from the initial spare frames for anchorless orientation;
[0056] 613) Z1, Z2, and Z3 are input into the anchorless bounding box orientation initial spare box generation network, which includes one convolutional module with a kernel size of 3×3, a stride of 1, and a padding size of 1, and three parallel convolutional modules with a kernel size of 1×1 and a stride of 1, finally obtaining a three-layer feature result.
[0057] The features of the first layer are processed by the anchorless orientation initial spare box generation network to obtain (BS,1,w1,h1), (BS,4,w1,h1), and (BS,1,w1,h1), respectively, representing the results obtained by class convolution, regression convolution, and angular convolution. In the result obtained by regression convolution, 4 represents the distance of the anchor point from the left, top, right, and bottom edges of the predicted box. The result obtained by angular convolution is the orientation angle encoding result of the predicted box.
[0058] 62) Anchorless box orientation generates the initial spare boxes for the distribution of positive and negative samples in the network:
[0059] First, an anchor point (x, y, stride, stride) is set for each point on the 3-layer feature map, for a total of (w1*h1+w2*h2+w3*h3) anchor points;
[0060] The angle result is decoded by the set angle decoder to obtain the orientation angle. Then, the decoded angle result and the regression result are combined and the anchor point is decoded by the set prediction box encoder to obtain the predicted orientation initial spare box. The orientation initial spare boxes of the three layers are set as P1, P2 and P3 respectively.
[0061] The sample allocation uses a dynamic sample allocation criterion, which includes using centroid deviation loss, location regression loss, and sample classification loss. The sum of these three losses is the final loss. Then, a dynamic allocation method is used to determine which positive samples should be assigned to each labeled instance and the number of positive samples.
[0062] 621) First, calculate the anchor points inside all instances and record them as valid anchor points. Anchor points are feature points of the 3-layer feature map.
[0063] 622) Calculate the intersection-union ratio (IUR) of the predicted bounding boxes obtained by decoding all instances and valid anchor point samples. For each instance, take the first m predicted bounding boxes with the largest IUR and calculate the sum of these m largest IURs as the number n of positive samples corresponding to this instance.
[0064] 623) Then, for each instance, select the n anchor point samples with the smallest loss from the loss matrix as positive samples;
[0065] 624) For a given anchor point, if it is assigned to multiple labeled instances at the same time, the instance with the smallest loss is selected as the label of the positive sample, ensuring that an anchor point matches only one labeled instance, but a label has multiple positive samples.
[0066] 63) The loss function of the anchorless orientation initial spare box generation network consists of three parts: classification loss, angle loss, and regression loss. The classification loss is calculated using the positive and negative samples selected in the previous step. The angle loss is calculated using the positive samples. The angle loss is calculated using the positive samples. The regression loss is calculated using the positive samples. The rotation intersection-over-union ratio loss of the predicted box and the corresponding labeled instance is calculated. The sum of the three is the loss of the anchorless orientation initial spare box generation network.
[0067] The loss calculation of the network that generates the final detection box for the harmful condition includes the following steps:
[0068] 71) Forward computation of the network that generates the final detection bounding box for the damage condition:
[0069] Using the tilt feature dimensionality reduction module and the final detection module of the damage detection network, the oriented prediction boxes P1, P2 and P3 of the three feature layers in the anchorless oriented initial spare box generation network are subjected to maximum suppression in different layers. The top K oriented initial spare boxes with the highest classification scores are then input into the damage final detection box generation network. The local features corresponding to the selected oriented initial spare boxes are dimensionality reduced to ks*ks*256. Finally, two fully connected layers are used to perform the final detection.
[0070] 72) Positive and negative sample allocation for the network that generates the final detection bounding boxes for the harmful condition:
[0071] For the top K initial spare bounding boxes, 512 positive and negative samples were selected, with positive samples accounting for 25% and the rest being negative samples. The selection method was that if the rotation intersection-union ratio of the initial spare bounding box and the labeled instance was greater than 0.5, it was a positive sample, and if it was less than 0.5, it was a negative sample.
[0072] 73) Loss calculation for the network generating the final detection bounding box for the damage:
[0073] The loss function of the network that generates the final detection box for the damage consists of two parts: classification loss and regression loss.
[0074] The classification loss is calculated using the positive and negative samples selected in the previous step.
[0075] CrossEntropyLoss loss and regression loss are calculated using positive samples. SmoothL1Loss loss is calculated, and the sum of the two is the loss of the network that generates the final detection box for the harmful condition.
[0076] Beneficial effects
[0077] The present invention provides a two-stage image detection method for rice leaf roller damage based on an anchorless frame. Compared with the prior art, it proposes a two-stage detection network that is more consistent with the growth environment of damage in the field, and the use of directional frames can more accurately locate the diseased area.
[0078] The anchorless orientation initial spare bounding box acquisition network proposed in this invention does not require setting predefined anchor boxes; it directly generates orientation initial spare bounding boxes based on feature points. Furthermore, it dynamically selects positive samples during training based on labeled instances, ensuring that each labeled instance can match a positive sample. Compared to horizontal detection methods, it has the advantage of accurate localization. Compared to other one-stage orientation algorithms, this invention has the advantage of high detection accuracy; compared to other two-stage orientation algorithms, it has higher quality initial spare bounding boxes and better detection performance. Attached Figure Description
[0079] Figure 1 This is a sequence diagram of the method of the present invention;
[0080] Figure 2a Original image with damage description;
[0081] Figure 2b This is a diagram showing the detection effect of the present invention. Detailed Implementation
[0082] To provide a better understanding of the structural features and effects achieved by the present invention, a detailed description is provided below, accompanied by preferred embodiments and accompanying drawings:
[0083] like Figure 1 As shown, the present invention provides a two-stage image detection method for rice leaf roller damage based on an anchorless frame, comprising the following steps:
[0084] The first step is to acquire and preprocess images of rice leaf roller damage: acquire the original images of rice leaf roller damage and perform preprocessing such as scale normalization, random flipping, and random rotation.
[0085] The second step is to construct a rice leaf roller damage detection model: a rice leaf roller damage detection model is constructed based on an anchorless, oriented initial spare frame generation network.
[0086] The anchorless orientation initial spare box generation network eliminates the need for manually setting a large number of predefined anchor boxes, avoiding the need to fine-tune the angle and size parameters of predefined anchor boxes. Furthermore, each labeled instance can obtain positive samples, and with the same number of initial spare boxes, higher quality initial spare boxes can be obtained, thereby improving the accuracy of lesion detection in the second stage.
[0087] The construction of a detection model for rice leaf roller damage includes the following steps:
[0088] (1) The rice leaf roller damage detection model is set up to include the CSPNEXt basic feature acquisition network, the CSPNEXtPAFPN feature enhancement network, the anchorless box orientation initial spare box generation network, and the damage detection box generation network.
[0089] (2) Set up the CSPNEXt basic feature acquisition network.
[0090] A1) The CSPNEXt basic feature acquisition network is set to consist of a stacked 5-layer structure, which consists of 1 backbone layer and 4 stage layers;
[0091] A2) The backbone layer is set to consist of three convolutional modules stacked together, each with a kernel size of 3x3 and a padding size of 1. The stride of the first convolutional module is 2, and the stride of the next two convolutional modules is 1.
[0092] The convolutional module consists of a stacked convolutional layer, a normalization layer, and an activation function layer. The preprocessed image I = (BS, 3, W, H) is transformed into X through the basal layer.
[0093] A3) Each of the three stages (1-3) consists of one convolutional module and one cross-stage local layer. The cross-stage local layer consists of three convolutional modules with a kernel size of 1×1, a stride size of 1, and a padding size of 0, and n CSPNEXtBlocks with residual connections.
[0094] CSPNeXtBlock consists of a convolutional module with a kernel size of 3x3, a stride size of 1, and a padding size of 1, and a depthwise separable convolutional module with a kernel size of 5x5, a stride size of 1, and a padding size of 2, with residual connections; X passes through stage layer 1 and outputs X1, X1 is input to stage layer 2 and outputs X2, X2 is input to stage layer 3 and outputs X3;
[0095] A4) Stage 4 is set to include one convolutional module, one spatial pyramid pooling module and one cross-stage local layer sequence. X3 is input to stage 4 and output is X4. The spatial pyramid pooling module passes the input sequentially through multiple 5x5 pooling layers to obtain features of a larger scale range.
[0096] A5) Set n for stages 1-4 to be 3, 6, 6, and 3 respectively.
[0097] (3) Set up the CSPNEXtPAFPN feature enhancement network.
[0098] B1) Configure the CSPNEXtFPN feature enhancement network as a bidirectional spatial pyramid network.
[0099] One of them is a top-down network, which includes two convolutional modules, two upsampling layers and two top-down layers. The convolutional modules are used to unify the number of channels, the upsampling layers are used to unify the image resolution, and each top-down layer consists of a cross-stage local layer.
[0100] Another is the bottom-up network, which includes two convolutional modules and two bottom-up layers. The two convolutional modules are used for downsampling with a stride of 2. Each bottom-up layer consists of a cross-stage local layer.
[0101] B2) In the top-down network, the output X4 = (BS,1024,20,20) of the CSPNEXt basic feature acquisition network is changed to Y3 = (BS,512,20,20) through the convolution module, and then upsampled to (BS,512,40,40) and stacked with the output X3 = (BS,512,40,40) of the CSPNEXt basic feature acquisition network to form (BS,1024,40,40), which is used as the input of the top-down layer 2;
[0102] The output of layer 2 from top to bottom changes the number of channels through the convolution module to become Y2 = (BS,256,40,40), and after upsampling, it becomes (BS,256,80,80). This is then stacked with the output X2 = (BS,256,80,80) of the CSPNEXt basic feature acquisition network to form (BS,512,80,80), which serves as the input of layer 1 from top to bottom. Its output is denoted as Y1, and Z1 = Y1.
[0103] B3) In the bottom-up network, Z1 is downsampled by the convolution module and stacked with Y2 to form (BS, 512, 40, 40) as the input of the bottom-up layer 1, and its output is denoted as Z2;
[0104] Z2 is downsampled by the convolution module and stacked with Y3 to form (BS, 1024, 20, 20) as the input from the lower layer to the upper layer 2, and its output is denoted as Z3;
[0105] B4) Finally, Z1, Z2 and Z3 are used as the initial spare frames for anchorless orientation to generate the network input.
[0106] (4) Set up an initial spare frame without anchor frame orientation to generate a network.
[0107] Existing orientation initial spare box generation networks are all designed based on predefined anchor boxes. A large number of predefined anchor boxes need to be set in advance to obtain more positive samples during network training. Furthermore, the size, aspect ratio, and angle of the predefined anchor boxes all need to be manually set to the optimal parameters through a large number of experiments, which is not suitable for practical application.
[0108] The network for generating initial spare bounding boxes without anchor boxes is set as a detection head network, including a convolutional module and three convolutional layers with a kernel size of 1×1 and a stride of 1. The convolutional module includes a convolutional layer with a kernel size of 3×3, a stride of 1, and a padding size of 1, a normalization layer, and an activation function.
[0109] The three-layer feature output of the CSPNeXtPAFPN feature enhancement network shares the same anchorless oriented initial spare box generation network. That is, the three layers of features use the same detection head network, except for the normalization layer in the convolutional module. The final oriented initial spare box prediction results of the three layers of features are as follows:
[0110] The first layer of features, after passing through the anchorless bounding box generation network, yields the results of class convolution, regression convolution, and angular convolution: (BS,1,w1,h1), (BS,4,w1,h1), and (BS,1,w1,h1), respectively. Decoding these convolution results yields the first layer's oriented prediction bounding box P1, where BS represents the number of images processed in the batch.
[0111] The second layer features, after passing through the anchorless oriented initial spare box generation network, yield the results of class convolution, regression convolution, and angular convolution: (BS,1,w2,h2), (BS,4,w2,h2), and (BS,1,w2,h2). These are then decoded to obtain the oriented prediction box P2 for the second layer.
[0112] The third layer features, after passing through the anchorless oriented initial spare box generation network, yield the results of class convolution, regression convolution, and angular convolution: (BS,1,w3,h3), (BS,4,w3,h3), and (BS,1,w3,h3), respectively. Decoding these results yields the oriented predicted bounding box P3 for the third layer.
[0113] Thus, the oriented initial spare frames of the anchorless oriented initial spare frame generation network are obtained.
[0114] (5) Set up the network for generating the final detection frame of the damage:
[0115] The symptom detection bounding box acquisition module is set to include a feature alignment module and a final detection module. The oriented initial spare boxes generated by the network without anchor boxes are subjected to maximum suppression at different layers, and at most the top K boxes are selected. The feature alignment module makes the feature regions of all the oriented initial spare boxes into feature maps ks*ks*256 with uniform size and number of channels. Then, the two fully connected layers in the final detection module are used to perform classification and regression for fine-tuning to obtain the final oriented detection bounding box.
[0116] The third step is to train the rice leaf roller damage detection model: input the preprocessed rice leaf roller damage images into the rice leaf roller damage detection model for training.
[0117] (1) Set up backpropagation of the total loss of the two-stage rice leaf roller damage image detection model. Use 36 epochs for training. The AdamW optimizer is used for training. The basic learning rate is 0.0005 and the basic weight decay factor is 0.05.
[0118] (2) Loss calculation of the network generated by the initial spare frame without anchor frame orientation.
[0119] C1) Forward computation of the network generated by the initial spare frame without anchor frame orientation:
[0120] C11) Input the preprocessed image I = (BS,3,W,H) into the CSPNEXt basic feature acquisition network, and sequentially pass through the backbone layer and 4 stage layers. After stage layers 2-4, the output feature maps X2-4 are of size (BS,256,w1,h1), (BS,512,w2,h2) and (BS,1024,w3,h3).
[0121] C12) Feature map X2-4 is input into the CSPNEXtPAFPN feature enhancement network.
[0122] In the top-down network, X4 = (BS,1024,20,20) changes the number of channels through the convolution module to become Y3 = (BS,512,20,20), and then upsamples it to become (BS,512,40,40), which is stacked with X3 = (BS,512,40,40) to form (BS,1024,40,40), which serves as the input to the second layer from the top down.
[0123] The output of layer 2 from top to bottom changes the number of channels through the convolution module to become Y2 = (BS, 256, 40, 40). After upsampling, it becomes (BS, 256, 80, 80) and is stacked with X2 = (BS, 256, 80, 80) to form (BS, 512, 80, 80), which serves as the input of layer 1 from top to bottom. Its output is denoted as Y1, and Z1 = Y1 is set.
[0124] In the bottom-up network, Z1 is downsampled by the convolution module and stacked with Y2 to form (BS, 512, 40, 40), which serves as the input to the bottom-up layer 1. Its output is denoted as Z2.
[0125] Z2 is downsampled by the convolution module and stacked with Y3 to form (BS, 1024, 20, 20) as the input from the lower layer to the upper layer 2, and its output is denoted as Z3;
[0126] Ultimately, Z1, Z2, and Z3 are used as inputs to generate the network from the initial spare frames for anchorless orientation;
[0127] C13) Z1, Z2, and Z3 are input into the anchorless bounding box orientation initial spare box generation network, which includes one convolutional module with a kernel size of 3×3, a stride of 1, and a padding size of 1, and three parallel convolutional modules with a kernel size of 1×1 and a stride of 1, ultimately yielding a three-layer feature result.
[0128] The features of the first layer are processed by the anchorless orientation initial spare box generation network to obtain (BS,1,w1,h1), (BS,4,w1,h1), and (BS,1,w1,h1), respectively, representing the results obtained by class convolution, regression convolution, and angular convolution. In the result obtained by regression convolution, 4 represents the distance of the anchor point from the left, top, right, and bottom edges of the predicted box. The result obtained by angular convolution is the orientation angle encoding result of the predicted box.
[0129] C2) Anchorless orientation of initial spare boxes generates the allocation of positive and negative samples in the network:
[0130] First, an anchor point (x, y, stride, stride) is set for each point on the 3-layer feature map, for a total of (w1*h1+w2*h2+w3*h3) anchor points;
[0131] The angle result is decoded by the set angle decoder to obtain the orientation angle. Then, the decoded angle result and the regression result are combined and the anchor point is decoded by the set prediction box encoder to obtain the predicted orientation initial spare box. The orientation initial spare boxes of the three layers are set as P1, P2 and P3 respectively.
[0132] The sample allocation uses a dynamic sample allocation criterion, which includes using centroid deviation loss, location regression loss, and sample classification loss. The sum of these three losses is the final loss. Then, a dynamic allocation method is used to determine which positive samples should be assigned to each labeled instance and the number of positive samples.
[0133] C21) First, calculate the anchor points inside all instances and record them as valid anchor points. Anchor points are feature points of the 3-layer feature map.
[0134] C22) Calculate the intersection-union ratio (IUR) of the predicted bounding boxes obtained by decoding all instances and valid anchor point samples. For each instance, take the first m predicted bounding boxes with the largest IUR and calculate the sum of these m largest IURs as the number n of positive samples corresponding to this instance.
[0135] C23) Then, for each instance, select the n anchor point samples with the smallest loss from the loss matrix as positive samples;
[0136] C24) For a given anchor point, if it is assigned to multiple labeled instances at the same time, the instance with the smallest loss is selected as the label of the positive sample, ensuring that an anchor point matches only one labeled instance, but a label has multiple positive samples.
[0137] C3) The loss function of the network for generating the initial spare bounding boxes without anchor boxes consists of three parts: classification loss, angle loss, and regression loss. The classification loss is calculated using the positive and negative samples selected in the previous step (Quality FocalLoss loss), the angle loss is calculated using the positive samples (SmoothL1Loss loss), and the regression loss is calculated using the positive samples (rotation intersection-over-union ratio loss of the predicted bounding box and the corresponding labeled instance). The sum of the three is the loss of the network for generating the initial spare bounding boxes without anchor boxes.
[0138] (3) Calculation of the loss of the network generated by the final detection frame of the damage condition.
[0139] D1) Forward computation of the network that generates the final detection box for the damage:
[0140] Using the tilt feature dimensionality reduction module and the final detection module of the damage detection network, the oriented prediction boxes P1, P2 and P3 of the three feature layers in the anchorless oriented initial spare box generation network are subjected to maximum suppression in different layers. The top K oriented initial spare boxes with the highest classification scores are then input into the damage final detection box generation network. The local features corresponding to the selected oriented initial spare boxes are dimensionality reduced to ks*ks*256. Finally, two fully connected layers are used to perform the final detection.
[0141] D2) Positive and negative sample allocation for the network that generates the final detection bounding box for the damage:
[0142] For the top K initial spare bounding boxes, 512 positive and negative samples were selected, with positive samples accounting for 25% and the rest being negative samples. The selection method was that if the rotation intersection-union ratio of the initial spare bounding box and the labeled instance was greater than 0.5, it was a positive sample, and if it was less than 0.5, it was a negative sample.
[0143] D3) Calculation of the loss of the network generated by the final detection box of the damage:
[0144] The loss function of the network that generates the final detection box for the damage consists of two parts: classification loss and regression loss.
[0145] The classification loss is calculated using the positive and negative samples selected in the previous step.
[0146] CrossEntropyLoss loss and regression loss are calculated using positive samples. SmoothL1Loss loss is calculated, and the sum of the two is the loss of the network that generates the final detection box for the harmful condition.
[0147] The fourth step is the acquisition and preprocessing of the images to be detected: acquiring images of rice leaf roller damage to be detected and performing preprocessing.
[0148] The fifth step is to obtain the image detection results of rice leaf roller damage: input the preprocessed images of rice leaf roller damage to be detected into the trained rice leaf roller damage detection model to obtain the image detection results of rice leaf roller damage.
[0149] like Figure 2a As shown, this is the original image with the damage description. Figure 2b As shown in the figure, this is the detection effect diagram of the present invention. It can be seen that rotation detection can more accurately locate the harmful area. As shown in Table 1, by comparison, it can be found that the present invention obtains the highest recall rate with the same number of initial backup boxes, indicating that the method described in the present invention can obtain higher quality initial backup boxes, thereby achieving the effect of improving accuracy with fewer candidate boxes.
[0150] Table 1 Comparison of detection rates between the present invention and existing algorithms.
[0151]
[0152] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.
Claims
1. A two-stage rice leaf roller damage pattern image detection method based on anchor box-free, characterized in that, The method comprises the following steps: 11) obtaining and preprocessing the Cnaphalocrocis medinalis damage image: obtaining the Cnaphalocrocis medinalis damage original image, and performing scale normalization, random flipping and random rotation preprocessing; 12) constructing a Cnaphalocrocis medinalis damage detection model: constructing a Cnaphalocrocis medinalis damage detection model based on an anchor-free directional initial candidate box generation network; The construction of the Cnaphalocrocis medinalis damage detection model comprises the following steps: 121) setting the Cnaphalocrocis medinalis damage detection model to comprise a CSPNeXt basic feature acquisition network, a CSPNeXt PAFPN feature enhancement network, an anchor-free directional initial candidate box generation network and a damage final detection box generation network; 122) setting the CSPNeXt basic feature acquisition network; 123) setting the CSPNeXt PAFPN feature enhancement network; 124) setting the anchor-free directional initial candidate box generation network; The anchor-free directional initial candidate box generation network is set as a detection head network comprising a convolution module and three convolution kernels with a size of 1x1 and a step size of 1, wherein the convolution module comprises a convolution kernel with a size of 3x3, a step size of 1 and a padding size of 1, a normalization layer and an activation function, The three layers of features output by the CSPNeXt PAFPN feature enhancement network share the anchor-free directional initial candidate box generation network, that is, the three layers of features use the same detection head network, except that the normalization layers in the convolution module are different, and finally the directional initial candidate box prediction results of the three layers of features are obtained: The first layer of features obtained through the anchor-free directional initial candidate box generation network are respectively a class convolution, a regression convolution and an angle convolution result (BS, 1, w1, h1), (BS, 4, w1, h1) and (BS, 1, w1, h1), and the convolution results are decoded to obtain the directional prediction box P1 of the first layer, BS being the number of image batches in batch processing, The second layer of features obtained through the anchor-free directional initial candidate box generation network are respectively a class convolution, a regression convolution and an angle convolution result (BS, 1, w2, h2), (BS, 4, w2, h2) and (BS, 1, w2, h2), which are decoded to obtain the directional prediction box P2 of the second layer, The third layer of features obtained through the anchor-free directional initial candidate box generation network are respectively a class convolution, a regression convolution and an angle convolution result (BS, 1, w3, h3), (BS, 4, w3, h3) and (BS, 1, w3, h3), which are decoded to obtain the directional prediction box P3 of the third layer, Thus, the directional initial candidate box of the anchor-free directional initial candidate box generation network is obtained; 125) setting the damage final detection box generation network: The damage state bounding box setting module includes a feature alignment module and a final detection module. After maximum suppression is performed on the directional initial spare bounding boxes obtained by the anchor-free directional initial spare bounding box generation network at different layers, at most topK bounding boxes are obtained. The feature regions of all the directional initial spare bounding boxes are changed into feature maps ks*ks*256 with uniform size and uniform number of channels by the feature alignment module, and then classification and regression are performed by using two full connections in the final detection module to obtain the final directional detection bounding box. 13) Training of the Cnaphalocrocis medinalis damage state detection model: the preprocessed Cnaphalocrocis medinalis damage image is input into the Cnaphalocrocis medinalis damage state detection model for training. 14) Acquisition and preprocessing of the image to be detected: the Cnaphalocrocis medinalis damage image to be detected is acquired and preprocessed. 15) Obtaining of the Cnaphalocrocis medinalis damage image detection result: the preprocessed Cnaphalocrocis medinalis damage image to be detected is input into the trained Cnaphalocrocis medinalis damage state detection model to obtain the Cnaphalocrocis medinalis damage image detection result.
2. The two-stage rice leaf roller damage pattern image detection method based on anchor box-free according to claim 1, characterized in that, The training of the Cnaphalocrocis medinalis damage state detection model includes the following steps: 21) Reverse propagation of the total loss of the two-stage Cnaphalocrocis medinalis damage state image detection model, 36 epochs are used during training, the AdamW optimizer is used as the training optimizer, the base learning rate is 0.0005, and the base weight decay factor is 0.05; 22) Loss calculation of the anchor-free directional initial spare bounding box generation network; 23) Loss calculation of the damage state final detection bounding box generation network.
3. The two-stage rice leaf roller damage pattern image detection method based on anchor box-free according to claim 1, characterized in that, The CSPNeXt base feature acquisition network includes the following steps: 31) The CSPNeXt base feature acquisition network is set to a 5-layer structure stack, which includes 1 backbone layer and 4 stage layers; 32) The backbone layer is set to be stacked by 3 convolution modules with a convolution kernel size of 3x3 and a padding size of 1, the step size of the first convolution module is 2, and the step size of the last two convolution modules is 1; wherein the convolution module is stacked and combined by a convolution layer, a normalization layer and an activation function layer, and the preprocessed image I=(BS,3,W,H) becomes X after passing through the backbone layer; 33) The stage layers 1-3 are set to be composed of 1 convolution module and 1 cross-stage local layer, and the cross-stage local layer is composed of 3 convolution modules with a convolution kernel size of 1x1, a step size of 1 and a padding size of 0, and n CSPNeXtBlock residual connections; CSPNeXtBlock is composed of 1 convolution module with a convolution kernel size of 3x3, a step size of 1 and a padding size of 1, and 1 depth separable convolution module with a convolution kernel size of 5x5, a step size of 1 and a padding size of 2 in residual connection; X passes through the stage layer 1 to output X1, X1 inputs the stage layer 2 to output X2, and X2 inputs the stage layer 3 to output X3; 34) Set stage layer 4 to include 1 convolution module, a spatial pyramid pooling module and 1 cross-stage local layer series, and the input X3 of stage layer 4 is the output X4, the spatial pyramid pooling module sequentially passes through multiple 5x5 size pooling layers to obtain features of a larger scale range; 35) Set n of stage layers 1-4 to 3, 6, 6 and 3 respectively.
4. The two-stage rice leaf roller damage pattern image detection method based on anchor box-free according to claim 1, characterized in that, The CSPNeXt PAFPN feature enhancement network comprises the following steps: 41) Set the CSPNeXt PAFPN feature enhancement network to be a bidirectional spatial pyramid network, One of which is a top-down network, including two convolution modules, two up-sampling layers and two top-down layers, the convolution modules are used to unify the number of channels, the up-sampling layers are used to unify the picture resolution, and each top-down layer is composed of a cross-stage local layer; The other is a bottom-up network, including two convolution modules and two bottom-up layers, the two convolution modules are used for down-sampling with a step size of 2, and each bottom-up layer is composed of a cross-stage local layer; 42) In the top-down network, the output X4=(BS, 1024, 20, 20) of the CSPNeXt basic feature acquisition network is changed to Y3=(BS, 512, 20, 20) by a convolution module, and then up-sampled to (BS, 512, 40, 40) and stacked with the output X3=(BS, 512, 40, 40) of the CSPNeXt basic feature acquisition network to form (BS, 1024, 40, 40) as the input of the second top-down layer; The output of the second top-down layer is changed to Y2=(BS, 256, 40, 40) by a convolution module, and then up-sampled to (BS, 256, 80, 80) and stacked with the output X2=(BS, 256, 80, 80) of the CSPNeXt basic feature acquisition network to form (BS, 512, 80, 80) as the input of the first top-down layer, and the output is recorded as Y1, denoted as Z1=Y1; 43) In the bottom-up network, Z1 is down-sampled by a convolution module and stacked with Y2 to form (BS, 512, 40, 40) as the input of the first bottom-up layer, and the output is recorded as Z2; Z2 is down-sampled by a convolution module and stacked with Y3 to form (BS, 1024, 20, 20) as the input of the second bottom-up layer, and the output is recorded as Z3; 44) Finally, Z1, Z2 and Z3 are used as the input of the anchor-free directional initial candidate box generation network.
5. The two-stage rice leaf roller damage pattern image detection method based on anchor box-free according to claim 2, characterized in that, The loss calculation of the anchor-free directional initial candidate box generation network comprises the following steps: 51) Forward calculation of the anchor-free directional initial candidate box generation network: 511) Input the preprocessed image I=(BS, 3, W, H) into the CSPNeXt basic feature acquisition network, sequentially pass through the backbone layer and the four stage layers, and output feature maps X2-4 with sizes of (BS, 256, w1, h1), (BS, 512, w2, h2) and (BS, 1024, w3, h3) after the stage layers 2-4; 512) Feature map X2-4 inputs CSPNeXt PAFPN feature enhancement network, From top to bottom network X4=(BS, 1024, 20, 20) changes the number of channels to Y3=(BS, 512, 20, 20) through a convolution module, and then up-sampling to (BS, 512, 40, 40) and X3=(BS, 512, 40, 40) are stacked into (BS, 1024, 40, 40) as the input of the top-down layer 2; The output of the top-down layer 2 changes the number of channels to Y2=(BS, 256, 40, 40) through a convolution module, and then up-sampling to (BS, 256, 80, 80) and X2=(BS, 256, 80, 80) are stacked into (BS, 512, 80, 80) as the input of the top-down layer 1, and its output is recorded as Y1, and Z1=Y1 is set; Z1 in the bottom-up network is down-sampled through a convolution module and stacked with Y2 into (BS, 512, 40, 40) as the input of the bottom-up layer 1, and its output is recorded as Z2; Z2 is down-sampled through a convolution module and stacked with Y3 into (BS, 1024, 20, 20) as the input of the bottom-up layer 2, and its output is recorded as Z3; Finally, Z1, Z2 and Z3 are inputs of the anchor-free directional initial bounding box generation network; 513) Z1, Z2 and Z3 are respectively input into the anchor-free directional initial bounding box generation network, which includes one convolution module with a convolution kernel size of 3x3, a step size of 1 and a padding size of 1, and three convolution kernels with a convolution kernel size of 1x1 and a step size of 1, finally obtaining the results of three layers of features, The first layer of features respectively obtains (BS, 1, w1, h1), (BS, 4, w1, h1) and (BS, 1, w1, h1) through the anchor-free directional initial bounding box generation network, which represents the results obtained by class convolution, regression convolution and angle convolution, wherein 4 in the result obtained by regression convolution represents the distance of the anchor point from the left, top, right and bottom edges of the predicted box, and the result obtained by angle convolution is the directional angle encoding result of the predicted box; 52) Assignment of positive and negative samples in the anchor-free directional initial bounding box generation network: First, set an anchor point (x, y, stride, stride) for each point on the 3-layer feature map, a total of (w1*h1+w2*h2+w3*h3) anchor points; The angle result is decoded by the set angle decoder to obtain the directional angle, and then the decoded angle result and the regression result are combined to obtain the predicted directional initial bounding box through the set prediction box encoder, and the directional initial bounding boxes of the three layers are set as P1, P2 and P3 respectively; The sample assignment adopts the dynamic sample assignment criterion, which includes using the center point deviation loss, the position regression loss, the sample classification loss, and the sum of the three losses as the final loss, and then using the dynamic assignment method to determine which positive samples each labeled instance should be assigned to and the number of positive samples: 521) First, calculate all instances inside the anchor point, marked as valid anchor points, and the anchor points are the feature points of the 3-layer feature map; 522) Calculate the intersection over union value of all instances and the prediction box decoded by the valid anchor point sample, and take the top m prediction boxes with the maximum intersection over union value for each instance, and calculate the sum of the m maximum intersection over union values as the number n of positive samples corresponding to this instance; 523) Then select the n anchor point samples with the smallest loss in the loss matrix as positive samples for each instance; 524) For a certain anchor point, if it is assigned to multiple labeled instances at the same time, select the instance with the smallest loss as the label of the positive sample, so that an anchor point only matches one labeled instance, but a label has multiple positive samples; 53) Set the loss function of the anchor-free oriented initial spare box generation network to include three parts: classification loss, angle loss, and regression loss; The classification loss is calculated using the positive and negative samples selected in the previous step, the Quality FocalLoss loss, the angle loss is calculated using the positive samples, the SmoothL1Loss loss, and the regression loss is calculated using the positive samples to calculate the rotational intersection over union loss of the prediction box and the corresponding labeled instance. The sum of the three is the loss of the anchor-free oriented initial spare box generation network.
6. The two-stage rice leaf roller damage pattern image detection method based on anchor box-free according to claim 2, characterized in that, The loss calculation of the final damage status detection box generation network includes the following steps: 61) Forward calculation of the final damage status detection box generation network: Use the tilt feature dimension reduction module and the final detection module of the damage status detection network to input the top K oriented initial spare boxes with the highest classification scores after maximum suppression of the oriented prediction boxes P1, P2 and P3 of the three feature layers in the anchor-free oriented initial spare box generation network into the final damage status detection box generation network. Reduce the local features corresponding to the selected oriented initial spare boxes to ks*ks*256, and then use 2 fully connected layers to perform the final detection; 62) Positive and negative sample assignment for the final damage status detection box generation network: For the top K oriented initial spare boxes, select 512 positive and negative samples, with positive samples accounting for 25%, and the rest being negative samples. The selection method is that if the rotational intersection over union value of the oriented initial spare box and the labeled instance is greater than 0.5, it is a positive sample, and less than 0.5 is a negative sample; 63) Loss calculation of the final damage status detection box generation network: Set the loss function of the final damage status detection box generation network to include two parts: classification loss and regression loss; The classification loss is calculated using the positive and negative samples selected in the previous step, the CrossEntropyLoss loss, and the regression loss is calculated using the positive samples, the SmoothL1Loss loss, and the sum of the two is the loss of the final damage status detection box generation network.