Rice leaf folder imago image detection and counting method and system oriented to complex environment

Abstract

The invention discloses a rice leaf folder imago image detection and counting method and system for a complex environment, and belongs to the technical field of image processing and computer vision. Comprising the following steps: collecting rice leaf folder imago images in an actual rice field scene or an insect trap, constructing an image training set, and rejecting seriously fuzzy or seriously shielded samples through an image cleaning strategy to improve data quality and model effectiveness; an efficient attention backbone module and a cross-scale fusion network are introduced on the basis of a YOLOv8 model structure, and the detection performance of the model on small insect bodies and multi-scale targets is enhanced; a freezing-unfreezing two-stage fine tuning training method is adopted for a newly added image, and the stability and recognition robustness of the model in a complex environment are improved; experimental results show that the method has good detection precision, applicability and real-time performance in a natural field environment, and is suitable for an insect situation monitoring terminal, an agricultural image recognition platform and a pest and disease damage intelligent diagnosis scene.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing and computer vision, and specifically relates to a rice leaf roller adult image detection and counting method and system for complex environments, which combines image acquisition, image preprocessing, deep learning target detection algorithm and front-end visual display system, and is used for improving the precision and automation level of agricultural pest detection. BACKGROUND

[0002] In the modern agricultural production process, as a major pest during the growth period of rice, the rice leaf roller has a high occurrence frequency, fast spreading speed and strong harmfulness, and has become one of the key factors affecting the yield and quality of rice. The traditional pest monitoring method mainly relies on light trapping and manual chasing and experience judgment, which is not only low in efficiency and high in work intensity, but also subjective and limited in recognition accuracy. In order to solve this problem, in recent years, a deep learning-based target detection algorithm has been gradually introduced, which uses image acquisition equipment and a deep learning model to automatically detect and identify pests.

[0003] However, the existing pest detection system still faces many challenges. On the one hand, there are problems such as uneven illumination, target occlusion, motion blur, and small target proportion in the actual environment of the rice field, which leads to a significant decrease in the recognition accuracy of the detection algorithm in the implementation and deployment; on the other hand, some systems rely on fixed devices for shooting, which cannot actively stimulate the activity of pests, resulting in insufficient sample collection and low image quality, limiting the detection ability of the model. In addition, current researches only focus on model design and ignore the linkage optimization with acquisition hardware and visual display system, making it difficult to build a complete closed-loop practical pest monitoring platform.

[0004] Therefore, there is an urgent need for an automatic pest detection method with field deployment capability and high performance, which optimizes the whole process from image acquisition, target detection to result display, realizes high-precision intelligent identification and statistical analysis of rice leaf roller adults. SUMMARY

[0005] The purpose of the present application is to overcome the following technical defects of the existing YOLO series target detection model in the application of rice field pest monitoring: first, the detection accuracy and recall rate of rice leaf roller adults under small sample conditions are insufficient; second, the recognition robustness is poor in motion blur, illumination change, target stacking and occlusion interference scenes; third, the network structure redundancy leads to the inability of edge devices to meet the real-time monitoring requirements.

[0006] To achieve the above purpose, the technical scheme adopted by the present application is: a rice leaf roller adult image detection and counting method for complex environments, comprising:

[0007] Step one, image collection and basic data enhancement. Collect rice leaf roller adult images, unify the format and normalize the size of the collected samples; then use general enhancement methods such as random rotation, scaling, cropping, noise disturbance, and color jitter to expand the dataset, so that limited small samples can have multiple perspectives, multiple lighting, and multiple background forms. Through the above enhancement process, a diversified training set that meets the pre-training needs of the improved network can be constructed, providing a data foundation for the accuracy and robustness of the overall detection model.

[0008] Step two, improve the YOLOv8 network structure. Based on the original architecture of YOLOv8, the backbone network is replaced with an EfficientViT multi-scale linear attention module to achieve lightweight modeling of global context; the CCFM lightweight cross-scale feature fusion module is introduced in the neck network to compress and cascade the multi-scale features P2, P3, and P4 output by the backbone in turn to strengthen small target response and multi-resolution information interaction; finally, the fused features are input into the decoupled detection head to significantly improve the detection accuracy and recall rate of rice leaf roller adults, while reducing the model parameter size and computational complexity.

[0009] Step three, training of the improved YOLOv8 model. After completing sample enhancement and network structure improvement, load the improved YOLO model with pre-trained weights, set training rounds, learning rate, and weight quantization hyperparameters, and use transfer learning to perform centralized iterative training; after training, automatically generate weight files.

[0010] Step four, collect new images of rice leaf roller adults in the field. The field image collection includes two scenarios: one is the natural scene of the rice field, a camera is installed on the field path or above the plants to record the natural activity images of rice leaf roller adults; this scenario covers the field environment with motion blur, uneven lighting, and complex background, which can truly reflect the posture and outline of the adult in the natural state. The second is the insect trap scene, which captures images of captured rice leaf roller adults, with multiple target occlusions. Through the above scene collection, raw images with motion blur, uneven lighting, and small target occlusion can be obtained, providing a data foundation for subsequent sample cleaning, advanced data enhancement, and fine-tuning training.

[0011] Step five, image cleaning and quality selection. For the raw images collected in the field, a combination of hierarchical rejection, sliding cropping, and manual verification is used to improve the sample labeling quality and alleviate the training noise caused by dense targets.

[0012] The data cleaning uses a rejection strategy based on target density threshold to directly reject raw image samples with extremely dense insect targets and severe blur, ensuring that the subsequent samples have good annotatability and training effectiveness.

[0013] The sliding cropping operation adopts fixed window size for overlapping sliding cropping based on reserved image size, discards the empty window region, and thus improves the target proportion of small-scale insect bodies in the image.

[0014] The artificial verification step implements a picture-by-picture review on the sub-pictures generated by automatic cropping, supplements the missing insect body targets in a man-machine cooperative manner, and removes images that still have serious uneven illumination and serious occlusion.

[0015] Through the above image cleaning and quality screening process, the background noise and image redundancy are effectively reduced, the annotation accuracy of the insect body target is improved, and the training effectiveness is improved, further improving the generalization ability of the model.

[0016] Step six, advanced data enhancement. This step uses a combination of occlusion enhancement, Gaussian blur, and random noise injection to improve the model's robustness to complex field lighting, occlusion, and imaging noise. Further, mirror flipping and random color domain transformation are introduced to expand the lighting and color distribution of the samples. The specific enhancement strategies are as follows:

[0017] The occlusion enhancement uses a random rectangular occlusion method. The input image I is formed under the occlusion mask M occ , thereby simulating leaf occlusion and multi-insect body stacking scenarios.

[0018] The Gaussian blur operation generates σ by convolving the Gaussian kernel G with the image I, simulating lens shake and depth of field blur.

[0019] The random noise injection adds zero-mean Gaussian noise N(0,σ 2 ) to the pixel intensity, where σ is set to 3-8 (on a 0-255 pixel grayscale scale) to simulate low-light imaging electrical noise.

[0020] The mirror flipping and color domain random transformation expand the color space distribution through random horizontal flipping, hue shifting, and saturation scaling operations, and further improve the model's adaptability to lighting changes.

[0021] Through the above advanced data enhancement, the samples are fully expanded in the dimensions of occlusion, blur, noise, and color distribution, providing diverse data input for subsequent freeze-thaw fine-tuning training.

[0022] Step seven, freeze-thaw two-stage model fine-tuning. This step takes the base weight obtained in step three as the starting point, and performs freeze-thaw fine-tuning in sequence to improve the small target recall rate in complex field scenarios.

[0023] The freezing phase employs a method of freezing the first 10 layers of the backbone network, updating only the neck cross-scale feature fusion module and the detection head. The number of training epochs is set to 20, the input size is 640×640 pixels, the batch size is 4, and the initial learning rate is 1×10. -3 A cosine decay strategy is adopted, with the learning rate starting from 1×10⁻⁶. -3 It gradually decreased to 0.1×10 -3 The optimizer uses SGD with a weight decay of 5×10⁻⁶. -4 Enable automatic mixed precision to accelerate training and reduce GPU memory usage.

[0024] During the unfreezing phase, all network layers are unfrozen, and training continues for 30 epochs. The input size is adjusted to 512×512 pixels, the batch size remains at 4, and the initial learning rate is reduced to 1×10⁻⁶. -4 Similarly, cosine decay is used to reduce the final value to 0.1 × 10⁻⁶. -4 The remaining optimizer and weight decay parameters are the same as those in the freeze phase.

[0025] Step 8: Model Deployment and System Implementation. This step aims to deploy the insect detection and counting model on a field edge computing platform to achieve automatic detection, counting, and real-time data transmission of adult rice leaf rollers.

[0026] On the server side, the insect detection model is trained using the PyTorch framework and exported as an ONNX format model file. Based on this ONNX model, the TensorRT toolchain is used to compile it into a .trt inference engine to adapt it for running on the Jetson platform.

[0027] During the deployment phase, the .trt format model and inference script are simultaneously deployed to the Jetson device to achieve real-time processing and inference of image data, including image input, preprocessing, and inference calculations. After inference is completed, the detection results and insect count results are uploaded to the host computer platform via an interface for unified analysis.

[0028] This deployment strategy combines lightweight model structure design with TensorRT compilation optimization to ensure efficient model operation on edge devices, making it suitable for practical application scenarios with limited computing power, such as in the field. Attached Figure Description

[0029] Fig. 1 A method for image detection and counting of adult rice leaf rollers in complex environments;

[0030] Fig. 2 An improved YOLOv8 network block diagram is provided in this embodiment of the invention;

[0031] Fig. 3 A schematic diagram of a freeze-thaw training strategy provided in an embodiment of the present invention;

[0032] Fig. 4 The improved YOLOv8 network model detection result graph provided by the embodiment of the present application;

[0033] Fig. 5 The freeze-thaw fine-tuning detection result graph provided by the embodiment of the present application. DETAILED DESCRIPTION

[0034] The present application will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present application, and cannot be used to limit the protection scope of the present application.

[0035] As Figs. 1-5 shown, the rice leaf roller adult image detection and counting method for complex environment proposed by the present application, the embodiment sequentially executes steps S101-S108:

[0036] S101, image acquisition and basic data enhancement details; clear samples with resolution not less than 480x480 are selected and uniformly converted to JPEG format. Then the samples are expanded by using a basic data enhancement strategy, which at least includes: random rotation (±90°), random scaling and cutting splicing (Mosaic), Gaussian noise injection, light Gaussian blur, color saturation and hue jitter, random erasure and size padding. The integrity of the insect body is maintained during the enhancement process, and the long side of the image is uniformly scaled and padded to 640x640 pixels to ensure consistent input size.

[0037] S102, improve the YOLOv8 network structure; the detection model realizes efficient feature extraction and real-time inference of the small target of the rice leaf roller adult through the lightweight of the backbone and the cross-scale fusion of the neck.

[0038] The backbone network adopts the EfficientViT multi-scale linear attention module, which is composed of a multi-scale linear attention unit and a deep separable convolution feedforward network in series: the input features are linearly mapped to obtain query Q, key K, and value V, and then 3x3 deep separable convolution is used to aggregate local information to form multi-scale markers; then the linear attention mechanism of ReLU activation is used to realize global context modeling, and the output is Att(Q,K,V)=ReLU(Q)(ReLU(K) Τ V), wherein ReLU(.) represents linear activation. Compared with Softmax, the structure reduces the computational complexity from O(N 2 ) to O(N) under the sequence length N=HxW, and combined with the weight sharing characteristics of deep separable convolution, the parameter amount and floating point operation amount of the backbone network are significantly reduced while maintaining the detection accuracy, which is suitable for real-time deployment on edge terminals.

[0039] The neck network adopts a CCFM light-weight cross-scale feature fusion module. The module first performs 1x1 convolution channel compression on the multi-scale feature maps P2, P3 and P4 output by the backbone network, and serially aggregates in a cascaded connection manner. During the aggregation process, multi-resolution context information is extracted by combining serial spatial pyramid pooling, and the fusion result is output by using residual accumulation Through the above processing, the redundant calculation amount can be suppressed while maintaining the small target rice leaf roller insect body texture details, thereby significantly enhancing the target response and improving the positioning accuracy.

[0040] S103, improving the YOLOv8 model training; after completing the sample expansion and network improvement, the improved YOLOv8 model is loaded with weights and iteratively trained using transfer learning.

[0041] First step, the rice leaf roller adult original image is expanded and unified to a data set of 640x640 pixels through rotation, cropping and splicing, noise, blurring and color transformation methods, and is divided into a training set and a validation set in a ratio of 7:1, ensuring feature diversity and annotation consistency under small sample conditions.

[0042] Second step, on the server platform (12vCPU Intel Xeon 8352V, 90GB RAM, RTX 4090 GPU, Ubuntu 22.04), the improved YOLOv8 model and pre-training parameters are loaded into the PyTorch framework, the training round is set to 200 rounds, the initial learning rate is set to 1x10 -4 and other hyperparameters, and automatic mixed precision is enabled to reduce memory usage.

[0043] During training, the classification branch uses binary cross-entropy loss (BCE), the regression branch uses distribution focal loss (DFL), and Task-Aligned Assigner is used for dynamic positive and negative sample distribution to improve the learning efficiency of difficult samples. After all iterations are completed, a.pt format weight file is generated. It is verified that the obtained basic model improves the mAP from 98.5% to 99.2%, the precision rate from 88.5% to 92.1%, and the recall rate from 65.3% to 68.2% under the premise of maintaining real-time, which is significantly better than the existing YOLOv8 baseline model, and meets the accuracy and speed requirements of real-time identification of rice leaf roller adult small targets.

[0044] S104, add field collection of rice leaf roller adult samples; during implementation, a high-definition camera is used to track and shoot the adult insects freely moving in the natural environment of the rice field, and to obtain rice leaf roller adult images in complex scenes. The obtained images and static samples in the trap together constitute a diversified original data set, providing a real and reliable data basis for subsequent sample cleaning, advanced data enhancement and model fine-tuning training.

[0045] S105, image cleaning and quality screening; in this embodiment, the original static and dynamic images obtained in step S104 are pre-processed.

[0046] In this embodiment, the original images obtained in step S104 are first manually preliminarily removed. The images with obvious motion blur, too dark or too bright light samples are directly manually removed; the remaining images enter the next step of sliding cropping and manual checking process.

[0047] Further, the remaining images are overlapped and sliding cropped according to a fixed window of 640x640 pixels, the cropping step is set to half of the window size, and the cropped area without detecting any insect is discarded.

[0048] Further, the above cropped sub-images are manually audited one by one, supplemented with a labeling tool to supplement the missing label frame, and the images with obvious blur, severe occlusion or too dark light are removed again, and the audited images and labeling results are exported in a unified format. Through this process, the background and target overlap noise can be significantly reduced, the insect boundary and posture labeling accuracy can be improved, and high-quality data samples are provided for subsequent improved model fine-tuning training.

[0049] S106, advanced data augmentation; in this embodiment, the following five operations are applied to the data cleaned samples in step S105 to construct diversified and robust training samples.

[0050] One, for each input image, based on the labeled insect target region, a number of opaque rectangular masks are randomly generated around it, the mask width and height are floating in the range of 20% to 50% of the target frame size, so as to simulate different angles and thicknesses of leaf occlusion or multi-insect stacking;

[0051] Two, Gaussian blur processing is performed on the occluded image, the Gaussian convolution kernel radius is set to 3-5 pixels, and σ is set to 1.0-2.0, to reproduce the local blur caused by camera micro-shake or depth of field change;

[0052] Three, zero-mean, Gaussian distributed random noise is superimposed on the pixel intensity with a standard deviation of 3-8 (gray value range 0-255) to simulate sensor electrical noise under low illumination or high gain conditions;

[0053] Four, the processing result is randomly horizontally mirrored to increase the scene symmetry and view angle diversity;

[0054] Five, the image is converted to HSV space, the hue (±10°) and saturation (0.8-1.2 times) are randomly adjusted, and it is converted back to RGB, expanding the light and color distribution of the samples.

[0055] The above operations can be combined or repeated to generate multiple variants. All enhanced images are included together with the original cleaned samples in subsequent freeze-thaw fine-tuning training to significantly improve the model's adaptability to complex occlusion in the field, changes in illumination, and imaging noise.

[0056] S107. Two-stage freeze-thaw fine-tuning training: In this embodiment, the model weights from S103 are first loaded, and model = YOLO(·) is called, followed by the freeze layer warm-up stage. In this stage, the parameters of the first 10 layers of the backbone network are frozen, and only the cross-scale feature fusion module of the neck and the detection head are trained. The input image size is set to 640×640 pixels, the batch size is 4, the optimizer is stochastic gradient descent (SGD), the initial learning rate is set to 1×10⁻³, and the cosine learning rate is decayed to a final value of 0.1×10⁻³. -3 Weight decay 5×10 -4 Automatic Mixed Precision (AMP) was enabled, and the system was trained for 20 rounds to enable the detection head to quickly adapt to the characteristics of the rice paddy scene without compromising the pre-trained generalization ability of the backbone network.

[0057] After the frozen training phase is complete, the full unfreeze and fine-tuning phase begins. All layer parameters in the network are unfrozen, and fine-tuning continues. At this point, the input image size is reduced to 512×512 pixels, the batch size remains at 4, and the initial learning rate is reduced to 1×10⁻⁶. -4 Similarly, cosine decay is used to reduce the final value to 0.1 × 10⁻⁶. -4 SGD optimizer, weight decay 5×10 -4 The training iterations were extended to 30 epochs to fully learn the characteristics of small targets, occlusion, and lighting changes in complex field scenarios, further improving the model's recall and overall mAP performance. After training, the generated optimal weight file becomes the final fine-tuned model, which can be used for subsequent deployment in online detection systems.

[0058] S108. Model Deployment and System Implementation. In this embodiment, the fine-tuned detection model is first exported as an ONNX format file on the server side via torch.onnx.export; then, the ONNX model is copied to the edge computing platform, and the TensorRT toolchain is used to execute commands on the target device to generate a .trt inference engine file. Half-precision acceleration and layer fusion optimization are enabled for the backbone network and detection head to minimize inference latency and memory usage.

[0059] In the deployment stage, the trt file and the matching inference script are deployed to the working directory of the Jetson device. When the script is started, the RTSP stream is accessed through OpenCV. After obtaining the original candidate box and confidence through forward inference, redundant boxes are filtered out using non-maximum suppression, and the number of insect targets per frame is counted. The inference results are uploaded to the cloud in real time, and the script records the timestamp, device number, and target number of each inference in the local SQLite database to support offline caching and data backtracking when the network is unstable.

[0060] Through the above deployment strategy, combined with the lightweight design of the model and the TensorRT compilation optimization, real-time insect detection and counting are realized in the field environment with limited computing power, and the results are real-time returned to the monitoring platform, providing reliable technical support for accurate agricultural monitoring and prevention decision-making.

[0061] The above is only the preferred embodiment of the present application, it should be noted that for those skilled in the art, without departing from the technical principles of the present application, a number of improvements and modifications can be made, these improvements and modifications should also be considered as the protection scope of the present application.

Claims

1. A method for image detection and counting of adult rice leaf rollers in complex environments, characterized in that, Includes the following steps: Step 1: Based on the image samples of adult rice leaf rollers, perform basic data enhancement and expansion on the samples. The basic data enhancement includes rotation, scaling, cropping, and color jitter. Step 2: Integrate the EfficientViT high-efficiency multi-scale linear attention module into the YOLOv8 backbone network and replace it with the CCFM lightweight cross-scale feature fusion module in the neck network; Step 3: Use the expanded samples from Step 1 to train the improved network from Step 2 to obtain the weights of the improved YOLOv8 model. Step 4: Collect images of adult rice leaf rollers in actual field environments or in insect traps; Step 5: Clean the images acquired in the complex environment in Step 4, removing blurry, severely occluded, and duplicate samples; Step six, perform advanced data augmentation on the samples cleaned in step five, including occlusion enhancement, Gaussian blurring and random noise injection; Step 7: Based on the improved model weights obtained in Step 3, the samples obtained in Step 6 are trained using a two-stage freeze-thaw fine-tuning strategy. In the first stage, the first 10 layers of the backbone network are frozen, and only the neck fusion module and the detection head are trained. In the second stage, the entire network is unfrozen and the parameters of the entire network are fine-tuned with a small learning rate. Step 8: Deploy the fine-tuned detection model formed in Step 7 on the edge terminal, run the rice leaf roller adult detection model on the input image in the complex field environment, and output the detection results.

2. The method according to claim 1, characterized in that, In the initial training of step two, the classification loss is binary cross-entropy, and the regression loss is distribution focus loss.

3. The method according to claim 1, characterized in that, The basic data enhancement in step one also includes Mosaic splicing and random color gamut dithering.

4. The method according to claim 1, characterized in that, The image cleaning in step five includes: removing samples whose target overlap is higher than a preset threshold and whose labeling is difficult due to overlap.

5. The method according to claim 1, characterized in that, The advanced data augmentation in step six also includes mirror flipping and color gamut transformation.

6. The method according to claim 1, characterized in that, The small learning rate in step seven is 1×10⁻⁶. -5 .

7. The method according to claim 1, characterized in that, The occlusion enhancement in step six employs a random rectangular occlusion method, where the input image I is within the occlusion mask M. occ Formed under the influence of action.

8. The method according to claim 1, characterized in that, The Gaussian blurring operation in step six involves convolving a Gaussian kernel G with radius r and standard deviation σ on image I. σ .

9. The method according to claim 1, characterized in that, The random noise injection in step six involves zero-mean Gaussian noise N(0,σ) 2 Add to pixel intensity, σ is set to 3 to 8.

10. A detection system for adult rice leaf rollers under complex environments, characterized in that, The device includes an image acquisition unit, an edge computing device, a target detection algorithm module, and a display interface, wherein the target detection algorithm module is configured to implement the method described in any one of claims 1 to 9.

Images