Target detection model based on deep learning and application thereof

Abstract

The invention discloses a target detection model based on deep learning, and a training method and application thereof. The target detection model is a deep learning neural network model constructed based on a YOLOv8 model, when the target detection model is used for target detection of unhealthy areca plants, effective features can be automatically extracted from leaf images of the areca plants, different stages and types of areca plant yellowing are accurately identified, and the target detection efficiency is improved. The accuracy, efficiency and generalization ability of detection of the etiolated plants are remarkably improved, efficient and accurate etiolation and pest and disease damage detection of the areca-nut plants are achieved, and the method has important significance on improvement of agricultural production efficiency and guarantee of food safety.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning and plant disease detection, in particular to a target detection model based on deep learning, a training method thereof, and the use thereof in identifying non-healthy areca plants. BACKGROUND

[0002] Target detection algorithms based on deep learning, especially the YOLO (You Only Look Once) series of algorithms, have shown great potential in many applications due to their efficient detection speed and high accuracy. For example, YOLOv8 has been used to detect the growth conditions, diseases and pests of crops. However, in some practical application scenarios, its detection accuracy, missed detection rate, generalization ability and other aspects still cannot meet the actual needs and need to be further improved.

[0003] Areca catechu is an important economic crop widely planted in tropical regions, with high economic value. The growth of areca catechu is affected by various environmental and pathological factors, among which yellow leaf disease is one of the important problems affecting the growth of areca catechu. Areca catechu planting usually involves large areas of farmland, and traditional manual inspection and ground detection cannot cover all areas, and the cost is high and the efficiency is low.

[0004] Traditional methods for detecting yellow areca catechu plants usually rely on manual observation and visual judgment of the degree of yellowing. However, this method is not only subjective and easily affected by environmental lighting, observer experience and other factors, but also inefficient and difficult to meet the rapid monitoring needs of large-scale planting areas. Therefore, there is an urgent need for an efficient, accurate and rapid automated detection method to replace traditional manual inspection.

[0005] Currently, there are few related researches using remote sensing methods for detection, and there are still many deficiencies and limitations. Some researches use satellite images combined with machine learning algorithms to identify yellowing areca plants (Guo J, Jin Y, Ye, Huichun Huang, Wenjiang Zhao, Jinling Cui, Bei Liu, Fucheng Deng, Jiajian. Recognition of Areca Leaf Yellow Disease Based on PlanetScope Satellite Imagery [J]. Agronomy, 2022, 12(1). DOI: 10.3390 / agronomy12010014.). However, satellite images have low precision, high cost and weak timeliness. Some researches use unmanned aerial vehicle images combined with machine learning algorithms to identify yellowing areca plants (Lei S, Luo J, Tao X, et al. Remote Sensing Detecting of Yellow Leaf Disease of Arecanut Based on UAV Multisource Sensors [J]. Remote Sensing, 2021, 13(22). DOI: 10.3390 / rs13224562.). Unmanned aerial vehicle images improve the identification accuracy, but rely on manual feature extraction (such as HSV color space segmentation, texture feature analysis, etc.), and require complex preprocessing procedures (such as vegetation index calculation, contour extraction, etc.), resulting in feature engineering taking more than 70% of the entire process and being lagging. SUMMARY

[0006] In view of the problems in the prior art, the application provides a target detection model based on deep learning.

[0007] The target detection model based on deep learning of the application is constructed based on a YOLOv8 model, an SE module is added in the main stem of the YOLOv8 model, the activation function of the compression layer in the SE module is SiLU, and the activation function of the expansion layer is Sigmoid; preferably, the compression ratio of the SE module is 16, preferably 16; the depth coefficient of the main stem of the target detection model is 0.5, the width coefficient is 1.25, and the output channel number of the last layer is 256.

[0008] The application provides a training method of the target detection model, and the training method comprises the following steps:

[0009] (1) obtaining a target image to obtain a data set, the data set comprising an image of a target and annotation information, the annotation information comprising a position and a category of the target labeled by an anchor box;

[0010] (2) inputting at least a part of the data set into the target detection model to train the target detection model.

[0011] In some embodiments, in the training, the hyperparameters of the training comprise an optimizer adopting AdamW.

[0012] In some embodiments, in the training, a two-stage training strategy is adopted: in the first stage, the backbone network is frozen, and the head is trained; in the second stage, a learning rate decay strategy adopts a cosine strategy.

[0013] In some embodiments, in the training, a width-height ratio threshold of the anchor box is 2.5.

[0014] In some embodiments, in the training, a size of the anchor box is [12, 16], [30, 22], or [25, 51].

[0015] In some embodiments, the target image is an image with an inclination angle in the range of 0-10°, preferably an image with an inclination angle in the range of 0-5°.

[0016] The application also provides a target detection method for non-healthy Areca catechu plants, the target detection method comprising:

[0017] (a) obtaining an image of an Areca catechu plant;

[0018] (b) inputting the image into the trained target detection model obtained by the training method;

[0019] (c) performing target detection on the input image by using the trained target detection model.

[0020] In some embodiments, the non-healthy Areca catechu plant is a yellowing Areca catechu plant.

[0021] The application also provides a target detection system, the system comprising:

[0022] an image acquisition unit for acquiring an image, preferably, the image acquisition unit is loaded by a drone;

[0023] an image processing unit for receiving an input image to obtain a data set, the data set comprising an image of a target and annotation information, the annotation information comprising a position and a category of the target labeled by an anchor box;

[0024] A target detection model, wherein the target detection model is a trained target detection model obtained by using the training method.

[0025] A detection head configured to detect the data set and output a detection result.

[0026] The present application also provides a computer device, which comprises at least one processor and a memory storing computer instructions executable on the processor, wherein the instructions are executed by the processor to implement the steps of the training method or the target detection method.

[0027] The present application also provides a computer readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the steps of the method.

[0028] In some embodiments, yellowing plants and damaged plants can be used interchangeably; and non-yellowing plants and healthy plants can be used interchangeably.

[0029] Compared with the original YOLOv8, the target detection model of the present application significantly improves the detection accuracy and significantly reduces the detection missed detection rate. The target detection model of the present application improves the adaptability of the model to targets of different scales, different environments and different growth stages, thereby enhancing the generalization ability of the model.

[0030] Using the model of the present application or using the method of the present application can effectively identify and locate the target and generate an accurate bounding box in the image, which can be efficiently, accurately and quickly applied to automatic detection of non-healthy areca plants, and has a wide application in agricultural and forestry investigation and pest assessment. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is a prediction result of the YOLO-YLD model of the present application after deep learning training, which shows a prediction result graph of non-healthy areca plants.

[0032] Figure 2 is a distribution graph of areca plant images output by the YOLO-YLD model of the present application after deep learning training.

[0033] Figure 3 is a flowchart of a method for identifying yellowing areca plants based on deep learning and unmanned aerial vehicle image. DETAILED DESCRIPTION

[0034] TERMS EXPLANATION:

[0035] SE (Squeeze and Excitation) module is a simple and effective attention mechanism that adjusts the channel weights of feature maps through global average pooling (Squeeze) and fully connected layer (Excitation), thereby enhancing the feature expression ability.

[0036] Reduction ratio is a compression parameter that controls the number of neurons in the fully connected layer, affecting the computational complexity and performance of the model.

[0037] Optimizer is a hyperparameter of model training, which guides the parameters of the loss function to update in the right direction and appropriate size, so that the updated parameters make the loss function value continuously approach the global minimum.

[0038] Adam is an adaptive optimization algorithm that combines momentum and RMSprop (Root meanquare propagation), which maintains an adaptive learning rate for each parameter to accelerate the training process.

[0039] AdamW is an improved algorithm based on Adam, which introduces weight decay, separates weight decay and adaptive learning rate, and provides more stable learning rate adjustment and better generalization ability.

[0040] In this paper, "layer" refers to any combination of electronic digital interface circuits, microcontrollers, firmware (such as drivers), and / or computer programs or other software elements.

[0041] The target detection model of the present application uses a convolutional neural network (CNN) to process and analyze target images, thereby achieving efficient and accurate identification of target images.

[0042] In some embodiments, the target is a non-healthy Areca plant. The non-healthy Areca plant refers to an Areca plant in an unhealthy state caused by disease, pest, mechanical damage, natural withering, or climatic factors, including weakened Areca plants, dying Areca plants, and dead Areca plants. The disease includes anthracnose, bacterial leaf spot, and Areca yellowing disease, and the pest is preferably coconut leaf beetle, coconut webworm, and red brown weevil. Preferably, the non-healthy Areca plant is a yellowing Areca plant.

[0043] In the present application, the yellowing areca plant refers to the plant whose part of leaf color changes to yellow caused by areca yellowing disease, relative to the healthy areca plant, the characteristic of which is that the tree crown appears obvious yellowing phenomenon, the yellow leaf area is greater than 10% of the total leaf area, or the tree crown structure is incomplete, and obvious leaf loss appears. In contrast, the characteristic of the healthy plant is that the tree crown is green as a whole, the yellow leaf area is less than 10% of the total leaf area, and the tree crown structure is complete without obvious leaf loss.

[0044] The collection of the target image of the present application can be carried out by conventional image collection methods, for example, a general digital camera, an industrial camera, etc. can be used to collect the target image by taking pictures. When collecting outdoor target images, a drone equipped with a shooting device can be used. For example, when collecting images of areca plants in a forest, the following specific implementation scheme can be adopted:

[0045] 1. Ground situation confirmation

[0046] Control the drone to fly high (for example, height ≥ 60 m) to determine the overall size of the forest, the distribution of plants, and the situation of obstacles or obstructions.

[0047] 2. Route drawing

[0048] Based on the satellite images in the drone flight system, the boundary of the target area is determined, the flight route of the drone for multi-point shooting is planned, and it is ensured that the shooting range of the route contains the entire target area; according to the height of the areca plants in the target area, the flight height is set to ensure that clear images are captured, and preferably the flight height should be maintained at 30-50 m from the top of the plant to ensure that the single tree crown is clearly visible.

[0049] 3. Image collection

[0050] The planned flight parameters are uploaded to the drone section, and the flight is started, so that the drone collects images according to the planned route and flight parameters.

[0051] Shooting time: It is suggested to be from 9 to 11 am or from 2 to 4 pm to avoid strong shadows.

[0052] Data requirements: The resolution should generally be ≥0.5 cm / pixel; the image format is suggested to be JPG / PNG format; the coverage area of a single image should be adjusted according to the performance of the drone to ensure that the tree crown features are clear.

[0053] The collected images generally need to be preprocessed, and the image preprocessing steps can be as follows:

[0054] 1. Image cropping and image screening

[0055] Crop the large-scale orthographic image to a size suitable for model input (e.g., resolution 640x640); remove images that are unclear, display abnormalities, or do not capture the target area, and retain clear and appropriate images.

[0056] Select images with a tilt angle in the range of 0-10°, preferably images with a tilt angle of 0-5°.

[0057] 2. Image correction

[0058] Crop the image edge portion deformed due to lens distortion and different shooting angles, retain the orthographic image above the tree canopy, and use Pix4D (www.pix4d.com) or Agisoft Metashape (www.agisoft.com) to generate an orthographic image map, piece together the completed images, and obtain the overall orthographic image of the plantation, with each single tree canopy orthographic image being greater than 100*100 pixels.

[0059] 3. Image stitching

[0060] Stitch the completed images together, use Pix4D or Agisoft Metashape to generate an orthographic image map, and obtain the overall orthographic image of the target area.

[0061] 4. Image annotation

[0062] Use LabelImg (https: / / github.com / HumanSignal / labelImg) or CVAT for annotation, and the annotation box should tightly enclose the entire tree canopy; for overlapping tree canopies, make the annotation box distinguish individual plants as much as possible.

[0063] For images of betel nut plants, annotate whether they have yellowing based on the tree canopy condition, with the specific determination criteria being:

[0064] Category H (H classification) plant: indicates that the tree canopy has no yellowing, indicating that it is a healthy plant, the overall tree canopy is green, the total area of yellow leaves is less than 10% of the total area of all leaves, and the tree canopy structure is complete with no obvious leaf loss.

[0065] Category S (S classification) plant: indicates that the tree canopy has yellowing, indicating that it is a damaged plant, the tree canopy has obvious yellowing, the total area of yellow leaves is greater than 10% of the total area of all leaves, or the tree canopy structure is incomplete with obvious leaf loss.

[0066] Annotation quality control: plant pathologists participate in annotation or verification to ensure consistency of annotation.

[0067] Annotation quantity control: the number of each type of tree canopy annotation is not less than 100.

[0068] The images used for model training can be further processed, including image enhancement. In the case of insufficient suitable images, data enhancement techniques can be used to generate more diverse training data by performing geometric transformations or rotations, cropping, flipping, etc. on the images within the annotation box to enhance the generalization ability of the deep learning neural network to different scenes, angles, and pathological characteristics.

[0069] Specifically, geometric transformations include rotation, flipping, zooming and cropping, and color and illumination disturbance. Rotation can be random rotation (0°-360°), as tree crowns are usually not directionally limited, so large rotations are suitable for generating diverse training data. If the shooting angle is fixed (such as a drone view), the rotation angle can be limited (such as ±30°).

[0070] Flipping includes horizontal / vertical flipping, which is freely flipped due to the high symmetry of tree crowns.

[0071] Zooming and cropping include random zooming (e.g., 0.8-1.2 times) followed by cropping, which simulates the effect of shooting at different distances. Note that the cropped image should still contain the complete tree crown to avoid cutting off the main body of the tree crown.

[0072] Color and illumination disturbance refers to adjusting brightness / contrast to simulate different lighting conditions at different times (dawn / dusk / noon), for example: brightness ±20%, contrast ±15%.

[0073] Example 1 Construction and training of target detection model

[0074] Based on the YoloV8 model (Ultralytics Company, http: / / github.com / ultralytics / ultralytics), a target detection model of the present application is constructed for the detection of non-healthy areca plants. The method steps for constructing the model are:

[0075] 1. Constructing the data set

[0076] The images collected and preprocessed using the above method are used to construct the data set for the model, with 1 data per image. 70% of all data is used to construct the training set, 15% is used to construct the validation set, and 15% is used to construct the test set. The geographical distribution of data in different data sets should be similar.

[0077] 2. Model construction

[0078] Based on the YOLOv8 model, an improved new model named YOLO-YLD model is obtained. Compared with the YOLOv8 model, the improvement measures of the YOLO-YLD model are as follows:

[0079] Channel attention mechanism: add SE module at the end of the backbone, and set the compression ratio of the SE module to 16 to enhance the yellowing feature extraction capability.

[0080] Adaptive anchor box calculation: based on the statistical data of 5000+ areca trees, after comparative analysis, the optimized anchor box size is determined as [12, 16], [30, 22], [25, 51], which can optimize the calculation efficiency, reduce the background false detection rate, improve the detection accuracy, and more effectively identify areca plants.

[0081] Two-stage training strategy: first stage: freeze the backbone network, train the head; second stage: fine-tune the whole network. The learning rate decay adopts the cosine strategy, which means that the learning rate changes according to the cosine curve during training, and the smooth adjustment of the parameter update step can significantly improve the model convergence stability and final accuracy.

[0082] The parameters of the YOLO-YLD model architecture are as follows:

[0083] Table 1. Comparison of backbone network parameters

[0084]

[0085] Among them, setting the depth coefficient to 0.5 can optimize the identification of small targets; setting the output channel number of the last layer to 256 can better adapt to the SE module. The setting of the backbone network parameters significantly enhances the model's detection ability for small size yellowing disease targets, while maintaining high accuracy and significantly reducing the model complexity.

[0086] The SE module is constructed according to HU J, LI S, ALBANIE S, et al. Squeeze-and-Excitation Networks [J / OL]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023. DOI:10.1109 / TPAMI.2019.2913372, with specific modifications of the parameters as shown in Table 2.

[0087] Table 2. SE module parameters

[0088]

[0089] 3. Training of the model

[0090] The model running environment is as follows:

[0091] Hardware configuration:

[0092] GPU: NVIDIA RTX 3060 or higher (24 GB video memory)

[0093] RAM: ≥ 32 GB

[0094] Software configuration:

[0095] Python 3.8+ (USA, https: / / www.python.org / )

[0096] PyTorch 1.12+ (USA, https: / / pytorch.org / )

[0097] Ultralytics YOLOv8 (USA, https: / / github.com / ultralytics / ultralytics)

[0098] CUDA 11.3+ (USA, https: / / developer.nvidia.com / cuda-toolkit)

[0099] The hyperparameters for model training are set as follows:

[0100] Table 3. Comparison of basic training configurations

[0101]

[0102] Among them, the optimizer (optimizer) adopts AdamW, which can better handle unbalanced data distribution. As mentioned above, the basic training configuration can better handle the sample imbalance problem existing in the data of areca yellowing disease, effectively improving the learning ability of the model to minority class samples.

[0103] Table 4. Two-stage training parameter settings added

[0104]

[0105] Table 5. Comparison of data augmentation parameters

[0106]

[0107] Table 6. Comparison of anchor parameters

[0108]

[0109] Table 7. Comparison of inference parameters

[0110]

[0111] In addition, in order to optimize the output effect, a visual parameter setting can also be added.

[0112] Table 8. Added visual parameter settings

[0113]

[0114] The image and label information are imported into the target detection model for training, and the trained YOLO-YLD model is obtained.

[0115] Example 2: Using the trained YOLO-YLD model to identify yellowing areca plants

[0116] 1. Model recognition process

[0117] The preprocessed image is imported into the YOLO-YLD model constructed in Example 1, which will automatically identify the yellowing plants in each image and output a yellowing plant distribution map (as shown in Figure 2 ).

[0118] The operation process is as follows:

[0119] (1) Open the picture to be corrected;

[0120] (2) Modify the error bbox (drag adjustment or delete and redraw);

[0121] (3) Switch the correct label (H / S);

[0122] (4) Save as PASCAL VOC format XML.

[0123] The output result is shown in Figure 2 . The box in the output single image is the boundary of the labeled tree crown, and the boxes of different colors label whether it is damaged or not. The number on the box represents the confidence of the damaged plant.

[0124] 2. Evaluation of target detection model

[0125] The confusion matrix is drawn using Python, and the trained YOLO-YLD model is visually verified, with the prediction results superimposed (as shown in Figure 1The same test set was used to predict using a traditional vector machine (SVM) model and a native YOLOv8 model, respectively. The results of the test evaluation are shown in Table 9. The accuracy of the trained YOLO-YLD model is 83%; the precision of H classification is 82.9%, the recall is 76.3%, and the F1 score is 79.5%; the precision of S classification is 90.7%, the recall is 67.7%, and the F1 score is 77.5%, indicating that the trained YOLO-YLD model has excellent overall recognition performance, good classification stability, and appropriate misjudgment rate. Compared with the parameters of the traditional vector machine (SVM) model and the native YOLOv8 model, the parameters of the trained YOLO-YLD model are significantly improved.

[0126] Compared with the traditional SVM model, the F1 score of the trained YOLO-YLD model is improved by 27.7%, fully proving the superiority of the deep learning algorithm in the recognition task of areca yellowing disease; compared with the native YOLOv8 model, the F1 score of the trained YOLO-YLD model is improved by 9.2%, verifying the effectiveness of the model architecture optimization and training strategy improvement; based on maintaining the high-efficiency detection characteristics of YOLOv8, the recognition accuracy is significantly improved through targeted optimization, providing a more reliable solution for the intelligent diagnosis of areca yellowing disease.

[0127] Table 9

[0128]

[0129] The YOLO-YLD model trained through deep learning can directly perform multi-class classification and recognition on the leaf images of areca plants, recognize yellowing areca plants, and does not need to extract the crown contour, obtain the reflectivity, and calculate the vegetation index and other complex operations (more efficient compared with machine learning). The YOLO-YLD model has high expression ability and strong robustness when processing areca yellowing symptoms.

[0130] High precision and recall: This indicates that the YOLO-YLD model does not simply memorize some obvious features, but understands the deep and distinguishing patterns of "healthy" and "yellowing", indicating that the model has high expression ability.

[0131] Good robustness of performance: The F1 scores (a comprehensive indicator of precision and recall) of the H and S classes of the model are close in value, indicating that the model does not have a serious performance bias. This means that the model is not simply biased towards predicting samples as a certain class (for example, to improve accuracy by predicting all as healthy), but maintains robust and balanced performance in the identification of both classes.

[0132] Uniform distribution of confusion matrix: Although there are errors, the errors are within an acceptable range, and there is no extreme imbalance (for example, one side has much more errors than the other side). This predictable and relatively uniform error distribution indicates that the model decision is stable and consistent, rather than random or fragile. It shows that the model can resist the interference of some irrelevant noise in the image and focus on the relevant features.

[0133] Although the embodiments of the present application have been shown and described above, it is to be understood that the above-described embodiments are exemplary, and are not to be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-described embodiments within the scope of the present application.

Claims

1. A deep learning-based object detection model, characterized in that, The target detection model is built based on the YOLOv8 model, wherein an SE module is added to the backbone of the YOLOv8 model. The activation function of the compressed layer in the SE module is SiLU, and the activation function of the extended layer is Sigmoid. Preferably, the compression ratio of the SE module is 8 or 16, preferably 16. The YOLOv8 model has a depth factor of 0.5, a width factor of 1.25, and 256 output channels in the last layer.

2. The training method for the target detection model according to claim 1, the training method comprising the following steps: (1) Obtain the target image and obtain the dataset, which includes the target image and annotation information, and the annotation information includes the target's location and category marked with anchor boxes; (2) Input at least a portion of the dataset into the target detection model of claim 1 for training to obtain a trained target detection model.

3. The training method of claim 2, wherein the hyperparameters of the training include: The optimizer uses AdamW; In the training, a two-stage training strategy is adopted: in the first stage, the backbone network is frozen and the head is trained; In the second phase, the learning rate decay strategy adopts the cosine strategy.

4. The training method according to claim 2, characterized in that, The aspect ratio threshold of the anchor frame is 2.5; Preferably, the dimensions of the anchor frame are [12,16], [30,22], [25,51].

5. The training method according to claim 2, wherein the target image is an image with a tilt angle in the range of 0 to 10°, preferably an image with a tilt angle in the range of 0 to 5°.

6. A target detection method for unhealthy areca nut plants, wherein the target detection method includes: (a) Obtain an image of the areca plant; (b) Input the image into the trained target detection model obtained by the training method of any one of claims 2-5; (c) Perform target detection on the input image using the trained target detection model.

7. The target detection method for unhealthy areca plants as described in claim 6, wherein the unhealthy areca plants are yellowed areca plants.

8. A target detection system, the system comprising: An image receiving unit is used to receive the target image; An image processing unit is used to preprocess the received target image to obtain a dataset, the dataset including the target image and annotation information, the annotation information including the target's location and category marked with anchor boxes; An object detection model, wherein the object detection model is a trained object detection model obtained by the training method according to any one of claims 2-5; A detection head, which is used to detect the dataset and output the detection results.

9. A computer device comprising at least one processor and a memory storing computer instructions executable on the processor, the instructions, when executed by the processor, implementing the steps of the method according to any one of claims 2-7.

10. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 2-7.

Images