Method and device for quickly grading severity of strawberry angular leaf spot

Abstract

The invention discloses a strawberry angular leaf spot severity rapid grading method and device based on deep learning and computer vision, and belongs to the technical field of intelligent agriculture. The grading method comprises the following steps: acquiring original images of strawberry leaves; inputting the obtained strawberry leaf image into the trained strawberry angular leaf spot leaf segmentation model to obtain a segmentation result; fusing the image and the segmentation result to obtain a segmented color image; based on the obtained infected strawberry leaf segmentation color image, separating a disease spot part from a normal part to obtain a ratio of a disease area to the strawberry leaf in the image; and outputting the severity level of the strawberry angular leaf spot according to the proportion in combination with the environmental data. By adopting the method and the device disclosed by the invention, the infection condition of the strawberry angular leaf spot in the greenhouse can be rapidly and accurately obtained in real time, and support is provided for subsequent accurate pesticide application, new variety cultivation and the like.

Description

Technical Field

[0001] This invention relates to the field of smart agriculture, specifically to a method and apparatus for rapidly classifying the severity of strawberry angular leaf spot based on deep learning and computer vision. Background Technology

[0002] Strawberries are perennial herbaceous plants belonging to the genus *Fragaria* of the Rosaceae family, widely distributed and cultivated globally. Strawberry fruits are soft, juicy, and sweet-and-sour, rich in amino acids, vitamins, minerals, and other nutrients, giving them high nutritional value. Besides being primarily eaten fresh, they are also a raw material for processing various beverages and foods. With their short growth cycle and ease of management, and the strong demand for fresh strawberries in large and medium-sized cities, strawberries have become one of my country's most widely cultivated and important economic crops. They also possess certain medicinal value due to their cooling and thirst-quenching properties, as well as their ability to aid digestion.

[0003] Strawberry plants are short and susceptible to various diseases and pests. Common strawberry diseases include powdery mildew, gray mold, anthracnose, and root rot. Among these, angular leaf spot is a common and important disease. Its symptoms manifest in the early stages as small black spots on the leaf edges, followed by water-soaked, irregular reddish-brown lesions on the lower surface of the leaves. Later, these lesions enlarge and merge on the leaf surface. Without timely intervention and control, it can lead to plant death, reduced quality and yield, and economic losses. Therefore, rapid grading of angular leaf spot that may occur during strawberry growth is of great importance to strawberry production.

[0004] In recent years, with the development of technologies such as drones, machine learning, and deep learning, computer vision technology has been successfully applied to the detection of angular leaf spot disease in strawberries. However, most current research and applications are limited to laboratory environments. Deploying deep learning-based object detection to actual production environments, especially when mounted on detection devices, still faces several shortcomings: First, deep learning models typically have a large number of parameters, while detection devices have limited computing resources, leading to reduced efficiency. Second, deep learning-based object detection algorithms need to balance real-time performance and accuracy. One type is two-stage object detection based on candidate regions, which has high recognition accuracy but complex network structures and long detection times, failing to meet real-time detection requirements. The other type is single-stage object detection based on regression, which is fast and computationally inexpensive, meeting real-time requirements, but its recognition accuracy for small targets needs improvement.

[0005] In the prior art, invention patent CN111027597A discloses a method and device for rapid classification of strawberry diseases and pests. It continuously acquires strawberry disease and pest image data from multiple platforms to form a strawberry disease and pest database; performs unified calibration on the strawberry disease and pest image data; selects a sample set from the calibrated strawberry disease and pest database; establishes a deep convolutional neural network model, the input of which is the original strawberry disease and pest image, and the output is a strawberry disease and pest classification vector; feeds the sample set into the deep convolutional neural network model for training, and uses S-fold cross-validation to select the model; inputs the strawberry disease and pest image to be classified into the selected model for calculation, and obtains the classification result of the strawberry disease and pest to be classified. The deep convolutional neural network model and S-fold cross-validation algorithm used in this invention consume large amounts of computational resources and are difficult to deploy.

[0006] Chinese patent application CN114596513A discloses a method for identifying mutilated fruit disease in highbush blueberries based on deep multi-task learning. The method involves: S2: collecting a dataset of images of mutilated fruit disease in highbush blueberries; S3: annotating the image dataset to generate datasets for identifying infected parts, classifying infection stages, and estimating infection severity; S4: augmenting the annotated image dataset; S5: building a deep multi-task blueberry mutilated fruit disease identification model based on the EfficientNet model; S6: training the deep multi-task blueberry mutilated fruit disease identification model built in S3 using the augmented image dataset from step S2; and S7: using the trained model to identify infected parts, classify infection stages, and estimate infection severity in blueberry mutilated fruit. However, this invention suffers from decreased accuracy due to the use of the EfficientNet model and the augmented image dataset, making it difficult to accurately identify the disease. Summary of the Invention

[0007] To address the problems of limited computing resources, low computational accuracy, and difficulty in deploying existing strawberry leaf spot detection methods in actual production environments, this invention provides a rapid grading method and device for the severity of strawberry leaf spot based on deep learning and computer vision.

[0008] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows: A rapid grading method for the severity of strawberry angular leaf spot, including S1. Obtain the original image of the strawberry leaf and perform preprocessing; S2. Input the preprocessed image into the trained strawberry angular leaf spot leaf segmentation model to perform angular leaf spot detection and leaf region segmentation, and obtain the segmentation mask image of the infected strawberry leaf. S3. The segmentation mask image of the infected strawberry leaf is fused with the original image of the strawberry leaf to obtain a color image of the segmentation of the infected strawberry leaf; S4. Based on the obtained segmented color images of diseased strawberry leaves, separate the lesion areas from the normal areas, calculate the pixel area of ​​each part, obtain the pixel area of ​​the lesion and the pixel area of ​​the leaf, and obtain the ratio of the diseased area to the strawberry leaf area, i.e., the percentage of leaf lesion area. ; S5. Based on the obtained percentage of leaf lesion area By combining environmental temperature and humidity data, a joint objective function of "disease spot-microclimate" is constructed to output the severity level of strawberry angular leaf spot disease.

[0009] The beneficial effects of this invention are as follows: First, this invention provides a rapid grading method for the severity of angular leaf spot disease in greenhouse strawberries based on deep learning and computer vision. It uses a convolutional neural network model and a threshold segmentation algorithm to complete the grading task in two stages, separating the identification and segmentation of leaves from the grading of the severity of angular leaf spot disease. Compared with the previous one-stage algorithm that directly grades the disease using a convolutional neural network model, the model training speed is faster and the amount of computation is greatly reduced. It is more suitable for real-time detection tasks and provides a strong guarantee for better detection of strawberry leaf damage.

[0010] Second, the convolutional neural network model of this invention is the best model developed based on YOLOv11. A lightweight adaptive multi-scale fusion network is embedded in the neck: P3-P5 channel alignment is achieved through "nearest neighbor + 1×1 convolution", and multi-scale features are fused with 3D convolution, which significantly enhances the cross-layer semantic expression of small patches. At the same time, the original C3k2 module is completely replaced with CSPPC convolution, which reduces the computational cost by 20% and increases the inference speed by 1.3 times while maintaining gradient reuse. Furthermore, MLCA hybrid local-global channel attention is introduced in front of the detection head, which enables the network to focus on the reddish-brown lesion area under complex lighting and variety differences, and improves the background suppression ability. This model can detect and segment strawberry leaves well under complex backgrounds, improving the detection accuracy of the entire method.

[0011] Third, this invention provides a rapid grading device for the severity of angular leaf spot disease in greenhouse strawberries. The system has a high degree of integration; by using fewer sensors and a simple WiFi communication module, it improves system integration, simplifies hardware and software design, and reduces maintenance and upgrade difficulties. It is also easy to operate, requiring no professional agronomists or on-site measurements. Relevant personnel can control the vehicle and obtain real-time results of angular leaf spot disease severity detection in greenhouse strawberries simply through a computer interface, saving labor costs, improving efficiency, and providing strong support for better detection of strawberry leaf damage and subsequent precise pesticide application. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic diagram of the grading method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the image processing flow of the hierarchical method provided in the embodiments of the present invention; Figure 3 This is a schematic diagram of the convolutional neural network model architecture provided in an embodiment of the present invention; Figure 4 , Figure 5 A schematic diagram of the MLCA attention mechanism provided in this embodiment of the invention; Figure 6 This is a schematic diagram of the CSPPC module structure provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of the strawberry leaf angular spot disease detection cart provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] Furthermore, the following description is for illustrative purposes and not for limitation, and sets forth specific details such as particular system structures and techniques to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted to avoid unnecessary detail that could obscure the description of the invention.

[0016] Reference Figure 1 The first embodiment of the present invention relates to a rapid grading method for the severity of strawberry angular leaf spot disease, including... S1. Obtain the original image of the strawberry leaf and perform preprocessing; S2. Input the preprocessed image into the trained strawberry angular leaf spot leaf segmentation model to perform angular leaf spot detection and leaf region segmentation, and obtain the segmentation mask image of the infected strawberry leaf. S3. The segmentation mask image of the infected strawberry leaf is fused with the original image of the strawberry leaf to obtain a color image of the segmentation of the infected strawberry leaf; S4. Based on the obtained segmented color images of diseased strawberry leaves, separate the lesion areas from the normal areas, calculate the pixel area of ​​each part, obtain the pixel area of ​​the lesion and the pixel area of ​​the leaf, and obtain the ratio of the diseased area to the strawberry leaf area, i.e., the percentage of leaf lesion area. ; S5. Based on the obtained percentage of leaf lesion area By combining temperature and humidity data obtained from sensors in the environment, a joint objective function of "disease spot-microclimate" is constructed to output the severity level of strawberry angular leaf spot disease.

[0017] In some alternative embodiments, step S1, the preprocessing includes image distortion correction, data augmentation, and image scaling.

[0018] To accurately calculate the actual pixel size of each part of a strawberry leaf, image distortion correction is performed on the acquired original images of the strawberry leaf. This is achieved by obtaining the camera's intrinsic and distortion parameters through camera calibration, and then using these two parameters to correct image distortion introduced by the camera. Specifically, this includes: The coordinates (u, v) of the two-dimensional image of the strawberry leaf in the pixel coordinate system are converted to the coordinates (x, y) of the strawberry leaf in the image coordinate system according to the conversion formula. The origin of the pixel coordinate system is moved to the center of the image to serve as the origin of the image coordinate system. The conversion formula is as follows:

[0019] The pixel coordinates of the image center are (u0, v0), and the physical size of each pixel of the industrial camera's photosensitive device is dx*dy. The coordinates (x, y) of the strawberry leaf in the image coordinate system are converted to coordinates in the camera coordinate system with the center of the camera as the origin according to the conversion formula. , , The conversion formula is:

[0020] Where f is the camera focal length, which is the distance between the origin of the camera coordinate system and the image coordinate system, and x and y represent the number of horizontal and vertical pixels of the strawberry leaf in the image coordinate system. This refers to the distance of the strawberry leaf from the optical center. The camera coordinate system indices are converted to coordinates in the world coordinate system with the ground as the origin according to the conversion formula. , , The conversion formula is:

[0021] Where u and v are the center pixel coordinates of the image in the pixel coordinate system; R is a 3×3 rotation matrix that describes the rotation transformation of the camera; and T is a translation vector that describes the translation of the camera in the x, y, and z directions. Based on the above principles, the camera's intrinsic, extrinsic, and distortion parameters can be calibrated. The intrinsic parameter matrix is ​​as follows: With the extrinsic parameter matrix is They are respectively:

[0022]

[0023] Based on the camera's intrinsic and extrinsic parameters, the pixel coordinates (x, y) of the image after distortion correction are obtained as follows: .

[0024] The data augmentation methods include brightness and contrast adjustment, image inversion, noise addition, and image translation.

[0025] In some preferred embodiments, in step S2, the strawberry angular leaf spot disease segmentation model uses YOLOv11 as the baseline model and includes an input layer, a backbone feature extraction layer, an adaptive multi-scale fusion network, a detection head, and a segmentation head arranged sequentially. Based on this, a lightweight CSPPC convolutional structure is introduced into the adaptive multi-scale fusion network to replace the original C3k2 module, and an MLCA attention module is introduced into the detection head and the segmentation head.

[0026] Reference Figures 3 to 6 In the process of constructing the leaf segmentation model, an input layer is first constructed to receive the strawberry leaf image data to be processed, which is used as the initial input information of the model to lay the foundation for subsequent feature extraction and other operations.

[0027] Next, a backbone feature extraction layer is set up. The core function of this layer is to extract deep features from the input image. Through multiple convolutional layers, pooling layers, and other neural network structures, key features such as edges, textures, and shapes in the image are extracted step by step. These features can effectively characterize the appearance characteristics of strawberry leaves and their angular leaf spot disease, providing rich and discriminative feature information to the output of the backbone feature extraction layer, enabling subsequent networks to perform more accurate analysis and recognition based on these features.

[0028] Following the core feature extraction layer, a lightweight adaptive multi-scale fusion network (ASF-Net) is constructed. Its main purpose is to further optimize and enhance feature representation to better adapt to the requirements of strawberry angular leaf spot segmentation tasks. The adaptive multi-scale fusion network aligns channels at three different feature scales (P3, P4, and P5) using "nearest neighbor interpolation + 1×1 convolution," then stacks them into pseudo-3D tensors in a "low-medium-high" order. After 3D convolution, batch normalization, and max pooling, these tensors are compressed into features of a unified scale, achieving cross-scale semantic enhancement.

[0029] This embodiment innovatively introduces the CSPPC lightweight convolutional structure to replace the original C3k2 module in ASF-Net. While maintaining gradient reuse and cross-stage information flow, the CSPPC convolutional structure can reduce the overall computational cost by 20% and improve actual inference speed. Specifically, the CSPPC convolutional structure reduces computation and memory access through partial convolution (PConv) while maintaining the integrity of the feature information flow, thereby achieving fast and efficient computation.

[0030] Specifically, the CSPPC convolutional structure first reduces the number of channels in the input feature map by half using a 1×1 convolutional module, then applies partial convolution (PConv) to the remaining half of the channels, and finally restores the number of channels using another 1×1 convolutional module. This design not only reduces computation but also preserves important spatial features through partial convolution, making the network more efficient in processing strawberry leaf and angular leaf spot features.

[0031] Before feeding the aligned features into the detection head, a hybrid local-global channel attention (MLCA) mechanism is further introduced. MLCA first performs local and global average pooling on the input feature map, and then performs feature transformation through 1D convolution. The locally pooled features are combined with the original input features, while the globally pooled features are combined with the locally pooled features. Finally, unpooling is used to restore the original spatial dimension. This method significantly improves detection accuracy while maintaining computational efficiency.

[0032] Finally, a segmentation head is set up to perform the final segmentation prediction based on the features extracted and enhanced in the previous layers. An MLCA attention module is also introduced into the segmentation head. This module learns the importance weights of each feature channel. In this way, the model can filter out target regions with important information from many irrelevant background regions, focusing more attention on strawberry leaves and the angular leaf spot lesions on the leaves. This helps improve the accuracy of angular leaf spot detection and leaf segmentation, enabling the model to more accurately locate the angular leaf spot disease area and determine the complete outline of the leaf, providing a high-quality segmentation result foundation for subsequent angular leaf spot severity grading.

[0033] The strawberry angular leaf spot disease leaf segmentation model constructed using the above method, based on the YOLOv11 baseline model, integrates the CSPPC lightweight convolutional structure and MLCA attention module, which can effectively improve the segmentation effect of strawberry angular leaf spot disease leaves, and provide a reliable leaf segmentation basis for realizing rapid classification of the severity of angular leaf spot disease in greenhouse strawberries.

[0034] To train a strawberry angular leaf spot leaf segmentation model, original strawberry leaf images need to be collected as model data. Image acquisition can be performed using a rapid grading system for the severity of angular leaf spot in greenhouse strawberries, as described in the second embodiment of this invention, without limiting the steps described. The original strawberry leaf images can also be sourced from the following two channels: one is strawberry disease images publicly provided by Afzaal, the AI ​​laboratory of the Department of Computer Science and Engineering at Jeonbuk National University in South Korea; the other is strawberry angular leaf spot images obtained online through web crawling.

[0035] First, image annotation and data preparation are performed on the model data. In this embodiment, an annotation tool is used to annotate strawberry leaves. Semantic segmentation is used to completely cover the leaves and assign labels indicating whether they are infected with angular leaf spot disease. Subsequently, a convolutional neural network model is selected as the base model. The model training process includes data preprocessing, feature extraction, network training, and parameter optimization to improve the model's accuracy. Model validation and evaluation are then performed, and training parameters and model structure are adjusted until satisfactory recognition results are achieved. This constructs a strawberry angular leaf spot disease segmentation model. Based on this model, the identification and segmentation of infected strawberry leaves can be realized.

[0036] In some alternative embodiments, the training of the strawberry angular leaf spot disease leaf segmentation model uses binary cross-entropy as the loss function and the Adam optimizer is used for parameter optimization.

[0037] The mean accuracy of the evaluation index is selected when the intersection-union ratio threshold is 0.50. map50 represents the overall accuracy of the model. The final detection model is obtained after multiple validations and optimizations on the validation set. When map50 > 0.95, the model meets the requirements and can be used for leaf segmentation in strawberry angular leaf spot detection.

[0038] The segmentation mask image of the diseased strawberry leaf obtained in step S2 is a binary mask image. The leaf part is white with a corresponding pixel value of 255, and the background part is black with a corresponding pixel value of 0.

[0039] In some alternative embodiments, in step S3, a bitwise AND operation is used to fuse the segmented mask image with the original strawberry leaf image.

[0040] In some preferred embodiments, step S4 involves separating the lesion portion from the normal portion: The obtained color images of diseased strawberry leaves were preprocessed. The preprocessing method was median filtering. Specifically, let the pixel value of the x-th column and y-th row of the original image be f(x,y), and the pixel value of the x-th column and y-th row after median filtering be g(x,y): g(x,y)=median(f(xi,yj)|-k≤i≤k,-k≤j≤k) median means median, and k means filter radius; Step S4 specifically includes: S41. Obtain color segmentation images of infected strawberry leaves. Collect and obtain color segmentation images of infected strawberry leaves that have undergone preprocessing (such as washing and removing impurities) to ensure the accuracy of subsequent analysis.

[0041] S42. Using a color space conversion algorithm, convert the segmented color image of the diseased strawberry leaf from the RGB color space to the LAB color space (L represents brightness, A represents the components from green to red, and B represents the components from blue to yellow), and obtain the A channel (red-green axis) image I in the LAB space. A (x,y), where (x,y) are pixel coordinates. The A channel is sensitive to reddish-brown lesions and can significantly widen the grayscale distance between lesions and normal leaves, which helps to better separate and analyze the color differences between lesions and normal leaf parts in the image.

[0042] S43, for the A channel image I A Construct a grayscale histogram H(k) for (x, y):

[0043] Where δ(·) is the Kronecker delta function.

[0044] S44. Based on the histogram H(k), calculate three candidate adaptive thresholds in parallel. : Otsu threshold This is obtained by maximizing the inter-class variance:

[0045]

[0046]

[0047]

[0048]

[0049]

[0050] Where N: the total number of pixels in the image; Weight of the foreground (lesion); Weights for the background (normal blades); : Average value of background (normal leaves); : The average value of the foreground (lesion); : Between-class variance.

[0051] Valley-emphasis threshold Determined by maximizing concavity:

[0052]

[0053]

[0054] Where C(k): concavity in the Valley-emphasis method; Kapur Maximum Entropy Threshold Obtained by maximizing total entropy:

[0055]

[0056]

[0057]

[0058]

[0059] in: : Normalized histogram probability distribution; Entropy of the foreground (lesion); Entropy of the background (normal blades); Total entropy.

[0060] S45. Define an unsupervised quality metric Q(t) to select the optimal threshold t from candidate adaptive thresholds. best :

[0061]

[0062]

[0063]

[0064] in, The pixel value of the A channel image at position (x,y); : Average value of background (normal leaves); : The average value of the foreground (lesion); : Normalized histogram probability distribution.

[0065] S46, using t best Binarize the A channel image to obtain the initial lesion mask. :

[0066] S47. Perform morphological opening and area filtering on the initial lesion mask M0 to obtain the final lesion mask M(x,y): M=Open(M0,SE) Where SE is a 3×3 elliptical structural element, and Open(·) indicates erosion followed by expansion; S48. Calculate the percentage of leaf lesion area R:

[0067] Where W and H are the width and height of the segmented color image, respectively.

[0068] In some preferred embodiments, in step S5, a joint objective function for "lesion-microclimate" is constructed: S50, Microclimate Normalization S501, Obtain Temperature Rating S T Because the optimal temperature range for strawberry angular leaf spot disease to spread is around 20°C, and the disease basically does not occur above 30°C,

[0069] Where T is the current temperature in degrees Celsius. This formula maps the temperature T to the interval (0,1) using a Gaussian function. The closer the temperature is to 20°C, the higher the score, indicating that the environment is more conducive to disease occurrence.

[0070] S502, Obtain humidity score S RH The pathogen requires RH ≥ 60% to produce spores.

[0071] Where RH is the current relative humidity, expressed as a percentage. This formula maps the humidity RH to the [0,1] interval using a piecewise linear function. The higher the humidity, the higher the score, indicating that the environment is more conducive to disease occurrence.

[0072] S51. Construct the joint objective function (Severity Score) Define a weighted geometric-arithmic hybrid objective function

[0073] Wherein, β is the visual weight, which is optimally 0.85 after searching through 1000 sets of field sample grids. It is fixed for all scenarios and does not require further parameter adjustment; R is the proportion of lesions, which is calculated by step S4.

[0074] The closer the SS value is to 1, the more favorable the current environment is for the spread of the disease and obvious lesions have appeared on the leaves, requiring immediate intervention.

[0075] S52. Establish dynamic hierarchical intervals The SS values ​​are mapped to the Level 4 warning range, as shown in Table 1: Table 1

[0076] Therefore, when inspecting strawberry plants, refer to Figure 2 : Raw images of strawberry leaves were acquired using a camera and then preprocessed. The image is input into the trained strawberry angular leaf spot disease segmentation model for angular leaf spot disease detection and leaf region segmentation to obtain a binarized mask image of the infected strawberry leaf. The binarized masked image of the infected strawberry leaf is fused with the original image of the strawberry leaf to obtain a segmented color image of the infected strawberry leaf; The obtained color images of diseased strawberry leaves were segmented and processed using median filtering to remove noise from the images; Convert the processed image from the RGB color space to the LAB color space; Segmenting lesions and leaves, and based on the color characteristics of infected strawberry leaves in the LAB color space, analyzing the A channel image I... A (x,y) A grayscale histogram is constructed. Based on the histogram H(k), three candidate adaptive thresholds are calculated in parallel. The optimal threshold is selected from the three adaptive candidate thresholds according to the unsupervised quality index. Check whether the A channel value of each pixel falls within the set lesion threshold range or the normal part threshold range. If the pixel falls within the lesion range, it is classified as a lesion part; if it falls within the normal range, it is classified as a normal part. The image is segmented into two regions: lesion area and normal area, thereby enabling the detection of lesions and identification of normal parts on infected strawberry leaves. A joint objective function of "lesion-microclimate" is constructed based on the obtained leaf lesion area ratio R and environmental temperature and humidity data, and finally outputs the severity level of strawberry angular leaf spot.

[0077] Reference Figure 7 The second embodiment of the present invention relates to a rapid grading device for the severity of strawberry angular leaf spot disease, comprising: Mobile chassis 1, working platform 2, image acquisition module, sensor group, control drive module, host computer, remote interaction module; The mobile chassis 1 is used to drive the grading device to move along the ground; The work platform 2 is mounted on the mobile chassis 1; The image acquisition module includes a lifting device 4 and a camera 5, which are used to acquire images of strawberry leaves. The camera 5 is connected to the lifting device 4 in a transmission manner, and the lifting device 4 is used to drive the camera 5 to rise and fall. The sensor group includes a lidar 6 and a depth camera 7; The control and drive module is connected to the mobile chassis 1, the image acquisition module, and the sensor group respectively. It is used to drive the mobile chassis 2 to move the entire device in the greenhouse and control the lifting device 4 to move the camera 5 to a suitable height. The host computer is used to control the camera 5 to take pictures, and to detect and save the grading results using the rapid grading method for the severity of angular leaf spot disease in greenhouse strawberries described in the first embodiment. The host computer is equipped with a robot operating system that can control the movement of the trolley and acquire images of strawberry leaves. It also embeds the rapid grading method for the severity of angular leaf spot disease in greenhouse strawberries based on deep learning and computer vision, and designs a UI interface based on Python and PyQt6 to realize the detection function, forming a portable, efficient, and remote detection device for grading the severity of angular leaf spot disease in greenhouse strawberries. The remote interaction module is used by users to remotely control the device and obtain real-time detection results via computer equipment.

[0078] As a further improvement to the above embodiments, the grading device described in this embodiment can be a ROS-based strawberry leaf angular spot disease detection and grading vehicle, including a mobile chassis 1, a working platform 2, an electrical frame 3, a lifting device 4, a camera 5, a lidar 6, a depth camera 7, a power supply mechanism 8, a microcontroller driver board 9, a host computer 10, and a WiFi communication module 11, etc.

[0079] The mobile chassis 1 is the supporting foundation of the vehicle, which includes a power supply mechanism 8, a base plate 101, a set of DC geared motors 103, two drive wheels 102, and two driven wheels 104. Each DC geared motor 103 is connected to one drive wheel 102 for transmission, and each DC geared motor 103 is also connected to the microcontroller driver board 9 for signal transmission.

[0080] The power supply mechanism 8 provides power support for the vehicle and includes a battery and an output port, which are electrically connected to the DC geared motor 103, the lifting device 4, the camera 5, the lidar 6, the depth camera 7, the microcontroller driver board 9, the host computer 10, and the WiFi communication module 11, respectively.

[0081] The work platform 2 is installed on the mobile chassis 1 and is used to support the lifting device 4 and other related equipment.

[0082] The electrical appliance frame 3 is a frame structure with upper and lower platforms, and is fixedly installed on the mobile chassis 1. The lidar 6, depth camera 7, and WiFi communication module 11 are installed on the upper platform of the electrical appliance frame 3 and are all fixedly connected to the upper platform; the microcontroller driver board 9 and the host computer 10 are installed on the lower platform of the electrical appliance frame 3 and are all fixedly connected to the lower platform.

[0083] The lifting device 4 is installed on the work platform 2 and is used to drive the camera 5 to move up and down.

[0084] The camera 5 is used to capture images of strawberry leaves. It is connected to the lifting device 4. The height of the camera 5 can be adjusted by the lifting action of the lifting device 4 to adapt to strawberry plants of different heights, ensuring the accuracy and comprehensiveness of image acquisition.

[0085] The aforementioned lidar 6 is used to acquire environmental information around the vehicle in real time, providing data support for the vehicle's navigation and path planning; The depth camera 7 is used to assist in locating and identifying the position and shape of strawberry plants; The WiFi communication module 11 is used to realize the signal connection between the vehicle and the remote control device 12, so that the user can control and operate the vehicle through the remote control device 12; The single-chip microcomputer driver board 9 is used to receive control commands from the host computer 10 and drive the DC geared motor 103 and other actuators to realize the movement and lifting of the trolley. The host computer 10 serves as the control core of the system. It receives instructions from the remote control device 12 through the WiFi communication module 11 and coordinates the control of the lidar 6, depth camera 7, lifting device 4, and camera 5 to achieve automated image acquisition of strawberry leaves and grade the severity of strawberry angular leaf spot disease.

[0086] In some optional embodiments, the specific steps for the strawberry leaf angular spot disease detection and grading cart to acquire strawberry leaf images are as follows: S1. The user sends a task command to the vehicle through the remote control device 12; S2. After receiving the instruction, the host computer 10 controls the LiDAR 6 to start, acquire information about the surrounding environment, and plan a suitable path according to the instruction content; at the same time, it controls the depth camera 7 to start, identify the position and shape of the strawberry plant; then, according to the height of the strawberry plant, it controls the lifting device 4 to adjust the height of the camera 5 to make it in the best acquisition position. S3. Finally, control camera 5 to start, acquire images of strawberry leaves, obtain the severity level of strawberry angular leaf spot according to the rapid grading method of strawberry angular leaf spot based on deep learning and computer vision described in the previous embodiment, and store or transmit the obtained results to remote control device 12 for subsequent pest and disease analysis and processing.

[0087] In some optional embodiments, the lidar 6 is a YDLidar G4 lidar, which features a 360-degree scanning range and high-precision measurement, enabling it to acquire real-time environmental information around the vehicle and provide accurate data for path planning. The depth camera 7 in this embodiment is an Intel RealSense D435 depth camera. This depth camera 7 can provide high-resolution depth images, assisting the vehicle in locating and identifying the position and shape of the strawberry plants.

[0088] In some optional embodiments, the path planning algorithm involved employs the A* algorithm, a heuristic search algorithm capable of quickly finding the optimal path from the starting point to the destination in complex environments. The specific steps are as follows: S1. Discretize the greenhouse environment into a grid map, where each grid represents a unit that a small vehicle can move in.

[0089] S2. Based on the environmental information obtained by LiDAR 6, mark the grid with obstacles as an impassable area.

[0090] S3. Starting from the current position of the car and ending at the target position, use the A* algorithm to search for the optimal path on the grid map.

[0091] S4. The car moves according to the searched path, while updating the LiDAR data in real time and dynamically adjusting the path.

[0092] In some alternative embodiments, the power supply mechanism uses a 24V / 20Ah lithium battery pack to provide stable power support for the vehicle. This battery pack has a large capacity, which can meet the vehicle's long-term operating needs inside the greenhouse, effectively extending the robot's working time and improving operational efficiency.

[0093] In some optional embodiments, the microcontroller development board uses an STM32F407 microcontroller. This microcontroller features a high-performance ARM Cortex-M4 core with a main frequency of up to 168MHz, and has rich peripheral interfaces to meet the control requirements of the vehicle, such as motor drive and sensor data acquisition. It receives data from cameras and LiDAR navigation, processes and analyzes this data, and performs calculation, decision-making, and control functions based on algorithms and design logic. For example, received image information and environmental data are processed to generate robot action commands, directing the drive module to execute movement. In addition, it connects to a host computer and a WiFi module for remote monitoring and control, thereby enabling more complex command and information sharing.

[0094] In some optional embodiments, the WiFi communication module is an ESP8266 module. This module features low power consumption and high performance, supports multiple communication protocols, and can achieve a stable signal connection between the vehicle and remote control equipment.

[0095] Considering the varying strawberry cultivation methods in different greenhouses and the complex operating environment of the robot, some optional embodiments employ a combination of camera and LiDAR navigation modules to provide environmental perception and navigation functions. LiDAR provides environmental contour information, while depth cameras provide precise obstacle distance information. The combination of these two improves navigation accuracy, enabling autonomous navigation and obstacle avoidance. During path planning, real-time information on the spatial structure and obstacle distribution within the greenhouse is collected to ensure the robot's safe movement. By emitting and receiving reflected laser signals, distances to surrounding objects are measured, generating two-dimensional or three-dimensional maps of the environment. This helps the robot update the map in real time, identify obstacles, and calculate the optimal path. Cameras capture image information of the surrounding environment, which, combined with LiDAR data, provides a more comprehensive environmental perception capability. Cameras can help identify dynamic objects, crop conditions, and potential obstacles. The navigation module can formulate efficient and safe path plans, thereby reducing human intervention and improving the accuracy and efficiency of inspections.

[0096] Considering that the vehicle may run out of power during long-term operation and that charging in the greenhouse is difficult and labor-intensive, a low-battery alarm mechanism is set in the power supply system of some optional embodiments. When the battery power is lower than 20%, the vehicle will automatically return to the charging area for charging and send an alarm signal to the remote control device to remind the user to replace the battery or charge it in time.

[0097] Considering the possibility of unstable WiFi signal strength in the greenhouse, some optional embodiments also provide a backup plan. If the signal strength is detected to be below the threshold, the vehicle will automatically switch to a backup communication method, such as a 4G module, to ensure a stable connection with the remote control device.

[0098] The strawberry pest and disease image detection and grading vehicle in this embodiment is designed based on the ROS framework. It achieves remote control through a lidar navigation system, a depth camera-assisted positioning system, and a WIFI communication module. It can efficiently and accurately collect images of strawberry leaves and obtain the severity grading results of strawberry angular leaf spot disease, providing strong technical support for the early detection and precise control of strawberry pests and diseases. It has broad application prospects and important practical significance.

[0099] Figure 8 This is a schematic diagram of an electronic device provided in another embodiment of the present invention. For example... Figure 8 As shown, the electronic device in this embodiment includes a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a rapid grading method program for the severity of strawberry angular leaf spot. When the processor executes the computer program, it implements the steps in the above-described rapid grading method embodiment for the severity of strawberry angular leaf spot, for example... Figure 1 The steps are shown.

[0100] For example, a computer program can be divided into one or more modules / units, one or more of which are stored in memory and executed by a processor to complete the present invention. One or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in an electronic device.

[0101] Electronic devices can be computing devices such as desktop computers, laptops, handheld computers, and cloud servers. Electronic devices may include, but are not limited to, processors and memory. Those skilled in the art will understand that... Figure 8 This is merely an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than shown, or combine certain components, or different components. For example, an electronic device may also include input / output devices, network access devices, buses, etc.

[0102] The processor referred to can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0103] Memory can be an internal storage unit of an electronic device, such as a hard drive or RAM. Memory can also be an external storage device, such as a plug-in hard drive, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal and external storage units. Memory is used to store computer programs and other programs and data required by the electronic device. Memory can also be used to temporarily store data that has been output or will be output.

[0104] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0105] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0106] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0107] In the embodiments provided by this invention, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0108] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0109] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0110] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0111] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A rapid grading method for the severity of strawberry angular leaf spot disease, characterized by: Includes the following steps S1. Obtain the original image of the strawberry leaf and perform preprocessing; S2. Input the preprocessed image into the trained strawberry angular leaf spot leaf segmentation model to perform angular leaf spot detection and leaf region segmentation, and obtain the segmentation mask image of the infected strawberry leaf. S3. The segmentation mask image of the infected strawberry leaf is fused with the original image of the strawberry leaf to obtain a color image of the segmentation of the infected strawberry leaf; S4. Based on the obtained segmented color image of the infected strawberry leaf, separate the lesion area from the normal area, calculate the pixel area of ​​each part, obtain the pixel area of ​​the lesion and the pixel area of ​​the leaf, and obtain the ratio of the diseased area to the strawberry leaf area, i.e., the percentage of leaf lesion area. ; S5. Based on the obtained percentage of leaf lesion area By combining environmental temperature and humidity data, a joint objective function of "lesion spot-microclimate" is constructed to output the severity level of strawberry angular leaf spot disease.

2. The rapid grading method according to claim 1, characterized in that: In step S2, the strawberry angular leaf spot disease segmentation model uses YOLOv11 as its backbone and includes, sequentially from input to output, an input layer, a backbone feature extraction layer, an adaptive multi-scale fusion network, a detection head, and a segmentation head; wherein... The adaptive multi-scale fusion network completes channel alignment at different feature scales through "nearest neighbor interpolation + 1×1 convolution", and then stacks them in order into pseudo 3D tensors. After 3D convolution, batch normalization and max pooling, they are compressed into features of a unified scale, thereby achieving cross-scale semantic enhancement. The adaptive multi-scale fusion network replaces the C3k2 module of YOLOv11 with the CSPPC convolution module. The CSPPC convolution module first reduces the number of channels of the input feature map by half through a 1×1 convolution module, then applies partial convolution to the remaining half of the channels, and finally restores the number of channels through a 1×1 convolution module. This achieves the reduction of overall computation and improvement of actual inference speed while maintaining gradient reuse and cross-stage information flow. Before the aligned features are fed into the detection head, a hybrid local-global channel attention mechanism (MLCA) is introduced. The segmentation head is used to perform the final segmentation prediction based on the features extracted and enhanced from the previous layers.

3. The rapid grading method according to claim 2, characterized in that: The training of the strawberry angular leaf spot disease leaf segmentation model uses binary cross-entropy as the loss function and the Adam optimizer is used for parameter optimization. The mean accuracy of the evaluation index is selected as the value of the cross-union ratio threshold of 0.

50. map50 represents the overall accuracy of the model. The final detection model is obtained after multiple validations and optimizations on the validation set.

4. The rapid grading method according to claim 1, characterized in that: Step S4 specifically includes: S41. Obtain color images of diseased strawberry leaves and separate the diseased and healthy parts: The obtained color images of diseased strawberry leaves were preprocessed, specifically as follows: Let f(x,y) be the pixel value in the x-th column and y-th row of the original image, and g(x,y) be the pixel value in the x-th column and y-th row after median filtering: g(x,y)=median(f(xi,yj)|-k≤i≤k,-k≤j≤k); median means median, and k means filter radius; S42. Using a color space conversion algorithm, the segmented color image of the diseased strawberry leaf is converted from the RGB color space to the LAB color space to obtain the A channel image I in the LAB space. A (x,y), where (x,y) are pixel coordinates; S43, for the A channel image I A Construct a grayscale histogram H(k) for (x, y): Where δ(·) is the Kronecker delta function; S44. Based on the grayscale histogram H(k), calculate three candidate adaptive thresholds in parallel. ; S45. Define unsupervised quality indicators Used to select the optimal threshold t from the candidate thresholds. best ; S46, using t best Binarize the A channel image to obtain the initial lesion mask. : S47. Perform morphological opening and area filtering on the initial lesion mask M0 to obtain the final lesion mask M(x,y): M=Open(M0,SE) Where SE is a 3×3 elliptical structural element, and Open(·) indicates erosion followed by expansion; S48. Calculate the percentage of leaf lesion area R: Where W and H are the width and height of the image, respectively.

5. The rapid grading method according to claim 4, characterized in that: In step S44, three candidate thresholds are calculated in parallel. The specific method is as follows: Otsu threshold This is obtained by maximizing the inter-class variance: Where N: the total number of pixels in the image; The weight of foreground lesions; : Weights of normal background blades; : Average value of normal leaves in the background; : The average value of foreground lesions; Between-class variance; Valley-emphasis threshold Determined by maximizing concavity: in, Concavity in the Valley-emphasis method; Kapur Maximum Entropy Threshold Obtained by maximizing total entropy: in, : Normalized histogram probability distribution; : Entropy of foreground lesions; : Entropy of a normal leaf in the background. Total entropy.

6. The rapid grading method according to claim 4, characterized in that: In step S45, unsupervised quality indicators are defined. Used to select the optimal threshold t from the candidate thresholds. best The specific method is as follows: in, The pixel value of the A channel image at position (x,y); : Average value of normal leaves in the background; : The average value of foreground lesions; : Normalized histogram probability distribution.

7. The rapid grading method according to claim 1, characterized in that: In step S5, the specific method for constructing the joint objective function of "lesion spot-microclimate" is as follows: S50, Microclimate Normalization S501, Obtain Temperature Rating S T Given that the optimal temperature range for strawberry angular leaf spot disease is around 20°C, and that the disease generally does not occur above 30°C, Where T is the current temperature; The temperature T is mapped to the (0,1] interval using a Gaussian function. The closer the temperature is to 20℃, the higher the score, indicating that the environment is more conducive to the occurrence of diseases. S502, Obtain humidity score S RH The pathogen requires RH ≥ 60% to produce spores. Where RH is the current relative humidity; The humidity RH is mapped to the [0,1] interval by a piecewise linear function. The higher the humidity, the higher the score, indicating that the environment is more conducive to the occurrence of diseases. S51. Constructing the joint objective function Define the weighted geometric-arithmic hybrid objective function: Where β is the visual weight and R is the proportion of lesions; the closer the SS value is to 1, the more favorable the current environment is for the spread of the disease and obvious lesions have appeared on the leaves, requiring immediate intervention. S52. Establish dynamic hierarchical intervals The SS value is mapped to a 4-level range, with the specific levels as follows: 。 8. A rapid grading device for the severity of strawberry angular leaf spot disease, characterized in that: include Mobile chassis, work platform, image acquisition module, sensor group, control and drive module, host computer, remote interaction module; The mobile chassis is used to drive the grading device to move along the ground; The work platform is mounted on the mobile chassis; The image acquisition module includes a lifting device and a camera, used to acquire images of strawberry leaves. The camera is connected to the lifting device via a transmission, and the lifting device is used to drive the camera to move up and down. The sensor array includes a lidar and a depth camera; The control and drive module is connected to the mobile chassis, the image acquisition module, and the sensor group respectively. It is used to drive the mobile chassis to move the entire device and control the lifting device to move the camera to a suitable height. The host computer is used to control the camera to take pictures. The host computer is equipped with a robot operating system that can control the movement of the trolley and acquire images of strawberry leaves. It also embeds the rapid grading method for the severity of strawberry angular leaf spot as described in any one of claims 1 to 7. The remote interaction module is used to remotely control the device and obtain real-time detection results via computer equipment.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the rapid grading method for the severity of strawberry angular leaf spot as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the rapid grading method for the severity of strawberry angular leaf spot as described in any one of claims 1 to 7.

Images