Targeted spraying plant protection method and system based on visual identification of unmanned aerial vehicle
By generating multiple sets of prediction boxes through deep learning networks and calculating the elliptical spraying range, the problem of mismatched spraying parameters in existing plant protection methods is solved, achieving efficient targeted spraying and reducing pesticide waste and environmental pollution.
Patent Information
- Application Number
- CN202511809896.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-03
- Publication Date
- 2026-03-06
AI Technical Summary
Existing plant protection methods cannot generate spraying parameters that closely match the actual conditions of diseases, leading to pesticide waste and environmental pollution, especially when dealing with irregular disease targets, where the positioning deviation is large or the feature expression is insufficient.
The disease feature map is extracted by deep learning backbone network, and the first set of candidate prediction boxes and the second set of candidate prediction boxes are generated. The elliptical spraying range is calculated by using the saliency of disease features and spatial clustering, and the plant protection equipment is controlled to perform targeted spraying.
It improved pesticide utilization, reduced accidental damage to healthy plants and environmental pollution, and achieved a high degree of consistency between the spraying range and the distribution of diseases.
Smart Images
Figure CN121616964A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of plant protection, and in particular relates to a targeted spraying plant protection method and system that utilizes the visual recognition of unmanned aerial vehicles (UAVs). Background Technology
[0002] Current plant protection methods often employ large-scale, indiscriminate spraying, leading to significant pesticide waste, increased production costs, and severe pollution of soil, water, and the ecological environment. Furthermore, it easily induces pesticide resistance in pests and diseases. With the development of agricultural technology, targeted spraying using drones, robots, and other equipment is becoming a trend. This technology identifies the location, extent, and severity of crop pests and diseases, enabling targeted and on-demand application of pesticides, thereby improving pesticide utilization and reducing environmental impact.
[0003] Currently, pest and disease identification technologies based on computer vision and deep learning are widely used in targeted spraying. Target detection models, such as YOLO and Faster R-CNN, are typically used to identify pests and diseases in farmland images and output their location information as rectangular candidate boxes. However, crop diseases often exhibit irregular, spot-like or diffuse distributions, and a single rectangular box is insufficient to delineate their true outline and affected area. Spraying decisions based on rectangular boxes can easily lead to missed or excessive spraying. In the label assignment stage of target detection, traditional assignment strategies based on single geometric metrics such as intersection-union ratio (IUU) struggle to balance the localization of pest targets with specific needs, especially when dealing with dense, small, or morphologically varied pest targets, often resulting in large localization errors or insufficient feature representation. Therefore, there is an urgent need for a plant protection method that can simultaneously identify the target location of pests and, based on their irregular distribution patterns, generate spraying parameters that highly match the actual condition of the pest. Summary of the Invention
[0004] This invention proposes a targeted spraying plant protection method using UAV visual recognition to solve the problem that existing plant protection methods cannot generate spraying parameters that highly match the actual situation of disease, including: An image of the crop area to be processed is acquired, and the image is input into a deep learning backbone network to extract feature maps containing pest and disease information; using the feature maps, a first set of candidate prediction boxes and a second set of candidate prediction boxes are generated in parallel in the detection head; For the first set of candidate prediction boxes, the salience of disease features is determined based on the irregularity of lesion shape, color gradient and texture entropy. Based on the weighted matching cost of disease feature salience, a set of multiple positive sample prediction boxes representing the distribution pattern are assigned to each identified disease target. For the second group of candidate prediction boxes, calculate the target spatial clustering degree of each candidate box within a preset neighborhood; determine the clustering degree threshold based on the average saliency of disease features in the local area where the candidate box is located; when the target spatial clustering degree of the candidate box is higher than the clustering degree threshold, determine a matching prediction box for localization for the same disease target based on the joint cost of classification confidence and normalized position deviation. The center coordinates of the matching prediction box are used as the target center, and the covariance matrix of the coordinate set of the multiple positive sample prediction boxes is calculated. The major and minor axes and direction parameters of the elliptical spraying range covering the distribution of the positive sample prediction boxes are determined according to the eigenvalues and eigenvectors of the covariance matrix. Based on the target center, major and minor axes and direction parameters, the spraying unit of the plant protection equipment is controlled to perform targeted spraying operations.
[0005] Optionally, the determination of disease feature salience based on lesion morphology irregularity, color gradient, and texture entropy is calculated using the following formula:
[0006] in, , , For preset weights, and ; The value is obtained by fitting the pixels of the lesion region to an circumscribed ellipse, and then subtracting the ratio of the lesion area to the area of the circumscribed ellipse from 1. The average gradient magnitude is obtained by applying the Sobel operator to the H and S channels in the HSV color space of the lesion area. The information entropy is obtained by converting the lesion area into a grayscale image, calculating the grayscale co-occurrence matrix of the grayscale image, and then calculating the information entropy based on the grayscale co-occurrence matrix.
[0007] Optionally, the calculation formula for the target spatial clustering degree of each candidate box within a preset neighborhood is as follows:
[0008] For each candidate box, a circular region with a radius of R and a center point of the candidate box is defined as a preset neighborhood; N is the number of center points of all other candidate boxes within the preset neighborhood, excluding the candidate box itself. The area is the neighborhood area.
[0009] Optionally, the clustering threshold is determined based on the mean saliency of disease features in the local area where the candidate box is located, and the calculation formula is as follows:
[0010] in, The value is the arithmetic mean of the disease characteristics significance of all identified lesions within the local area where the candidate box is located, and k is a preset proportional coefficient.
[0011] Optionally, the method of determining a matching prediction box for locating the same disease target based on the joint cost of classification confidence and normalized location bias includes: The formula for calculating the joint cost is as follows:
[0012] in, The classification confidence score for the candidate bounding boxes. The normalized Euclidean distance between the center point of the candidate box and the center point of its corresponding feature map grid cell is given by [reference to a specific location]. For coefficients; For each disease target, select the candidate box with the minimum joint cost C as its matching prediction box.
[0013] Optionally, determining the major and minor axes and direction parameters of the elliptical spraying range covering the positive sample prediction box distribution based on the eigenvalues and eigenvectors of the covariance matrix includes: Calculate the two eigenvalues of the covariance matrix and ,and Length of the major axis of the ellipse minor axis length ; Calculate and larger eigenvalues The corresponding feature vector is used to determine the angle between the feature vector and the principal axis of the coordinate system as the direction parameter of the elliptical spraying range.
[0014] Optionally, controlling the spraying unit of the plant protection equipment to perform targeted spraying operations based on the target center, major and minor axes, and direction parameters includes: The target center coordinates, the major and minor axes of the elliptical spraying range, and the direction parameters are encoded into a set of pulse width modulation control signals; The pulse width modulation (PWM) control signal is sent to the flight controller and spraying control module of the plant protection equipment. The flight controller adjusts the drone's hovering position based on the target center coordinates, and the spray control module adjusts the opening combination and pressure of multiple nozzles based on the ellipse parameters to form a spray profile that matches the elliptical spray range.
[0015] On the other hand, this invention proposes a targeted spraying plant protection system utilizing drone visual recognition, comprising the following modules: The feature extraction module is used to acquire an image of the crop area to be processed, input the image into a deep learning backbone network to extract feature maps containing pest and disease information, and use the feature maps to generate a first set of candidate prediction boxes and a second set of candidate prediction boxes in parallel in the detection head. The prediction box calculation module is used to determine the disease feature salience of the first group of candidate prediction boxes based on the morphological irregularity, color gradient and texture entropy of the lesions, and assign a set of multiple positive sample prediction boxes representing the distribution pattern to each identified disease target based on the weighted matching cost of the disease feature salience. The matching module is used to calculate the target spatial clustering degree of each candidate box in a preset neighborhood for the second group of candidate prediction boxes; determine the clustering degree threshold based on the average saliency of disease features in the local area where the candidate box is located; when the target spatial clustering degree of the candidate box is higher than the clustering threshold, a matching prediction box for localization is determined for the same disease target based on the joint cost of classification confidence and normalized position deviation. The control module is used to take the center coordinates of the matching prediction box as the target center, calculate the covariance matrix of the coordinate set of the multiple positive sample prediction boxes, determine the major and minor axes and direction parameters of the elliptical spraying range covering the distribution of the positive sample prediction boxes based on the eigenvalues and eigenvectors of the covariance matrix, and control the spraying unit of the plant protection equipment to perform targeted spraying operations based on the target center, major and minor axes and direction parameters.
[0016] Optionally, the determination of disease feature salience based on lesion morphology irregularity, color gradient, and texture entropy is calculated using the following formula:
[0017] in, , , For preset weights, and ; The value is obtained by fitting the pixels of the lesion region to an circumscribed ellipse, and then subtracting the ratio of the lesion area to the area of the circumscribed ellipse from 1. The average gradient magnitude is obtained by applying the Sobel operator to the H and S channels in the HSV color space of the lesion area. The information entropy is obtained by converting the lesion area into a grayscale image, calculating the grayscale co-occurrence matrix of the grayscale image, and then calculating the information entropy based on the grayscale co-occurrence matrix.
[0018] Optionally, the calculation formula for the target spatial clustering degree of each candidate box within a preset neighborhood is as follows:
[0019] For each candidate box, a circular region with a radius of R and a center point of the candidate box is defined as a preset neighborhood; N is the number of center points of all other candidate boxes within the preset neighborhood, excluding the candidate box itself. The area is the neighborhood area.
[0020] Optionally, the clustering threshold is determined based on the mean saliency of disease features in the local area where the candidate box is located, and the calculation formula is as follows:
[0021] in, The value is the arithmetic mean of the disease characteristics significance of all identified lesions within the local area where the candidate box is located, and k is a preset proportional coefficient.
[0022] Optionally, the method of determining a matching prediction box for locating the same disease target based on the joint cost of classification confidence and normalized location bias includes: The formula for calculating the joint cost is as follows:
[0023] in, The classification confidence score for the candidate bounding boxes. The normalized Euclidean distance between the center point of the candidate box and the center point of its corresponding feature map grid cell is given by [reference to a specific location]. For coefficients; For each disease target, select the candidate box with the minimum joint cost C as its matching prediction box.
[0024] Optionally, determining the major and minor axes and direction parameters of the elliptical spraying range covering the positive sample prediction box distribution based on the eigenvalues and eigenvectors of the covariance matrix includes: Calculate the two eigenvalues of the covariance matrix and ,and Length of the major axis of the ellipse minor axis length ; Calculate and larger eigenvalues The corresponding feature vector is used to determine the angle between the feature vector and the principal axis of the coordinate system as the direction parameter of the elliptical spraying range.
[0025] Optionally, controlling the spraying unit of the plant protection equipment to perform targeted spraying operations based on the target center, major and minor axes, and direction parameters includes: The target center coordinates, the major and minor axes of the elliptical spraying range, and the direction parameters are encoded into a set of pulse width modulation control signals; The pulse width modulation (PWM) control signal is sent to the flight controller and spraying control module of the plant protection equipment. The flight controller adjusts the drone's hovering position based on the target center coordinates, and the spray control module adjusts the opening combination and pressure of multiple nozzles based on the ellipse parameters to form a spray profile that matches the elliptical spray range.
[0026] Compared to existing technologies, this invention decomposes the task of locating disease targets and representing their distribution by generating two sets of candidate prediction boxes with different functions in parallel within the detection head. By calculating the spatial clustering degree of the candidate boxes and combining classification confidence and positional bias, reliable prediction boxes can be matched to disease targets, thereby locking the spraying target and avoiding the positioning errors caused by the center offset of traditional rectangular boxes. Using the feature saliency determined by the morphology, color, and texture of the disease, a set of multiple prediction boxes that fully reflect its distribution pattern are assigned to the disease targets. Based on the covariance matrix of its coordinate set, an elliptical spraying range that closely fits the irregular contour of the disease is generated. This ensures that the spraying range matches the actual distribution of the disease, improving pesticide utilization and reducing accidental damage to healthy plants and environmental pollution. Attached Figure Description
[0027] Figure 1 A flowchart of the first embodiment; Figure 2 A schematic diagram for generating the first set of prediction boxes and the second set of prediction boxes; Figure 3 This is a schematic diagram of an elliptical spray area. Detailed Implementation
[0028] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with relevant laws, regulations, and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.
[0029] In the first embodiment, the present invention proposes a targeted spraying method for plant protection using drone visual recognition, such as... Figure 1 ,include: S1, acquire an image of the crop area to be processed, input the image into a deep learning backbone network to extract feature maps containing pest and disease information; use the feature maps to generate a first set of candidate prediction boxes and a second set of candidate prediction boxes in parallel in the detection head; A high-resolution visible light camera mounted on a drone or ground robot is used to capture images of the crop area to be processed at an angle perpendicular to the ground, obtaining digital images with a resolution of at least 1920×1080 pixels. The acquired images are then scaled to a preset size, such as 640×640 pixels, and normalized. The preprocessed images are then input into a deep convolutional neural network (CNN) pre-trained on a large image dataset, such as the ResNet-50 network, as the backbone network. The backbone network extracts semantic information from the image layer by layer through multiple layers of convolution, pooling, and activation functions, ranging from low-level texture and edges to high-level lesion shapes and color combinations. It outputs feature maps at multiple scales at different network depths, for example... , , Feature maps of the hierarchy.
[0030] Multi-scale feature maps output from the backbone network are input into the Feature Pyramid Network (FPN) for cross-level information fusion to enhance the perception of disease targets of different sizes. The fused feature maps are then simultaneously fed into two structurally identical but parameter-distributed detection heads. Each detection head employs an anchorless bounding box design, predicting the position coordinates, confidence score of containing the target, and probability of belonging to a specific disease category for each spatial candidate bounding box on the feature map through independent convolutional branches. The first detection head generates the first set of candidate predicted bounding boxes for subsequent morphological representation, while the second detection head generates the second set for subsequent localization. The generation of the two sets of predicted bounding boxes is processed in parallel, as follows: Figure 2 As shown.
[0031] S2, For the first group of candidate prediction boxes, the salience of disease features is determined based on the irregularity of the lesion shape, color gradient and texture entropy. Based on the weighted matching cost of the salience of disease features, a set of multiple positive sample prediction boxes representing the distribution pattern are assigned to each identified disease target. The saliency of each true disease-marked region in the image is calculated. In one embodiment, morphological irregularity is represented by the ratio of the square of the perimeter of the lesion region's outline to its area; a larger ratio indicates a more irregular shape. Color gradient is obtained by converting the lesion region to the CIELAB color space and calculating the average gradient magnitude of the L, a, and b channels; a larger gradient indicates a more drastic color change. Texture entropy is calculated by determining the texture complexity through the entropy of the gray-level co-occurrence matrix of the lesion region. The saliency of the disease features is obtained by normalizing and weighting the sum of these three indicators.
[0032] In an optional embodiment, the determination of disease feature salience based on lesion morphological irregularity, color gradient, and texture entropy is calculated using the following formula:
[0033] in, , , For preset weights, and ; The value is obtained by fitting the pixels of the lesion region to an circumscribed ellipse, and then subtracting the ratio of the lesion area to the area of the circumscribed ellipse from 1. The average gradient magnitude is obtained by applying the Sobel operator to the H and S channels in the HSV color space of the lesion area. The information entropy is obtained by converting the lesion area into a grayscale image, calculating the grayscale co-occurrence matrix of the grayscale image, and then calculating the information entropy based on the grayscale co-occurrence matrix.
[0034] For example, for a potato late blight lesion with a highly irregular shape, its pixel area is 950 units, while the area of its smallest bounding ellipse is 1500 units. Therefore, the irregularity of the shape... The calculated value is 0.37. Regarding color characteristics, a well-defined downy mildew lesion will exhibit dramatic color changes in the H and S channels of the HSV color space, as calculated using the Sobel operator. The mean value can reach 50.5. For anthrax lesions with complex textures, which exhibit concentric rings of varying depths, the texture entropy calculated from the gray-level co-occurrence matrix after conversion to grayscale is... It may be 6.2, which is higher than that of powdery mildew lesions with a single texture.
[0035] The significance S of the disease feature is obtained by weighted summation of the three feature values mentioned above. Assume that the weights of shape, color, and texture are set empirically. , , The values were 0.4, 0.3, and 0.3, respectively, and each eigenvalue was normalized. If the three normalized eigenvalues of the above late blight lesions are 0.8, 0.7, and 0.6, respectively, then the significance S is 0.71. Since the lesions exhibit obvious disease characteristics in morphology, color, and texture, they are a priority target for treatment.
[0036] During label assignment, for any box in the first set of candidate prediction boxes, the matching cost with a real disease target consists of classification loss and weighted localization loss. The weight of the localization loss is the feature saliency of the real disease target. For each real disease target, the top N candidate prediction boxes with the lowest matching costs, for example, the top 5, are selected as their positive sample set to collectively represent the overall distribution of the disease target.
[0037] S3, For the second group of candidate prediction boxes, calculate the target spatial clustering degree of each candidate box in the preset neighborhood; determine the clustering degree threshold based on the average saliency of the disease features in the local area where the candidate box is located; when the target spatial clustering degree of the candidate box is higher than the clustering degree threshold, determine a matching prediction box for the same disease target for localization based on the joint cost of classification confidence and normalized position deviation. For the candidate predicted bounding boxes in the second group, a fixed-size neighborhood is defined around the center point of each candidate predicted bounding box, such as a square region with a side length twice the length of its own diagonal. The number of other candidate bounding boxes within this neighborhood that are predicted to belong to the same disease category as the current candidate bounding box is counted; this number is the target spatial clustering degree. Simultaneously, the entire image is divided into multiple grids, and the average feature saliency of all real disease annotations within each grid is calculated. The clustering degree threshold is inversely proportional to this average saliency; for example, the threshold T = C / (1 + ... ), where C is a constant, This represents the average feature saliency of the grid containing the candidate box. When the spatial clustering of a candidate box exceeds the threshold corresponding to its region, the matching phase begins. In the matching phase, the joint cost between the candidate box and the actual disease target is calculated. The joint cost consists of the negative value of its classification confidence and the center point position deviation, where the position deviation is the Euclidean distance between the center of the predicted box and the center of the actual target. For each actual disease target, the candidate box with the minimum joint cost is selected as the matched predicted box.
[0038] To distinguish between clustered lesions and isolated artifacts, in an optional embodiment, the calculation of the target spatial clustering degree of each candidate box within a preset neighborhood is performed using the following formula:
[0039] For each candidate box, a circular region with a radius of R and a center point of the candidate box is defined as a preset neighborhood; N is the number of center points of all other candidate boxes within the preset neighborhood, excluding the candidate box itself. The area is the neighborhood area.
[0040] Assuming that when processing a high-resolution cucumber leaf image, the neighborhood radius R is set to 100 pixels, then the neighborhood area A is 31416 square pixels. For a candidate bounding box located in a concentrated downy mildew outbreak area, within a circular neighborhood of 100 pixels around its center point, the center points of 15 other candidate bounding boxes are detected, i.e., N=15. In this case, the target spatial clustering degree ρ of the candidate bounding box is approximately 0.000477.
[0041] In contrast, another isolated candidate box in the image might be caused by mud spots or shadows, and within its same neighborhood, there might be no other candidate boxes, i.e., N=0. Therefore, its target spatial clustering degree ρ is 0. By comparing these two ρ values, it can be determined that the former belongs to a high-incidence area of disease and has a higher probability of being a true positive, while the latter is likely to be random noise or interference and should be suppressed. The spatial density-based assessment method utilizes the biological characteristic that diseases are usually distributed in patches or clusters.
[0042] To select the optimal prediction box from multiple overlapping prediction boxes for the same lesion, in an optional embodiment, determining a matching prediction box for localization of the same disease target based on the joint cost of classification confidence and normalized location bias includes: The formula for calculating the joint cost is as follows:
[0043] in, The classification confidence score for the candidate bounding boxes. The normalized Euclidean distance between the center point of the candidate box and the center point of its corresponding feature map grid cell is given by [reference to a specific location]. For coefficients; For each disease target, select the candidate box with the minimum joint cost C as its matching prediction box.
[0044] Suppose that for a single black spot lesion, two candidate boxes, candidate box A and candidate box B, are generated. The balancing weight α is 0.5. What is the classification confidence score of candidate box A? The value is very high, at 0.98, but the deviation between its center point and the center point of the corresponding feature map grid cell is large, resulting in a calculated normalized positional deviation. The cost is 0.4. Therefore, the joint cost C of candidate box A is 0.21.
[0045] Another candidate box B, its classification confidence score Slightly lower at 0.92, but its position almost coincides with the center of the grid cell, normalized position deviation. The joint cost C of candidate box B is 0.05. Therefore, the joint cost C of candidate box B is 0.065. By comparison, the joint cost of candidate box B (0.065) is less than that of candidate box A (0.21), so candidate box B is selected as the location box.
[0046] To determine whether a candidate box belongs to a disease cluster based on the context of disease development, in an optional embodiment, the clustering threshold is determined based on the mean saliency of disease features in the local area where the candidate box is located. The calculation formula is as follows:
[0047] in, The value is the arithmetic mean of the disease characteristics significance of all identified lesions within the local area where the candidate box is located, and k is a preset proportional coefficient.
[0048] For example, in a localized area of a leaf, multiple mature lesions have been identified, and the average disease characteristic significance of the lesions is... The value is very high, for example, 0.9. Assuming the preset proportionality coefficient k is 0.01, then the aggregation threshold T here is approximately 0.0053. This is a relatively low threshold, meaning that even if a candidate box has only a few neighbors, as long as its aggregation degree ρ is greater than 0.0053, it is still considered part of the disease because its environment itself is already a high-risk area. Conversely, on another leaf that appears very healthy, the average significance of only a few suspicious points... Very low, for example, only 0.1. In this case, the threshold T is approximately 0.0091. A relatively high threshold requires the candidate box to exhibit strong spatial clustering characteristics, that is, its ρ value needs to be greater than 0.0091 to be confirmed as a disease.
[0049] S4, take the center coordinates of the matching prediction box as the target center, and calculate the covariance matrix of the coordinate set of the multiple positive sample prediction boxes. Determine the major and minor axes and direction parameters of the elliptical spraying range covering the distribution of the positive sample prediction boxes based on the eigenvalues and eigenvectors of the covariance matrix. Based on the target center, major and minor axes and direction parameters, control the spraying unit of the plant protection equipment to perform targeted spraying operations.
[0050] Specifically, the center point coordinates (x, y) of the matching prediction box determined in the previous step are used as the target of the spraying operation. Simultaneously, multiple positive sample prediction boxes assigned to the disease target are extracted, and the center point coordinates of all these prediction boxes are collected to form a two-dimensional point set. A 2x2 covariance matrix is calculated based on this point set. Eigenvalue decomposition is performed on the covariance matrix to obtain two eigenvalues and two corresponding eigenvectors. The larger eigenvalue defines the variance along the major axis of the ellipse, and the smaller eigenvalue defines the variance along the minor axis. The square roots of the two eigenvalues are taken and multiplied by a scaling factor, such as 2, to serve as the lengths of the semi-major and semi-minor axes of the ellipse's spraying range, respectively. Figure 3 As shown. The eigenvectors corresponding to the larger eigenvalues determine the direction of the major axis of the ellipse, i.e., the spraying direction parameter.
[0051] The calculated target center coordinates, the lengths of the major and minor axes of the ellipse, and the directional parameters are packaged into a single spraying command. This command is transmitted to the flight control system of the agricultural drone via a wireless communication module. Based on the received target center coordinates, the flight control system guides the drone to fly directly above the target disease area. According to the major and minor axes and directional parameters of the ellipse, the system adjusts the opening, closing, and spraying angles of multiple nozzles or variable-direction nozzles to create a spray coverage area on the ground that matches the calculated ellipse shape, size, and direction. This applies pesticides to the irregular disease distribution area represented by multiple positive sample prediction boxes.
[0052] To transform a discrete set of lesions into a continuous, regular operating area, in an optional embodiment, determining the major and minor axes and direction parameters of the elliptical spraying range covering the positive sample prediction box distribution based on the eigenvalues and eigenvectors of the covariance matrix includes: Calculate the two eigenvalues of the covariance matrix and ,and Length of the major axis of the ellipse minor axis length ; Calculate and larger eigenvalues The corresponding feature vector is used to determine the angle between the feature vector and the principal axis of the coordinate system as the direction parameter of the elliptical spraying range.
[0053] Suppose a series of rust lesions distributed along the veins are identified on a corn leaf, and the coordinates of the center points of these lesions form a two-dimensional point set. By calculating the covariance matrix of these point coordinates, two eigenvalues are obtained. It is 3600. It is 225. According to... > It can be seen that the lesions are distributed in a distinctly elongated shape. The major axis length 'a' of the elliptical sprayed area is 120 pixels; the minor axis length 'b' is 30 pixels. Simultaneously, the larger feature values are calculated. That is, the feature vector corresponding to 3600, for example, yields vectors of 0.8 and 0.6. This vector represents the most predominant direction of lesion distribution. By calculating the angle between this vector and the X-axis of the image coordinate system, a rotation angle can be obtained; for example, the arctangent function yields a result of 36.87 degrees. An elliptical region, 120 pixels long and 30 pixels short, tilted at an angle of 36.87 degrees, is determined, which can encompass all lesions.
[0054] To materialize the calculated virtual spraying area into an actual pesticide spraying action, in one optional embodiment, controlling the spraying unit of the plant protection equipment to perform targeted spraying operations based on the target center, major and minor axes, and direction parameters includes: The target center coordinates, the major and minor axes of the elliptical spraying range, and the direction parameters are encoded into a set of pulse width modulation control signals; The pulse width modulation (PWM) control signal is sent to the flight controller and spraying control module of the plant protection equipment. The flight controller adjusts the drone's hovering position based on the target center coordinates, and the spray control module adjusts the opening combination and pressure of multiple nozzles based on the ellipse parameters to form a spray profile that matches the elliptical spray range.
[0055] Taking the aforementioned ellipse with a length of 120cm, a length of 30cm, and an inclination of 36.87 degrees as an example, its target center coordinates are, for example, X=120.5 and Y=340.2. These parameters are converted into specific control commands. The target center coordinates 120.5 and 340.2 are encoded into a set of PWM signals and sent to the UAV's flight controller, instructing it to move and hover directly above the target.
[0056] Simultaneously, another set of PWM signals sends the major and minor axes and directional parameters of the ellipse to the spray control module. This module controls a spray boom consisting of five linearly arranged nozzles. To create a 120cm long spray pattern, the control module might instruct the three middle nozzles, namely nozzles 2, 3, and 4, to activate. To match the 30cm minor axis width, it applies a higher pressure value to nozzle 3, such as a 1800-microsecond PWM signal, to create a wider spray cone, while applying a lower pressure value to the two outer nozzles, such as a 1400-microsecond PWM signal, to create a narrower spray cone. The tilt angle of 36.87 degrees is achieved by fine-tuning the drone's yaw angle or activating the angled spray function. The coordinated action creates an elliptical spray coverage area on the ground that matches the calculated result.
[0057] In the second embodiment, the present invention proposes a targeted spraying plant protection system utilizing drone visual recognition, comprising the following modules: The feature extraction module is used to acquire an image of the crop area to be processed, input the image into a deep learning backbone network to extract feature maps containing pest and disease information, and use the feature maps to generate a first set of candidate prediction boxes and a second set of candidate prediction boxes in parallel in the detection head. The prediction box calculation module is used to determine the disease feature salience of the first group of candidate prediction boxes based on the morphological irregularity, color gradient and texture entropy of the lesions, and assign a set of multiple positive sample prediction boxes representing the distribution pattern to each identified disease target based on the weighted matching cost of the disease feature salience. The matching module is used to calculate the target spatial clustering degree of each candidate box in a preset neighborhood for the second group of candidate prediction boxes; determine the clustering degree threshold based on the average saliency of disease features in the local area where the candidate box is located; when the target spatial clustering degree of the candidate box is higher than the clustering threshold, a matching prediction box for localization is determined for the same disease target based on the joint cost of classification confidence and normalized position deviation. The control module is used to take the center coordinates of the matching prediction box as the target center, calculate the covariance matrix of the coordinate set of the multiple positive sample prediction boxes, determine the major and minor axes and direction parameters of the elliptical spraying range covering the distribution of the positive sample prediction boxes based on the eigenvalues and eigenvectors of the covariance matrix, and control the spraying unit of the plant protection equipment to perform targeted spraying operations based on the target center, major and minor axes and direction parameters.
[0058] In a more specific embodiment, the determination of disease feature salience based on lesion morphological irregularity, color gradient, and texture entropy is calculated using the following formula:
[0059] in, , , For preset weights, and ; The value is obtained by fitting the pixels of the lesion region to an circumscribed ellipse, and then subtracting the ratio of the lesion area to the area of the circumscribed ellipse from 1. The average gradient magnitude is obtained by applying the Sobel operator to the H and S channels in the HSV color space of the lesion area. The information entropy is obtained by converting the lesion area into a grayscale image, calculating the grayscale co-occurrence matrix of the grayscale image, and then calculating the information entropy based on the grayscale co-occurrence matrix.
[0060] In a more specific embodiment, the calculation formula for the target spatial clustering degree of each candidate box within a preset neighborhood is as follows:
[0061] For each candidate box, a circular region with a radius of R and a center point of the candidate box is defined as a preset neighborhood; N is the number of center points of all other candidate boxes within the preset neighborhood, excluding the candidate box itself. The area is the neighborhood area.
[0062] In a more specific embodiment, the clustering threshold is determined based on the mean saliency of disease features in the local area where the candidate box is located, and the calculation formula is as follows:
[0063] in, The value is the arithmetic mean of the disease characteristics significance of all identified lesions within the local area where the candidate box is located, and k is a preset proportional coefficient.
[0064] In a more specific embodiment, determining a matching prediction box for locating the same disease target based on the joint cost of classification confidence and normalized location bias includes: The formula for calculating the joint cost is as follows:
[0065] in, The classification confidence score for the candidate bounding boxes. The normalized Euclidean distance between the center point of the candidate box and the center point of its corresponding feature map grid cell is given by [reference to a specific location]. For coefficients; For each disease target, select the candidate box with the minimum joint cost C as its matching prediction box.
[0066] In a more specific embodiment, determining the major and minor axes and direction parameters of the elliptical spraying range covering the positive sample prediction box distribution based on the eigenvalues and eigenvectors of the covariance matrix includes: Calculate the two eigenvalues of the covariance matrix and ,and Length of the major axis of the ellipse minor axis length ; Calculate and larger eigenvalues The corresponding feature vector is used to determine the angle between the feature vector and the principal axis of the coordinate system as the direction parameter of the elliptical spraying range.
[0067] In a more specific embodiment, controlling the spraying unit of the plant protection equipment to perform targeted spraying operations based on the target center, major and minor axes, and direction parameters includes: The target center coordinates, the major and minor axes of the elliptical spraying range, and the direction parameters are encoded into a set of pulse width modulation control signals; The pulse width modulation (PWM) control signal is sent to the flight controller and spraying control module of the plant protection equipment. The flight controller adjusts the drone's hovering position based on the target center coordinates, and the spray control module adjusts the opening combination and pressure of multiple nozzles based on the ellipse parameters to form a spray profile that matches the elliptical spray range.
[0068] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (this), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0069] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. A targeted spraying plant protection method using unmanned aerial vehicle visual recognition, characterized in that, The method comprises the following steps: acquiring an image of a crop area to be processed, inputting the image into a deep learning backbone network to extract a feature map containing disease and pest information; generating a first group of candidate prediction boxes and a second group of candidate prediction boxes in parallel in a detection head using the feature map; determining a disease feature saliency according to the morphological irregularity, color gradient and texture entropy of a disease spot, assigning a plurality of positive sample prediction boxes representing the distribution form to each identified disease target based on the disease feature saliency weighted matching cost; calculating the target spatial aggregation degree of each candidate box in a preset neighborhood, determining an aggregation threshold according to the average disease feature saliency of the local area where the candidate box is located, and when the target spatial aggregation degree of the candidate box is higher than the aggregation threshold, determining a matching prediction box for positioning for the same disease target based on the joint cost of the classification confidence and the normalized position deviation; taking the center coordinates of the matching prediction box as a target center, calculating the covariance matrix of the coordinate set of the plurality of positive sample prediction boxes, determining the long and short axes and direction parameters of an elliptical spray range covering the distribution of the positive sample prediction boxes according to the eigenvalues and eigenvectors of the covariance matrix, and controlling the spray unit of the plant protection equipment to perform targeted spraying operation based on the target center, the long and short axes and the direction parameters.
2. The method of claim 1, wherein, The formula for determining the disease feature saliency according to the morphological irregularity, color gradient and texture entropy of the disease spot is: ; wherein, , , is a preset weight, and ; obtained by subtracting 1 from the ratio of the area of the lesion to the area of the circumscribed ellipse fitted to the pixels of the lesion area; The average of the calculated gradient amplitudes is obtained by applying a Sobel operator to the H and S channels in the HSV color space of the lesion area. By converting the lesion area into a gray image, a gray level co-occurrence matrix of the gray image is counted, and the information entropy is calculated based on the gray level co-occurrence matrix.
3. The method of claim 1, wherein, The formula for calculating the target spatial aggregation degree of each candidate box in a preset neighborhood is: ; wherein, for each candidate box, a circular region with the center point of the candidate box as the center and a radius R as the radius is defined as a preset neighborhood; N is the number of center points of all candidate boxes in the preset neighborhood except itself; is the neighborhood area.
4. The method of claim 1, wherein, The formula for determining the aggregation threshold according to the average disease feature saliency of the local area where the candidate box is located is: ; wherein, is the arithmetic mean of the disease feature saliencies of all identified disease spots in the local region where the bounding box is located, and k is a preset proportion coefficient.
5. The method of claim 1, wherein, The formula for determining a matching prediction box for positioning for the same disease target based on the joint cost of the classification confidence and the normalized position deviation is: The formula for calculating the joint cost is: ; wherein, is a classification confidence of a candidate box, is a normalized Euclidean distance between a center point of the candidate box and a center point of a feature map grid cell corresponding to the candidate box, is a coefficient; Select the candidate box with the minimum joint cost as the matching prediction box for each disease target.
6. The method of claim 1, wherein, The method for determining the long and short axes and direction parameters of an elliptical spray range covering the distribution of the positive sample prediction boxes according to the eigenvalues and eigenvectors of the covariance matrix comprises: two eigenvalues of the covariance matrix and , and , length of the major axis of the ellipse , length of the minor axis ; calculate the eigenvector corresponding to the larger eigenvalue , and determine the angle between the eigenvector and the principal axis of the coordinate system as the direction parameter of the elliptical spray range.
7. The method of claim 1, wherein, The method for controlling the spray unit of the plant protection equipment to perform targeted spraying operation based on the target center, the long and short axes and the direction parameters comprises: encoding the target center coordinates, the long and short axes and the direction parameters of the elliptical spray range into a group of pulse width modulation control signals; sending the pulse width modulation control (PWM) signals to the flight controller and the spray control module of the plant protection equipment; the flight controller adjusts the hovering position of the unmanned aerial vehicle according to the target center coordinates, and the spray control module adjusts the opening combination and pressure of the plurality of spray heads according to the elliptical parameters to form a spray profile matching the elliptical spray range.
8. A targeted spraying plant protection system using unmanned aerial vehicle visual recognition, characterized in that, The method comprises the following modules: a feature extraction module, configured to acquire an image of a crop area to be processed, input the image into a deep learning backbone network to extract a feature map containing disease and pest information, and generate a first group of candidate prediction boxes and a second group of candidate prediction boxes in parallel in a detection head using the feature map; The prediction box calculation module is configured to determine a disease feature saliency for the first set of candidate prediction boxes according to morphological irregularity, color gradient and texture entropy of the disease spot, and assign a set of positive sample prediction boxes representing distribution morphology to each identified disease target based on a weighted matching cost of the disease feature saliency. The matching module is configured to calculate a target spatial aggregation degree of each candidate box in a preset neighborhood for the second set of candidate prediction boxes, determine an aggregation threshold according to a mean value of disease feature saliency of a local region where the candidate box is located, and determine a matching prediction box for positioning for the same disease target based on a joint cost of classification confidence and normalized position deviation when the target spatial aggregation degree of the candidate box is higher than the aggregation threshold. The control module is configured to take a center coordinate of the matching prediction box as a target center, calculate a covariance matrix of a coordinate set of the plurality of positive sample prediction boxes, determine a long and short axis and a direction parameter of an elliptical spray range covering distribution of the positive sample prediction boxes according to eigenvalues and eigenvectors of the covariance matrix, and control a spraying unit of the plant protection equipment to perform a targeted spraying operation based on the target center, the long and short axis and the direction parameter.
9. The system of claim 8, wherein, The disease feature saliency is determined according to the morphological irregularity, the color gradient and the texture entropy of the disease spot, and a calculation formula is as follows: ; wherein, , , is a preset weight, and ; obtained by subtracting 1 from the ratio of the area of the lesion to the area of the circumscribed ellipse fitted to the pixels of the lesion area; The average of the calculated gradient magnitudes is obtained by applying a Sobel operator to the H and S channels in the HSV color space of the lesion area. By converting the lesion area into a gray image, a gray level co-occurrence matrix of the gray image is counted, and the information entropy is obtained based on the gray level co-occurrence matrix.
10. The system of claim 8, wherein, The target spatial aggregation degree of each candidate box in the preset neighborhood is calculated, and a calculation formula is as follows: ; wherein, for each candidate box, a circular region with the center point of the candidate box as the center and a radius R as the radius is defined as a preset neighborhood; N is the number of center points of all candidate boxes in the preset neighborhood except itself; is the neighborhood area.