Visual Monitoring-Based Method for Detecting the Spraying Effect of Agricultural Drones

Abstract

This invention relates to the field of agricultural drone spraying technology, specifically to a method for detecting the spraying effect of agricultural drones based on visual monitoring. The invention acquires canopy images during the spraying process of the agricultural drone, then analyzes the airflow resistance parameters of pixels based on texture and optical flow to determine gap regions; analyzes the pixel-canopy scale coefficient to determine the flow capacity parameters of each pixel, thereby obtaining the airflow impedance coefficient of the pixel; then determines the airflow impact center and obtains the cumulative impedance map of airflow propagation; further, based on the distance between pixels in the gap region and the airflow impact center, and the pixel-canopy scale coefficient, obtains the pesticide penetration index. This invention introduces the pixel-canopy scale coefficient to analyze the visual physical scale of canopy gaps and the airflow resistance, thereby assessing the physical flow capacity of canopy gaps and simulating airflow propagation to assess the difficulty of pesticide penetration through the canopy, thus quantitatively evaluating the spraying effect of the agricultural drone.

Description

Technical Field

[0001] This invention relates to the field of agricultural drone spraying technology, and specifically to a method for detecting the spraying effect of agricultural drones based on visual monitoring. Background Technology

[0002] When agricultural drones spray densely planted, hard-leaved crops such as citrus and grapefruit, the drone's flight control system maintains high rotor speeds to create a strong vertical downwash airflow in order to deliver the pesticide solution into the canopy. This causes the blades to tumble and create gaps, allowing the pesticide solution to seep through the canopy and improve spraying effectiveness. Monitoring the spraying effect of agricultural drones is a crucial step in ensuring the quality of agricultural operations, improving resource utilization efficiency, safeguarding ecological security, and achieving precision agriculture.

[0003] Current methods for detecting spraying effectiveness typically rely on visual monitoring of leaf movement amplitude or porosity. However, under sustained high wind pressure, stiff leaves are highly susceptible to aerodynamic closure, where the surface leaves are compacted and overlapped to form a dense, tile-like layer, creating a high-pressure air cushion on the canopy surface. This causes the airflow carrying the pesticide to slide laterally, preventing effective penetration into the inner canopy. Furthermore, in this aerodynamically closed state, the compacted leaves generate high-frequency surface flutter when disturbed by strong winds. Conventional algorithms struggle to distinguish between ineffective surface flutter and effective penetrating flipping, leading to false positives and impacting the accuracy of detecting agricultural drone spraying effectiveness. Summary of the Invention

[0004] To address the technical problem of low accuracy in detecting the spraying effect of agricultural drones, the present invention aims to provide a visual monitoring-based method for detecting the spraying effect of agricultural drones. The specific technical solution adopted is as follows:

[0005] Real-time acquisition of flight altitude, attitude data, and canopy images of agricultural drones during spraying;

[0006] The airflow obstruction parameters of each pixel are obtained based on the texture and optical flow information of the canopy image, and the gap region in the canopy image is determined based on the airflow obstruction parameters; the pixel-canopy scale coefficient of the canopy image is determined based on the flight altitude, and the flow capacity parameter of each pixel is determined by combining the positional distribution of the pixels in the gap region.

[0007] The airflow impedance coefficient of each pixel is obtained based on the airflow stagnation parameter and the flow capacity parameter. The airflow impact center of the canopy image is determined based on the attitude data. The airflow impact center is used as the propagation source point. The airflow propagation cumulative impedance map is obtained based on the path propagation algorithm. The airflow cumulative impedance coefficient of each pixel in the airflow propagation cumulative impedance map is obtained.

[0008] At each moment, the drug penetration index is obtained based on the distance between each pixel in the gap region of the canopy image and the airflow impact center, the airflow cumulative impedance coefficient, and the pixel-canopy scale coefficient.

[0009] Furthermore, the method for obtaining the airflow resistance parameters includes:

[0010] In the canopy image, the airflow occlusion probability of each pixel is obtained based on the negative correlation normalization result of the texture entropy within the preset window corresponding to each pixel; the airflow closure probability of each pixel is determined based on the divergence of the optical flow vector of the pixel.

[0011] By combining the airflow obstruction probability and the airflow closure probability, the airflow stagnation parameter of each pixel is determined.

[0012] Furthermore, the method for obtaining the gap region includes:

[0013] In the canopy image, the region corresponding to the pixel whose airflow obstruction parameter is less than the preset obstruction threshold is designated as the gap region.

[0014] Furthermore, the method for obtaining the pixel-canopy scale coefficient includes:

[0015] In the canopy image, based on the pinhole imaging principle, the pixel-canopy scale coefficient is obtained according to the flight altitude and focal length.

[0016] Furthermore, the method for obtaining the flow capacity parameter includes:

[0017] In the slit region, the distance between each pixel and the nearest slit region boundary is taken as the pixel slit half-width. The pixel slit half-width is weighted using the pixel-canopy scale coefficient to obtain the physical slit half-width of each pixel.

[0018] Based on the difference between the physical gap half-width and the preset effective gap threshold, the flow capacity parameter of each pixel is obtained; the flow capacity parameter of pixels in the non-gap region is set to zero.

[0019] Furthermore, the method for obtaining the airflow resistance coefficient includes:

[0020] In the canopy image, for each pixel, the negative correlation mapping result of the flow capacity parameter is fused with the airflow stagnation parameter to obtain the airflow impedance coefficient.

[0021] Furthermore, the method for obtaining the airflow impact center includes:

[0022] The rotor center axis of the agricultural drone is determined based on the attitude data, and the airflow impact center in the canopy image is calculated based on the rotor center axis.

[0023] Furthermore, the path propagation algorithm is a fast-moving algorithm; wherein, in the canopy image, the grid wavefront propagation speed of the fast-moving algorithm is a negative correlation mapping result of the airflow impedance coefficient of the pixel.

[0024] Furthermore, the method for obtaining the drug penetration index includes:

[0025] At each moment, in the slit region of the canopy image, the pixels with the smallest cumulative airflow impedance coefficients of the first preset number are selected as dominant flow pixels.

[0026] The distance between each dominant flow pixel and the airflow impact center is weighted using the pixel-canopy scale coefficient to obtain the airflow transmission distance of each dominant flow pixel.

[0027] Based on the cumulative impedance coefficient of the airflow and the airflow transmission distance of each dominant flow pixel, the liquid transport loss parameter of each dominant flow pixel is obtained;

[0028] By negatively mapping the mean values ​​of the drug transport loss parameters of all the advantageous flow pixels, the drug penetration index is obtained.

[0029] Furthermore, after obtaining the drug penetration index, the following is also included:

[0030] When the penetration index of the pesticide solution is greater than or equal to a preset index threshold, the agricultural drone is kept in its current flight state.

[0031] When the penetration index of the pesticide solution is less than a preset index threshold, a pulse unloading command is triggered on the agricultural drone. The pulse unloading command includes at least reducing the operating speed of the rotor to a preset unloading speed within a preset unloading time, maintaining the preset unloading speed for a preset unloading time, and then allowing the rotor speed to recover to the operating speed based on a preset linear slope.

[0032] The present invention has the following beneficial effects:

[0033] This invention acquires real-time flight altitude, attitude data, and canopy images of agricultural drones during spraying, providing a data foundation for subsequent real-time analysis of spraying effects. Then, based on the texture and optical flow information of the canopy images, it analyzes and obtains the airflow resistance parameters of each pixel from two levels: leaf gaps and leaf opening / closing motion. Based on these airflow resistance parameters, it determines the gap regions in the canopy images. Next, based on the flight altitude, it determines the pixel-to-canopy scale coefficient of the canopy images to establish the mapping relationship between image space and physical space. Combining the positional distribution of pixels in the gap regions, it analyzes the airflow within the gaps to determine the flow capacity parameters of each pixel. Further, it obtains the airflow impedance coefficient for each pixel. Then, based on the attitude data, it determines the airflow impact center of the canopy images. Using the airflow impact center as the propagation source point, it simulates the airflow propagation path on the canopy using a path propagation algorithm to obtain an airflow propagation cumulative impedance map and the airflow cumulative impedance coefficient of each pixel in the map. Finally, at each moment, based on the distance between each pixel in the gap regions of the canopy images and the airflow impact center, as well as the airflow cumulative impedance coefficient and the pixel-to-canopy scale coefficient, it obtains the pesticide penetration index, reflecting the spraying effect. This invention introduces a pixel-canopy scale coefficient to analyze the visual physical scale of canopy gaps and airflow obstruction, thereby assessing the physical flow capacity of canopy gaps and simulating airflow propagation to evaluate the difficulty of pesticide penetration through the canopy, so as to quantitatively evaluate the spraying effect of agricultural drones. Attached Figure Description

[0034] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart illustrating a method for detecting the spraying effect of agricultural drones based on visual monitoring, as provided in one embodiment of the present invention. Detailed Implementation

[0036] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a visual monitoring-based agricultural drone spraying effect detection method proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0038] The following description, in conjunction with the accompanying drawings, details a specific scheme for a visual monitoring-based method for detecting the spraying effect of agricultural drones provided by this invention.

[0039] Please see Figure 1 The diagram illustrates a flowchart of a visual monitoring-based method for detecting the spraying effect of agricultural drones, according to an embodiment of the present invention, specifically including:

[0040] Step S1: Real-time acquisition of flight altitude, attitude data, and canopy images of the agricultural drone during the spraying process.

[0041] The present invention targets the implementation scenario of multi-rotor agricultural drones flying at a preset operating height (e.g., 1.5m-3m) above the crop canopy to perform spraying (pesticide application) operations. During the spraying process, the penetration index of pesticide application is evaluated (spraying effect detection) by collecting and analyzing the gaps in the airflow in the canopy images.

[0042] The multi-rotor agricultural drone is equipped with radar to measure its height relative to the canopy (i.e., flight altitude) in real time; it is also equipped with an inertial data acquisition unit to collect the drone's attitude data in real time to determine its flight attitude such as pitch, roll, or tilt; and it is also equipped with a visual sensor (camera) to collect canopy images in real time.

[0043] In one embodiment of the present invention, the above-mentioned airborne acquisition device (radar, inertial acquisition unit and visual sensor) is first used to collect the flight altitude, attitude data and canopy image of the plant protection drone in real time during the spraying operation. In this embodiment, the optical axis of the visual sensor is parallel to the rotor centerline, that is, the acquisition angle of the canopy image will be deflected as the attitude of the plant protection drone changes.

[0044] In this embodiment, the airborne data acquisition device acquires data synchronously at a fixed frequency (e.g., once every 10 seconds). The flight altitude refers to the height of the agricultural drone relative to the canopy. The attitude data includes the quaternion of the aircraft attitude. The canopy image corresponds to the flight altitude and attitude data at the same time. In other embodiments, the implementer may also adjust the acquisition frequency according to the actual application.

[0045] It should be noted that, in one embodiment of the present invention, the acquisition frequency is the detection frequency of the spraying effect of the agricultural drone.

[0046] Step S2: Obtain the airflow obstruction parameter of each pixel based on the texture information and optical flow information of the canopy image, and determine the gap region in the canopy image based on the airflow obstruction parameter; determine the pixel-canopy scale coefficient of the canopy image based on the flight altitude, and determine the flow capacity parameter of each pixel in combination with the position distribution of the pixels in the gap region.

[0047] It should be noted that during the spraying process of agricultural drones, the analysis method for evaluating the penetration index of pesticide based on the canopy image (to detect the spraying effect) is consistent at each acquisition time. Here, we will only take any acquisition time as an example for analysis and description, and will not go into detail again. In particular, since the optical flow information of the pixel cannot be determined at the first acquisition time, the evaluation of the spraying effect will start from the second acquisition time.

[0048] When hard-leaved, densely planted crops undergo aerodynamic closure under continuous high-intensity rotor wind pressure, the compacted leaf surface exhibits a highly uniform smoothness or specular reflection characteristic, while effective airflow channels (canopy gaps) typically appear as rough edges or dark shadows. Furthermore, when airflow can effectively penetrate, the blades open radially around the airflow center; when airflow is obstructed, the blades are compacted and fixed by vertical wind pressure, exhibiting a static or slightly high-frequency fluttering state. The essence of optical flow is to capture the minute displacement field of canopy image pixels over time, thereby helping to assess the driving effect of the wind field on the blades.

[0049] Based on this, the embodiments of the present invention obtain the airflow stagnation parameters of each pixel according to the texture information and optical flow information of the canopy image; the airflow stagnation parameters analyze the static stagnation distribution from the orderliness of the local texture and analyze the dynamic stagnation distribution from the motion posture of the blade under the action of the wind field, thereby comprehensively evaluating the canopy pore opening status and providing a basis for subsequent determination of the slit area.

[0050] Preferably, in one embodiment of the present invention, considering that texture entropy can reflect the degree of disorder in image information, the surface texture of the leaf compacted by airflow is simple, exhibiting a low entropy value; while the texture of the gaps where branches and leaves intersect is complex, exhibiting a high entropy value; the negative correlation normalization result of texture entropy can help evaluate the degree of leaf compaction at each pixel, and thus help evaluate the probability of airflow obstruction; considering that the divergence of the optical flow vector can characterize the flux source and sink characteristics of the leaf motion field, positive divergence represents outward diffusion of pixels (leaf flips open), negative divergence or zero divergence represents inward compression or stillness of pixels (leaf is compacted); then divergence can help evaluate the opening and closing of the leaf at each pixel, and thus help evaluate the probability of airflow closure; then comprehensively evaluate the airflow obstruction parameter of each pixel; therefore, the method for obtaining the airflow obstruction parameter includes:

[0051] In the canopy image, the airflow occlusion probability of each pixel is obtained based on the negative correlation normalization result of the texture entropy within the preset window corresponding to each pixel; the airflow closure probability of each pixel is determined based on the divergence of the optical flow vector of the pixel; and the airflow obstruction parameter of each pixel is determined by combining the airflow occlusion probability and the airflow closure probability.

[0052] As an example, in a canopy image, taking any pixel as an example, a preset window of a preset size is constructed with that pixel as the center; in this example, the preset size is 5×5, but the implementer can adjust it as needed; for pixels whose image edges are smaller than the preset window size, mirror filling or ignoring is used; the texture entropy of the pixels within the preset window is calculated, wherein the method for calculating the texture entropy includes: counting the frequency of gray values ​​of all pixels within the preset window, normalizing them to a probability distribution, and then applying the entropy calculation formula to calculate the entropy value, i.e., the texture entropy; in other embodiments, the implementer can also fuse the gradient entropy to calculate the texture entropy within the preset window, which will not be elaborated further.

[0053] Then, an entropy threshold is set, and the difference between the texture entropy and the entropy threshold is calculated and mapped to a variant function of the sigmoid activation function. In this method, the texture entropy is negatively correlated and normalized, and the value of the variant function is used as the probability of airflow occlusion of the pixel; implementers may also use other negatively correlated normalization methods; where e is the natural constant; This is the first sensitivity coefficient, with a value ranging from 5 to 10. In this example, it is set to 10, which is used to adjust the steepness of the variant function. For pixels Corresponding to the texture entropy within the preset window; The entropy threshold is an empirical threshold used to distinguish between smooth blades and complex gaps. In this example, the median of the texture entropy within a preset window corresponding to all pixels in the canopy image is used as the entropy threshold.

[0054] Then, dense optical flow algorithms such as Farneback are used to calculate the optical flow vector of each pixel in the canopy image (before the first acquisition time), and the divergence of the optical flow vector of each pixel is calculated (a well-known technique, which will not be elaborated here). A divergence threshold is set to filter out optical flow noise and invalid micro-motions, thereby improving the effectiveness of airflow opening and closing assessment. In this example, the divergence threshold is set to 0.05. When the divergence of the optical flow vector of a pixel is greater than the divergence threshold, the probability of the blades at the pixel flipping open is greater, and the airflow closing probability is set to 0. When the divergence of the optical flow vector of a pixel is less than or equal to the divergence threshold, the probability of the blades at the pixel being compacted or having noise or invalid micro-motions is greater, and the airflow closing probability is set to 1.

[0055] In other examples, implementers can also directly map the divergence threshold to the sigmoid function to obtain the airflow closure probability.

[0056] Furthermore, the maximum value between the airflow obstruction probability and the airflow closure probability is used as the airflow stagnation parameter for determining the pixel, wherein the value range of the airflow stagnation parameter is 0-1; in other embodiments, the implementer may also use the weighted fusion result or the mean of the airflow obstruction probability and the airflow closure probability as the airflow stagnation parameter for the pixel.

[0057] After obtaining the airflow obstruction parameters at each pixel, the gap regions in the canopy image can be further determined. The gap regions are potential canopy gaps identified at the visual level, providing a preliminary basis for subsequent evaluation of the spraying effect.

[0058] Preferably, in one embodiment of the present invention, the method for obtaining the gap region includes: in the canopy image, identifying the region corresponding to pixels whose airflow obstruction parameter is less than a preset obstruction threshold as the gap region. The preset obstruction threshold ranges from 0.6 to 0.8, and in this example, it is set to 0.6.

[0059] It should be noted that when the total area of ​​the gap region is less than 1% of the area of ​​the canopy image, the canopy is determined to be in a completely closed state, and the drug penetration index calculated in the subsequent step S4 is directly set to 0, and no further analysis is performed.

[0060] During drone flight operations, terrain undulations cause real-time changes in the drone's flight altitude relative to the canopy surface. These changes in flight altitude lead to fluctuations in imaging magnification, resulting in potentially drastically different physical dimensions of the canopy region corresponding to a single pixel in different canopy images. To standardize subsequent analysis and measurement criteria, this embodiment of the invention calculates the ratio between pixels and physical space based on flight altitude, determining the pixel-canopy scale coefficient of the canopy image. The pixel-canopy scale coefficient quantifies the ratio between pixels and the physical space of the canopy.

[0061] Preferably, in one embodiment of the present invention, considering that based on the pinhole imaging principle, the image pixel coordinate system can be mapped to the physical space coordinate system, thereby helping to evaluate the physical size of the canopy corresponding to the pixel; then the method for obtaining the pixel-canopy scale coefficient includes:

[0062] In canopy images, based on the pinhole imaging principle, the pixel-canopy scale coefficient is obtained according to the flight altitude, focal length, and pixel size.

[0063] As an example, based on the principle of pinhole imaging: Then the pixel-canopy scale factor , where p is the pixel size, in meters per pixel, which needs to be determined based on the hardware parameters of the vision sensor; the pixel-canopy scale coefficient is in meters per pixel and is used to characterize the physical size of the canopy corresponding to a pixel.

[0064] By obtaining the pixel-to-canopy scale coefficient of the canopy image, and further combining the positional distribution of pixels in the crevice region, we can assess the physical flow capacity in the crevice region, determine the flow capacity parameter of each pixel, and the flow capacity parameter initially reflects the possibility of the liquid seeping into the interior of the canopy, thus preparing for subsequent analysis of the spraying effect.

[0065] Preferably, in one embodiment of the present invention, based on the wall adhesion effect (also known as the Coanda effect) in fluid mechanics, it is known that visually connected gap regions in canopy images may not necessarily have physical flow capacity. That is, due to the viscosity of air fluid, when the physical width of the gap is too narrow, the flow velocity in the gap channel will drop sharply or even stagnate. Although the gap exists visually, the liquid cannot be carried through by the airflow. Therefore, the physical half-width distance of each pixel in the gap region relative to the gap boundary can be calculated first by combining the pixel-canopy scale coefficient, and then the flow capacity parameter at the pixel can be evaluated. Pixels in non-gap regions can be directly regarded as having no flow capacity. Therefore, the method for obtaining the flow capacity parameter includes:

[0066] In the slit region, the distance between each pixel and the nearest slit region boundary is taken as the pixel slit half-width. The pixel slit half-width is weighted by the pixel-canopy scale coefficient to obtain the physical slit half-width of each pixel.

[0067] Based on the difference between the physical gap half-width and the preset effective gap threshold, the flow capacity parameter of each pixel is obtained; the flow capacity parameter of pixels in the non-gap region is set to zero.

[0068] It should be noted that there may be multiple gap regions in the canopy image. The embodiments of the present invention take any pixel point in any gap region as an example for analysis and description.

[0069] As an example, firstly, in the canopy image, the pixel distance between a pixel in the slit region and its nearest slit region boundary, i.e., the pixel slit half-width, is calculated. Then, the pixel-canopy scale coefficient is multiplied by the pixel slit half-width, and the product is used as the physical distance of each pixel from the nearest slit boundary in the real canopy physical space, i.e., the physical slit half-width.

[0070] A minimum effective gap threshold is further defined, which characterizes the minimum physical width through which viscous airflow can carry droplets; in this example, the minimum effective gap threshold is set to 0.02m. The difference between the physical gap half-width and the minimum effective gap threshold is calculated, and this difference is mapped to a variant function of the sigmoid activation function. In this context, the value of the variant function is used as the flow capability parameter of the pixel; where e is the natural constant. This is the second sensitivity coefficient, set to 100 in this example, used to adjust the steepness of the variant function; The physical gap half-width of a pixel; The minimum effective gap threshold;

[0071] When the physical gap half-width of a pixel is significantly greater than the minimum effective gap threshold, the flow capacity parameter approaches 1, indicating high flow capacity; when the physical gap half-width of a pixel is less than or equal to the minimum effective gap threshold, the flow capacity parameter approaches 0, indicating low flow capacity; for pixels in non-gap regions, the flow capacity parameter is directly set to 0.

[0072] Step S3: Obtain the airflow impedance coefficient of each pixel based on the airflow stagnation parameter and the flow capacity parameter. Determine the airflow impact center of the canopy image based on the attitude data. Using the airflow impact center as the propagation source point, obtain the airflow propagation cumulative impedance map based on the path propagation algorithm, and obtain the airflow cumulative impedance coefficient of each pixel in the airflow propagation cumulative impedance map.

[0073] Since the airflow obstruction parameter assesses the degree of obstruction of airflow by the gap from a visual perspective, while the flow capacity parameter assesses the degree of obstruction of airflow by the physical size of the gap from a fluid dynamics perspective, both can help evaluate the airflow obstruction at the pixel. Therefore, this embodiment of the invention further obtains the airflow impedance coefficient of each pixel based on the airflow obstruction parameter and the flow capacity parameter. The airflow impedance coefficient comprehensively quantifies the airflow impedance at the pixel, helping to evaluate the resistance effect of airflow carrying pesticide into the canopy, and preparing for subsequent analysis of pesticide penetration index to evaluate the spraying effect of agricultural drones.

[0074] Preferably, in one embodiment of the present invention, the method for obtaining the airflow resistance coefficient includes:

[0075] In the canopy image, for each pixel, the negative correlation mapping result of the flow capacity parameter is fused with the airflow stagnation parameter to obtain the airflow impedance coefficient.

[0076] As an example, the flow capacity parameter is mapped to an exponential function exp(-x) with the natural constant e as the base, where x is the independent variable. The negative correlation mapping result is then multiplied with the airflow resistance parameter, and the product is used as the airflow resistance coefficient.

[0077] In other examples, implementers can also achieve negative correlation mapping by adding a very small positive parameter, such as 0.01, and then taking the reciprocal; they can also use a weighted summation method for fusion, which will not be elaborated further; implementers can also take the maximum value of the negative correlation mapping result of the flow capacity parameter and the airflow stagnation parameter as the airflow impedance coefficient.

[0078] Since the viewing angle of the canopy image will deflect with the attitude change of the agricultural drone, the projection position of the central axis of the downwash airflow generated by the rotor on the image plane will move in real time with the attitude, rather than being fixed at the center of the image. In order to facilitate the subsequent analysis of the airflow propagation, this embodiment of the invention further determines the airflow impact center of the canopy image based on the attitude data. The airflow impact center is the airflow source point in the downwash wind field generated by the rotor, which prepares for the subsequent simulation analysis of the airflow transmission network.

[0079] Preferably, in one embodiment of the present invention, considering that the wind field is generated by the high-speed rotation of the rotor of the agricultural drone, and that attitude data can help assess the current flight attitude of the agricultural drone, thereby determining whether it is flying horizontally or tilted, and thus helping to determine the rotor center axis of the fuselage; the intersection of the rotor center axis of the fuselage with the line and plane on the canopy can be regarded as the airflow impact center on the canopy; therefore, the method for obtaining the airflow impact center includes:

[0080] The rotor center axis of the agricultural drone is determined based on attitude data, and the airflow impact center in the canopy image is calculated based on the rotor center axis.

[0081] As an example, firstly, a body space coordinate system is constructed, and the attitude data of the agricultural drone is used to calculate its pose state in the body space coordinate system, thereby determining the rotor central axis of the agricultural drone (a well-known technical means, which will not be elaborated here). The intersection of the rotor central axis and the canopy is the airflow impact center in physical space. Next, an image coordinate system is constructed, and then the airflow impact center in physical space is projected into the image coordinate system using the pinhole imaging principle (a well-known technical means, which will not be elaborated here), to obtain the airflow impact center in the canopy image.

[0082] After determining the airflow impact center in the canopy image, the canopy airflow transmission model can be further constructed based on the path propagation algorithm, using the airflow impact center as the propagation source point. This allows for the acquisition of the airflow propagation cumulative impedance map and the acquisition of the airflow cumulative impedance coefficient for each pixel in the airflow propagation cumulative impedance map. The airflow cumulative impedance coefficient characterizes the airflow propagation hindrance at each pixel, preparing for subsequent evaluation of the penetration of drug-carrying airflow into the canopy.

[0083] Since the process of airflow propagating laterally on the canopy surface to find low-impedance gap entrances is similar to wavefront propagation, and the Fast Marching Method (FMM) is a standard method for numerically solving the equations of motion, it can simulate the physical process of wave propagation in a medium with varying refractive index (i.e., airflow in a medium with varying impedance). By calculating the minimum cumulative impedance from the airflow impact center to each pixel in the canopy image, it conforms to the physical intuition that airflow always tends to spread along the path of least resistance.

[0084] Based on this, in a preferred embodiment of the present invention, the path propagation algorithm used is the fast travel algorithm; wherein, the grid wavefront propagation speed of the fast travel algorithm is the negative correlation mapping result of the airflow impedance coefficient of the pixel.

[0085] It should be noted that the application of the fast-moving algorithm is a well-known technique and will not be elaborated further; the negative correlation mapping method used is to map to the exponential function exp(-x) with the natural constant e as the base, but other negative correlation mapping methods can also be used.

[0086] After obtaining the cumulative impedance map of airflow propagation corresponding to the canopy image using the fast travel algorithm, the cumulative impedance map of airflow propagation is the same size as the canopy image, but the pixel value of each pixel in the cumulative impedance map of airflow propagation is the cumulative impedance coefficient of airflow, that is, the total airflow propagation resistance overcome from the airflow impact center along the optimal path to reach that pixel (i.e., the cumulative impedance coefficient of airflow, the sum of the airflow impedance coefficients).

[0087] Step S4: At each time step, based on the distance between each pixel in the gap region of the canopy image and the airflow impact center, as well as the airflow cumulative impedance coefficient, and combined with the pixel-canopy scale coefficient, the drug penetration index is obtained.

[0088] Since changes in the drone's operating altitude alter the absolute path length from the source point to the gap, thus affecting the absolute value of the cumulative airflow impedance coefficient, the absolute path length from the source point to the gap in the physical space of the canopy can be assessed first based on the distance between each pixel in the gap region of the canopy image and the airflow impact center, as well as the pixel-to-canopy scale coefficient. Since airflow typically tends to diffuse along the path of least resistance, the unit transmission loss can be assessed by combining the absolute path length, thereby eliminating the interference of path length and more accurately measuring whether the canopy itself is compacted and closed, making it difficult for the drug to penetrate the canopy.

[0089] Based on this, the embodiments of the present invention further obtain the liquid penetration index by combining the distance between each pixel in the gap region of the canopy image and the airflow impact center, the cumulative airflow impedance coefficient, and the pixel-canopy scale coefficient; the liquid penetration index helps to quantify the spraying effect of the liquid.

[0090] Preferably, in one embodiment of the present invention, considering that when the airflow center propagates along the path of minimum airflow impedance, there is still a large cumulative airflow impedance, it indicates that although there are gaps in the canopy, the efficiency of airflow permeation is low, and it is still difficult to carry the drug solution through; therefore, in order to reduce the amount of analysis and calculation, a preset number of dominant flow pixels are first selected in the gap region of the canopy image, and then the unit transmission loss of the airflow is evaluated by combining its corresponding absolute path length to accurately assess the drug solution penetration; therefore, the method for obtaining the drug solution penetration index includes:

[0091] At each moment, in the gap region of the canopy image, the pixels with the smallest cumulative airflow impedance coefficient of the first preset number are selected as the dominant flow pixels; the distance between each dominant flow pixel and the airflow impact center is weighted by the pixel-canopy scale coefficient to obtain the airflow transmission distance of each dominant flow pixel;

[0092] Based on the cumulative impedance coefficient of airflow and the airflow transmission distance of each dominant flow pixel, the liquid transport loss parameter of each dominant flow pixel is obtained; the mean of the liquid transport loss parameters of all dominant flow pixels is negatively correlated to obtain the liquid penetration index.

[0093] As an example, the preset number is set to 5% of the total number of pixels in the slit region of the canopy image, which can be adjusted by the implementer. The pixels are sorted in ascending order based on the cumulative airflow impedance coefficient, and the pixels with the smallest cumulative airflow impedance coefficient in the top 5% of the sorted pixels are taken as dominant flow pixels. Dominant flow pixels are areas that are easily reached by airflow and can carry the liquid through, excluding the airflow impact center.

[0094] Then, in the image coordinate system corresponding to the canopy image, calculate the Euclidean distance between each dominant flow pixel and the coordinates corresponding to the airflow impact center. Multiply the Euclidean distance by the pixel-canopy scale coefficient to obtain the airflow transmission distance, which reflects the absolute length of the airflow propagation path in the physical space of the canopy.

[0095] Then, the cumulative impedance coefficient of airflow at each dominant flow pixel is divided by the airflow transmission distance to obtain the drug transport loss parameter for each dominant flow pixel. The drug transport loss parameter, stripped of the interference of path length, characterizes the unit transport loss in the physical space of the canopy, and more accurately measures the resistance or airflow loss of drug penetration through the canopy. Then, the drug transport loss parameters of all dominant flow pixels are averaged, and the mean is negatively correlated and mapped to the exponential function exp(-x) with the natural constant e as the base to obtain the drug penetration index. Implementers can also use other negative correlation mapping methods.

[0096] It should be noted that the pesticide penetration index is an indicator parameter used to quantify the spraying effect of agricultural drones; the value range of the pesticide penetration index is 0-1; the higher the pesticide penetration index, the better the pesticide spraying effect of the agricultural drone at that time.

[0097] Considering that a low pesticide penetration index indicates that the wind field generated by the rotor of the current agricultural drone may cause aerodynamic closure of the canopy leaves, resulting in leaf compaction and preventing the pesticide from effectively penetrating the canopy; to further improve the spraying effect, in a preferred embodiment of the present invention, after obtaining the pesticide penetration index, the following is also included:

[0098] When the pesticide penetration index is greater than or equal to the preset index threshold, the agricultural drone is instructed to maintain its current flight status.

[0099] When the pesticide penetration index is less than the preset index threshold, the agricultural drone's pulse unloading command is triggered. The pulse unloading command includes at least reducing the rotor's operating speed to the preset unloading speed within the preset unloading time, maintaining the preset unloading speed for the preset unloading time, and then allowing the rotor speed to recover to the operating speed based on the preset linear slope.

[0100] As an example, the preset index threshold is 0.25, but implementers can adjust it themselves.

[0101] When the penetration index of the pesticide solution is greater than or equal to 0.25, the agricultural drone is kept in its current flight state, that is, the rotor speed is not changed, and the pesticide solution is sprayed.

[0102] When the pesticide penetration index is less than 0.25, the pulse unloading command of the agricultural drone is triggered: the rotor speed of the agricultural drone is reduced to the preset unloading speed within a preset unloading time, such as 0.5s (the preset unloading speed is 60% to 70% of the minimum throttle required to maintain the hovering attitude of the agricultural drone, which aims to eliminate vertical dynamic pressure to the greatest extent while ensuring flight safety, thereby breaking the aerodynamic compaction steady state of the blades). The preset unloading speed is maintained for a preset unloading time, such as 1.5s-2s, and then the rotor speed is restored to the operating speed based on a preset linear slope (such as increasing the throttle by 10% per second or restoring the operating speed by 10%). As the speed increases linearly, the downwash airflow gradually penetrates into the canopy cavity along the new path opened by the blade rebound. The dynamic pressure of the airflow itself plays the role of supporting the channel and preventing the blades from closing again.

[0103] In summary, this invention acquires the flight altitude, attitude data, and canopy image of the agricultural drone during spraying in real time; obtains the airflow resistance parameter of each pixel based on the texture and optical flow information of the canopy image, and determines the gap region in the canopy image based on the airflow resistance parameter; determines the pixel-canopy scale coefficient of the canopy image based on the flight altitude, and determines the flow capacity parameter of each pixel based on the positional distribution of pixels in the gap region; obtains the airflow impedance coefficient of each pixel based on the airflow resistance parameter and flow capacity parameter; determines the airflow impact center of the canopy image based on the attitude data; uses the airflow impact center as the propagation source point; obtains the airflow propagation cumulative impedance map based on the path propagation algorithm; and obtains the airflow cumulative impedance coefficient of each pixel in the airflow propagation cumulative impedance map; at each time step, obtains the pesticide penetration index based on the distance between each pixel in the gap region of the canopy image and the airflow impact center, the airflow cumulative impedance coefficient, and the pixel-canopy scale coefficient. This invention introduces a pixel-canopy scale coefficient to analyze the visual physical scale of canopy gaps and airflow obstruction, thereby assessing the physical flow capacity of canopy gaps and simulating airflow propagation to evaluate the difficulty of pesticide penetration through the canopy, so as to quantitatively evaluate the spraying effect of agricultural drones.

[0104] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0105] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for detecting the spraying effect of agricultural drones based on visual monitoring, characterized in that, The method includes: Real-time acquisition of flight altitude, attitude data, and canopy images of agricultural drones during spraying; The airflow obstruction parameters of each pixel are obtained based on the texture and optical flow information of the canopy image, and the gap region in the canopy image is determined based on the airflow obstruction parameters; based on the pinhole imaging principle, the pixel-canopy scale coefficient of the canopy image is determined according to the flight altitude, focal length and pixel size, and the flow capacity parameter of each pixel is determined in combination with the position distribution of the pixels in the gap region. The airflow impedance coefficient of each pixel is obtained based on the airflow stagnation parameter and the flow capacity parameter. The airflow impact center of the canopy image is determined based on the attitude data. The airflow impact center is used as the propagation source point. The airflow propagation cumulative impedance map is obtained based on the path propagation algorithm. The airflow cumulative impedance coefficient of each pixel in the airflow propagation cumulative impedance map is obtained. At each moment, the drug penetration index is obtained based on the distance between each pixel in the gap region of the canopy image and the airflow impact center, the airflow cumulative impedance coefficient, and the pixel-canopy scale coefficient.

2. The method for detecting the spraying effect of agricultural drones based on visual monitoring according to claim 1, characterized in that, The method for obtaining the airflow resistance parameters includes: In the canopy image, the airflow occlusion probability of each pixel is obtained based on the negative correlation normalization result of the texture entropy within the preset window corresponding to each pixel; the airflow closure probability of each pixel is determined based on the divergence of the optical flow vector of the pixel. By combining the airflow obstruction probability and the airflow closure probability, the airflow stagnation parameter of each pixel is determined.

3. The method for detecting the spraying effect of agricultural drones based on visual monitoring according to claim 1, characterized in that, The method for obtaining the gap region includes: In the canopy image, the region corresponding to the pixel whose airflow obstruction parameter is less than the preset obstruction threshold is designated as the gap region.

4. The method for detecting the spraying effect of agricultural drones based on visual monitoring according to claim 1, characterized in that, The method for obtaining the flow capacity parameter includes: In the slit region, the distance between each pixel and the nearest slit region boundary is taken as the pixel slit half-width. The pixel slit half-width is weighted using the pixel-canopy scale coefficient to obtain the physical slit half-width of each pixel. Based on the difference between the physical gap half-width and the preset effective gap threshold, the flow capacity parameter of each pixel is obtained; the flow capacity parameter of pixels in the non-gap region is set to zero.

5. The method for detecting the spraying effect of agricultural drones based on visual monitoring according to claim 1, characterized in that, The method for obtaining the airflow resistance coefficient includes: In the canopy image, for each pixel, the negative correlation mapping result of the flow capacity parameter is fused with the airflow stagnation parameter to obtain the airflow impedance coefficient.

6. The method for detecting the spraying effect of agricultural drones based on visual monitoring according to claim 1, characterized in that, The method for obtaining the airflow impact center includes: The rotor center axis of the agricultural drone is determined based on the attitude data, and the airflow impact center in the canopy image is calculated based on the rotor center axis.

7. The method for detecting the spraying effect of agricultural drones based on visual monitoring according to claim 1, characterized in that, The path propagation algorithm is a fast-moving algorithm; wherein, in the canopy image, the grid wavefront propagation speed of the fast-moving algorithm is a negative correlation mapping result of the airflow impedance coefficient of the pixel.

8. The method for detecting the spraying effect of agricultural drones based on visual monitoring according to claim 1, characterized in that, The method for obtaining the drug penetration index includes: At each moment, in the slit region of the canopy image, the pixels with the smallest cumulative airflow impedance coefficients of the first preset number are selected as dominant flow pixels. The distance between each dominant flow pixel and the airflow impact center is weighted using the pixel-canopy scale coefficient to obtain the airflow transmission distance of each dominant flow pixel. Based on the cumulative impedance coefficient of the airflow and the airflow transmission distance of each dominant flow pixel, the liquid transport loss parameter of each dominant flow pixel is obtained; By negatively mapping the mean values ​​of the drug transport loss parameters of all the advantageous flow pixels, the drug penetration index is obtained.

9. The method for detecting the spraying effect of agricultural drones based on visual monitoring according to claim 1, characterized in that, After obtaining the drug penetration index, the following is also included: When the penetration index of the pesticide solution is greater than or equal to a preset index threshold, the agricultural drone is kept in its current flight state. When the penetration index of the pesticide solution is less than a preset index threshold, a pulse unloading command is triggered on the agricultural drone. The pulse unloading command includes at least reducing the operating speed of the rotor to a preset unloading speed within a preset unloading time, maintaining the preset unloading speed for a preset unloading time, and then allowing the rotor speed to recover to the operating speed based on a preset linear slope.

Images