A system and method for diagnosing crop water shortage

Abstract

The present invention relates to a system and method for diagnosing crop water shortage, belonging to the field of machine learning technology. The system includes: an unmanned aerial vehicle (UAV) control module, an image acquisition module, a data transmission module, and a water shortage diagnosis module; the UAV control module is used to remotely control the UAV; the image acquisition module is installed on the UAV and is used to collect spectral images of crops in a target area; the image acquisition module includes a programmable image sensor; the data transmission module is used for data transmission between the image acquisition module and the water shortage diagnosis module; the water shortage diagnosis module is used to analyze the spectral image to determine the degree of water shortage of crops in the target area and issue a water shortage warning. The present invention adjusts the flight path of the UAV through an obstacle avoidance model, uses programming to control the window position and size of the image sensor, selects the required spectral band, and can quickly complete the diagnosis of the degree of water shortage of crops, with a high degree of intelligence.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning and relates to a system and method for diagnosing water shortage in crops. Background Art

[0002] Currently, diagnosing crop water shortages typically involves drones following a pre-set flight path to capture images, then exporting, correcting, and analyzing the data to diagnose the extent of the crop water shortage. This process is cumbersome, time-consuming, and costly, and it easily consumes significant server memory, severely limiting the practicality of this method. Furthermore, given the limitations of sensors and onboard computing resources, drones cannot provide the comprehensive and detailed environmental perception information they require. Consequently, real-time identification and avoidance of obstacles such as utility poles and tree branches present significant challenges.

[0003] Furthermore, multi- and hyperspectral imaging technology, as an important remote sensing tool, can acquire spectral information about crops at multiple wavelengths, revealing their physiological state and environmental conditions. Spectral information from different bands can reveal crop moisture status, nutritional composition, and pest and disease status. However, existing multi- and hyperspectral imaging systems typically use fixed wavelengths, such as red, green, blue, and near-infrared. These systems have poor spectral selectivity, making them suitable only for certain applications and subject to significant limitations. Furthermore, the total number of pixels in the imaging detector also impacts image quality. Summary of the Invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a crop water shortage diagnosis system and method, which can remotely select the required spectral bands through programming and quickly analyze the degree of crop water shortage.

[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0006] In one aspect, the present invention provides a crop water shortage diagnosis system, comprising: a drone control module, an image acquisition module, a data transmission module, and a water shortage diagnosis module;

[0007] The drone control module is used to remotely control the drone;

[0008] The image acquisition module is installed on the UAV and is used to collect spectral images of crops in the target area;

[0009] The image acquisition module includes an image sensor, and the image sensor can control its window position and window size through programming to select a spectral band;

[0010] The data transmission module is used for data transmission between the image acquisition module and the water shortage diagnosis module, including transmission of the acquired crop spectral images;

[0011] The water shortage diagnosis module is used to analyze the received spectral image to determine the water shortage degree of crops in the target area and issue a water shortage warning.

[0012] Furthermore, the drone control module includes a microcomputer and a depth camera provided on the drone;

[0013] The depth camera is used to obtain a depth image in front of the UAV flight path;

[0014] A pre-trained obstacle avoidance model is built on the microcomputer. By inputting the depth image into the obstacle avoidance model, a safe flight window of the UAV on the depth image is obtained. The obstacle avoidance model is a deep neural network model.

[0015] The drone's control system adjusts the drone's horizontal, vertical and forward flight distance based on the safe flight window.

[0016] Furthermore, the horizontal, vertical, and forward flight distances of the drone are adjusted according to the safe flight window, including:

[0017] Assume that the coordinates of the upper left corner of the depth image are , the coordinates of the lower right corner are , the coordinates of the upper left corner of the safe flight window are , the coordinates of the lower right corner are , the center point is ;

[0018] if and , the UAV moves horizontally and vertically to adjust its flight according to the center point of the safe flight window. The forward flight distance of the UAV is:

[0019] ,

[0020] in, is the proportionality coefficient; is the attenuation coefficient; The minimum safe distance for drone flight;

[0021] Otherwise, the center point of the safe flight window is not within the depth image, and the drone currently has no area to safely pass through. At this time, the drone uses radar for auxiliary navigation and obstacle avoidance.

[0022] Furthermore, the data transmission module includes a ground server and a control panel installed on the drone;

[0023] The ground server transmits the control command to the control board in a programming manner, and the control board controls the image acquisition module based on the received control command.

[0024] Furthermore, the image sensor window position and window size are controlled programmatically, including:

[0025] Based on the required spectral band, the ground server sends control commands to the control board in a programming form, and the control board controls the window position and window size of the image sensor based on the received control commands.

[0026] Furthermore, the image acquisition module further includes a filter and a wedge-shaped film;

[0027] The filter and the wedge-shaped film are combined to form a spectroscopic device with a wedge-shaped FP resonant cavity. The spectroscopic device allows the input light source to pass through a spectral range of 400nm to 1000nm with a spectral resolution of 5nm. The central wavelength of the filter is 500nm.

[0028] Furthermore, the image sensor is used to convert the received light signal into an electrical signal, and to pre-process the collected crop spectral image, including denoising, color correction and white balance adjustment.

[0029] Furthermore, the water shortage diagnosis module includes an image stitching unit, a crop water shortage diagnosis unit, a visualization unit, and a water shortage early warning unit;

[0030] The image stitching unit stitches the received crop spectral images based on the latitude and longitude coordinate information to integrate the crop spectral images of the target area;

[0031] The crop water shortage diagnosis unit analyzes the spliced ​​spectral images to determine the degree of water shortage of crops in the target area;

[0032] The visualization unit displays the degree of water shortage of crops through a bar graph and / or a line graph;

[0033] The water shortage warning unit performs a water shortage warning for crops in a target area based on the analysis result of the crop water shortage diagnosis unit, and indicates the water shortage location.

[0034] Furthermore, the water shortage diagnosis unit performs feature extraction on the spliced ​​spectral images based on the trained machine learning model to obtain the soil moisture content in the crop spectral images.

[0035] In another aspect, the present invention further provides a method for diagnosing water shortage in crops, comprising:

[0036] Determine the spectral band according to the application scenario;

[0037] According to the determined spectral band, the data transmission module controls the window position and window size of the image sensor through programming;

[0038] The drone flies according to the planned route and adjusts the flight path through the drone control module. The image acquisition module carried by the drone collects images of the target area;

[0039] The collected crop spectral images are transmitted to the water shortage diagnosis module through the data transmission module;

[0040] The water shortage diagnosis module analyzes the spectral images of crops, determines the degree of water shortage of crops in the target area, and issues water shortage warnings based on the degree of water shortage of crops.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] The present invention proposes a crop water shortage diagnosis system. The system uses an unmanned aerial vehicle (UAV) control module to preliminarily plan the flight path of a UAV. During flight, the UAV's onboard microcomputer and depth camera are used to obtain a safe flight window for the UAV. The UAV's flight path is adjusted based on the safe flight window to enable the UAV to avoid obstacles. An image acquisition module is mounted on the UAV and is used to capture crop images. The module includes an image sensor whose window position and size can be controlled programmatically, thereby remotely controlling the image sensor's spectral band. The selectable spectral range can reach 400-1000 nm, and the spectral resolution can reach 5 nm, meeting the needs of various multi-(high-)spectral remote sensing application scenarios and different crop water shortage diagnosis requirements. A data transmission module can rapidly transmit the captured spectral images to a ground server, eliminating the steps of externally exporting and importing images. This system is real-time, fast, and reliable. The water shortage diagnosis module uses a machine learning model to rapidly analyze the water shortage level of the acquired crop spectral images, completing crop water shortage diagnosis and issuing water shortage warnings in a short period of time, with a high degree of intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic structural diagram of a crop water shortage diagnosis system provided by an embodiment of the present invention;

[0044] Figure 2 : An example of an image captured by a depth camera in an embodiment of the present invention, where (a) is an original image and (b) is a depth image;

[0045] Figure 3 Schematic diagram of the structure of the RepVGG model in an embodiment of the present invention;

[0046] Figure 4A schematic flow chart of a method for diagnosing water shortage in crops provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. The same reference numerals in the drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. The embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations of the technical solution of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0048] The term "and / or" herein is merely a description of the association relationship between associated objects, indicating that three possible relationships exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. Furthermore, the character " / " herein generally indicates that the front and back associated objects are in an "or" relationship. The directions or positional relationships indicated by terms such as "center," "longitudinal," "lateral," "upper," "lower," "front," "back," "left," "right," "inside," and "outside" are based on the directions or positional relationships shown in the accompanying drawings and are used only to explain the relative positional relationships and movement conditions between components in a specific posture. If the specific posture changes, the directional indication will also change accordingly. These terms are merely for the convenience of describing this application and to simplify the description, and are not intended to indicate or imply that the device or component referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limitations on this disclosure / application.

[0049] Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature identified with "first," "second," etc., may explicitly or implicitly include one or more of the features. Throughout the description of this application, unless otherwise specified, "plurality" means two or more.

[0050] In the description of this application, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on specific circumstances.

[0051] Example 1:

[0052] like Figure 1As shown, an embodiment of the present invention provides a crop water shortage diagnosis system, which includes: an unmanned aerial vehicle (UAV) control module, an image acquisition module, a data transmission module, and a water shortage diagnosis module; the UAV control module is used to remotely control the UAV; the image acquisition module is installed on the UAV and is used to collect spectral images of crops in a target area; the image acquisition module includes an image sensor, and the image sensor can control its window position and window size through programmable control to select spectral bands; the data transmission module is used to transmit data between the image acquisition module and the water shortage diagnosis module, including transmitting the collected crop spectral images; the water shortage diagnosis module is used to analyze the received spectral images to determine the degree of water shortage of the crops and issue a water shortage warning.

[0053] The drone control module includes a microcomputer and a depth camera mounted on the drone. The depth camera is used to capture a depth image of the drone's flight path. A pre-trained obstacle avoidance model, built on the microcomputer, is used to input the depth image into the obstacle avoidance model, a deep neural network model, to determine the drone's safe flight window. The drone's control system adjusts the drone's horizontal, vertical, and forward flight distances in real time based on the acquired safe flight window.

[0054] When drones are performing missions in complex and ever-changing environments, efficient autonomous obstacle avoidance strategies can help them circumvent potential obstacles. Such strategies not only increase the drone's autonomy but also significantly reduce the risk of accidental collisions. Therefore, the present invention continuously adjusts the drone's flight path by acquiring a safe flight window from depth images of the drone's forward flight path.

[0055] Examples of images captured by the depth camera are as follows: Figure 2 As shown in the figure, (a) is the original image and (b) is the depth image. Each pixel in the depth image represents the image at a specific (x, y) coordinate in the field of view of the depth sensor, using the depth image distance as the pixel value. The closer the distance, the darker the visual color representation.

[0056] In the embodiment of the present invention, the depth camera uses Intel's Realsense D435 sensor.

[0057] The present invention uses an unmanned aerial vehicle (UAV) equipped with a microcomputer to perform the task of collecting crop spectral images. The limitation of onboard computing resources puts forward requirements on the computational efficiency and lightweightness of the obstacle avoidance model. It is necessary to ensure that the UAV can process sensor data in real time and make obstacle avoidance decisions quickly without sacrificing the accuracy of safe flight window identification.

[0058] In the embodiment of the present invention, the backbone network of the obstacle avoidance model uses the RepVGG model, and a regression model is added at the end of the RepVGG network to predict the safe flight window. Figure 3 As shown in Figure 1, the RepVGG model is constructed by repeatedly stacking RepVGGBlocks to form a deep convolutional neural network. The core of the model lies in the 3×3 and 1×1 convolution operations performed in parallel within each RepVGGBlock. The outputs of these two convolutions are accumulated together and then processed through the ReLU activation function to map the input features into a nonlinear feature space.

[0059] When training the obstacle avoidance model, a self-built dataset is used for training. The dataset contains obstacles in different scenarios and labels marked according to the direction in which the drone controls the obstacle avoidance.

[0060] Adjust the drone's horizontal, vertical, and forward flight distance based on the safe flight window, including:

[0061] Assume that the coordinates of the upper left corner of the depth image are , the coordinates of the lower right corner are , the coordinates of the upper left corner of the safe flight window are , the coordinates of the lower right corner are , the center point is ;

[0062] if and , the UAV moves horizontally and vertically to adjust its flight according to the center point of the safe flight window. The forward flight distance of the UAV is:

[0063] ,

[0064] in, is the proportional coefficient used to adjust the relationship between the safe flight window size and the forward flight distance; is the attenuation coefficient, which is used to control the sensitivity of the distance map; The minimum safe distance for the drone to fly, ensuring that the drone does not get too close to obstacles. Map the relationship between the safe flight window size and the distance between the drone and the obstacle. The larger the safe window, the closer the distance between the drone and the obstacle, and the smaller the safe window, the farther the distance between the drone and the obstacle.

[0065] Otherwise, the center point of the safe flight window is not within the depth image, and the drone currently has no area to safely pass through. At this time, the drone uses radar for auxiliary navigation and obstacle avoidance.

[0066] Image sensors convert received light signals into electrical signals. Currently, most sensors use fixed spectral bands, significantly limiting their ability to analyze different crops. The programmable image sensor used in the image acquisition module of the present invention can programmatically control its window position and size, allowing remote control of the image sensor's spectral band based on the application scenario.

[0067] The image acquisition module also includes a camera, a filter and a wedge film.

[0068] The filter and wedge-shaped film combine to form a spectroscopic device with a wedge-shaped FP resonant cavity. The wedge-shaped FP resonant cavity is formed by two flat plates, forming a wedge-shaped structure rather than a traditional parallel structure. This structure causes light to experience angular changes during reflection, affecting the light propagation path and interference effect. The wedge-shaped FP resonant cavity can adjust and change the transmission, reflection, polarization, and phase state of light waves by adjusting the wedge angle. The filter can switch between bandpass, highpass, and lowpass functions. The spectroscopic device allows the input light source to pass through a spectral range of 400nm to 1000nm with a spectral resolution of 5nm. The filter has a center wavelength of 500nm, which can meet the needs of various multi-(hyper)spectral remote sensing applications and different crop water shortage diagnosis needs.

[0069] The image sensor of this invention has a size of 4.25μm and a total pixel size of 5056*2968, capable of providing high-quality multi-spectral (hyper-spectral) images. In addition to converting received optical signals into electrical signals and performing preliminary analog signal processing, such as denoising, color correction, and white balance adjustment, it also serves as a spectral band control platform, programmatically controlling the sensor's window position and size to select the desired band.

[0070] In this embodiment of the present invention, the camera uses 10 lens groups, achieving a full-field MTF greater than 0.4 and a maximum distortion less than 0.1%, reaching state-of-the-art standards. The entire image acquisition module weighs less than 2 kg, making it suitable for most drones.

[0071] The data transmission module mainly includes a ground server and a control board installed on the drone. The ground server transmits control commands to the control board through programming, and the control board controls the image acquisition module based on the received control commands.

[0072] The control board controls the image acquisition module based on the received control command, including controlling the window position and window size of the image sensor.

[0073] In this embodiment of the present invention, the data transmission system between the image acquisition module and the ground server is built using the Netty framework. Netty is a high-performance network application framework based on Java NIO. It provides an asynchronous, event-driven network application framework and tools for rapidly developing high-performance, highly reliable network servers and client programs, ensuring the rapid and secure transmission of images to the water shortage diagnosis module.

[0074] The water shortage diagnosis module includes an image stitching unit, a crop water shortage diagnosis unit, a visualization unit, and a water shortage warning unit; the image stitching unit is used to stitch received crop spectral images based on latitude and longitude coordinate information to integrate the crop spectral images of the target area; the crop water shortage diagnosis unit is used to analyze the stitched spectral images to determine the degree of water shortage of crops in the target area; the visualization unit is used to display the degree of water shortage of crops in the target area through a bar chart and / or a line chart; the water shortage warning unit is used to issue a crop water shortage warning in the target area and indicate the location of the water shortage based on the analysis results of the crop water shortage diagnosis unit.

[0075] The crop water shortage diagnosis unit of the present invention performs feature extraction on the spliced ​​spectral images based on the trained machine learning model to obtain the soil moisture content in the crop spectral images.

[0076] In this embodiment of the present invention, the machine learning model uses a random forest model (RS) and is pre-trained using a large number of crop spectral images with known soil moisture content. Different machine learning models can be established for different types of crops.

[0077] The water shortage diagnosis module of the present invention can quickly analyze the acquired crop spectral images, realize water shortage diagnosis of various types of crops, and promptly reflect the degree and specific location of crop water shortage, which is beneficial to precision irrigation.

[0078] Example 2:

[0079] like Figure 4 As shown, the embodiment of the present invention provides a method for diagnosing crop water shortage. This method can be implemented based on the crop water shortage diagnosis system described in Example 1. Therefore, the specific definitions of the drone control module, image acquisition module, data transmission module, and water shortage diagnosis module provided below can refer to the definitions of modules in the crop water shortage diagnosis system above, and will not be repeated here. This flow chart only shows the logical sequence of the method described in this embodiment. Under the premise of no conflict, in other possible embodiments of the present invention, different methods can be used. Figure 4 The steps shown or described are completed in the order shown. The method specifically includes the following steps:

[0080] Determine the spectral band according to the application scenario;

[0081] According to the determined spectral band, the data transmission module controls the window position and window size of the image sensor through programming;

[0082] The drone flies according to the planned route and adjusts the flight path through the drone control module. The image acquisition module carried by the drone collects images of the target area;

[0083] The collected crop spectral images are transmitted to the water shortage diagnosis module through the data transmission module;

[0084] The water shortage diagnosis module analyzes the spectral images of crops, determines the degree of water shortage of crops in the target area, and issues water shortage warnings based on the degree of water shortage of crops.

[0085] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of this disclosure / this application.

Claims

1. A crop water shortage diagnosis system, characterized in that: include: UAV control module, image acquisition module, data transmission module and water shortage diagnosis module; The drone control module is used to remotely control the drone; The image acquisition module is installed on the UAV and is used to collect spectral images of crops in the target area; The image acquisition module includes an image sensor, and the image sensor can control its window position and window size through programming to select a spectral band; The data transmission module is used for data transmission between the image acquisition module and the water shortage diagnosis module, including transmission of the acquired crop spectral images; The water shortage diagnosis module is used to analyze the received spectral image to determine the degree of water shortage of crops in the target area and issue a water shortage warning; The drone control module includes a microcomputer and a depth camera installed on the drone; The depth camera is used to obtain a depth image in front of the UAV flight path; A pre-trained obstacle avoidance model is built on the microcomputer, which is used to obtain a safe flight window for the UAV on the depth image by inputting the depth image into the obstacle avoidance model, wherein the obstacle avoidance model is a deep neural network model; The drone’s control system adjusts the drone’s horizontal, vertical, and forward flight distance based on the safe flight window; Among them, according to the safe flight window, adjust the horizontal, vertical and forward flight distance of the drone, including: Assume that the coordinates of the upper left corner of the depth image are , the coordinates of the lower right corner are , the coordinates of the upper left corner of the safe flight window are , the coordinates of the lower right corner are , the center point is ; if and , the UAV moves horizontally and vertically to adjust its flight according to the center point of the safe flight window. The forward flight distance of the UAV is: ， in, is the proportionality coefficient; is the attenuation coefficient; The minimum safe distance for drone flight; Otherwise, the center point of the safe flight window is not within the depth image, and the drone currently has no area to safely pass through. At this time, the drone uses radar for auxiliary navigation and obstacle avoidance.

2. The crop water shortage diagnosis system according to claim 1, characterized in that: The data transmission module includes a ground server and a control panel installed on the drone; The ground server transmits the control command to the control board in a programming manner, and the control board controls the image acquisition module based on the received control command.

3. The crop water shortage diagnosis system according to claim 2, characterized in that: Programmatically control the image sensor window position and size, including: Based on the required spectral band, the ground server sends control commands to the control board in a programming form, and the control board controls the window position and window size of the image sensor based on the received control commands.

4. The crop water shortage diagnosis system according to claim 1, characterized in that: The image acquisition module also includes a filter and a wedge-shaped film; The filter and the wedge-shaped film are combined to form a spectroscopic device with a wedge-shaped FP resonant cavity. The spectroscopic device allows the input light source to pass through a spectral range of 400nm to 1000nm with a spectral resolution of 5nm. The central wavelength of the filter is 500nm.

5. The crop water shortage diagnosis system according to claim 1, characterized in that: The image sensor is used to convert the received light signal into an electrical signal and perform pre-processing on the collected crop spectral image, including denoising, color correction and white balance adjustment.

6. The crop water shortage diagnosis system according to claim 1, characterized in that: The water shortage diagnosis module includes an image splicing unit, a crop water shortage diagnosis unit, a visualization unit, and a water shortage early warning unit; The image stitching unit stitches the received crop spectral images based on the latitude and longitude coordinate information to integrate the crop spectral images of the target area; The crop water shortage diagnosis unit analyzes the spliced ​​spectral images to determine the degree of water shortage of crops in the target area; The visualization unit displays the degree of water shortage of crops through a bar graph and / or a line graph; The water shortage warning unit performs a water shortage warning for crops in a target area based on the analysis result of the crop water shortage diagnosis unit, and indicates the water shortage location.

7. The crop water shortage diagnosis system according to claim 6, characterized in that: The water shortage diagnosis unit performs feature extraction on the spliced ​​spectral images based on the trained machine learning model to obtain the soil moisture content in the crop spectral images.

8. A method for diagnosing crop water shortage based on the crop water shortage diagnosis system according to any one of claims 1 to 7, characterized in that: include: Determine the spectral band according to the application scenario; According to the determined spectral band, the data transmission module controls the window position and window size of the image sensor through programming; The drone flies according to the planned route and adjusts the flight path through the drone control module. The image acquisition module carried by the drone acquires images of the target area; The collected crop spectral images are transmitted to the water shortage diagnosis module through the data transmission module; The water shortage diagnosis module analyzes crop spectral images to determine the degree of water shortage in the target area and issues water shortage warnings based on the degree of water shortage in the crops. Among them, the drone flies according to the planned route and adjusts the flight path through the drone control module, including: The depth camera acquires a depth image in front of the UAV flight path; The obstacle avoidance model obtains a safe flight window for the UAV on the depth image by inputting the depth image into the obstacle avoidance model, and the obstacle avoidance model is a deep neural network model; The drone’s control system adjusts the drone’s horizontal, vertical, and forward flight distance based on the safe flight window; Among them, according to the safe flight window, adjust the horizontal, vertical and forward flight distance of the drone, including: Assume that the coordinates of the upper left corner of the depth image are , the coordinates of the lower right corner are , the coordinates of the upper left corner of the safe flight window are , the coordinates of the lower right corner are , the center point is ; if and , the UAV moves horizontally and vertically to adjust its flight according to the center point of the safe flight window. The forward flight distance of the UAV is: ， in, is the proportionality coefficient; is the attenuation coefficient; The minimum safe distance for drone flight; Otherwise, the center point of the safe flight window is not within the depth image, and the drone currently has no area to safely pass through. At this time, the drone uses radar for auxiliary navigation and obstacle avoidance.

Images