Crop water shortage diagnosis system and method

Abstract

The invention relates to a crop water shortage diagnosis system and method, and belongs to the technical field of machine learning. The system comprises an unmanned aerial vehicle control module, an image acquisition module, a data transmission module and a water shortage diagnosis module. The unmanned aerial vehicle control module is used for remotely controlling the unmanned aerial vehicle; the image acquisition module is mounted on the unmanned aerial vehicle and is used for acquiring a crop spectral image of a target area; the image acquisition module comprises a programmable image sensor; the data transmission module is used for data transmission between the image acquisition module and the water shortage diagnosis module; and the water shortage diagnosis module is used for analyzing the spectral image so as to judge the water shortage degree of crops in the target area and carry out water shortage early warning. The flight route of the unmanned aerial vehicle is adjusted through the obstacle avoidance model, the windowing position and the window size of the image sensor are controlled in a programming mode, the required spectral band is selected, the water shortage degree of crops can be quickly diagnosed, and the intelligent degree is high.

Description

Technical Field

[0001] The invention belongs to the technical field of machine learning and relates to a system and method for diagnosing water shortage in crops. Background Art

[0002] At present, the way to diagnose water shortage in crops is usually to use drones to take images along a preset route, and then export, calibrate, and analyze the data to diagnose the degree of water shortage in the crops. The whole process is cumbersome, time-consuming, and costly, and it is easy to occupy a large amount of server memory, which seriously affects the practicality of this method of diagnosing water shortage in crops. At the same time, in the face of unpredictable flight environments, considering the limitations of sensors and onboard computing resources, it is impossible to provide the comprehensive and detailed environmental perception information required by drones. Therefore, it is particularly difficult for drones to recognize and avoid obstacles such as telephone poles and tree branches in real time.

[0003] In addition, multi-(hyper)spectral imaging technology, as an important remote sensing method, can obtain spectral information of crops at multiple wavelengths and reveal the physiological state and environmental conditions of crops. Spectral information in different bands can reflect the moisture status, nutritional components and pest and disease conditions of crops. However, existing multi-(hyper)spectral imaging systems are usually fixed-band imaging, such as red light, green light, blue light and near-infrared, with poor spectral selectivity, only applicable to some occasions, and have great limitations; at the same time, the total number of pixels in the imaging detector also has a certain impact on the image quality. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and to provide a crop water shortage diagnosis system and method, which can remotely select the required spectral bands through programming and quickly analyze the degree of crop water shortage.

[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0006] In one aspect, the present invention provides a crop water shortage diagnosis system, comprising: a drone control module, an image acquisition module, a data transmission module, and a water shortage diagnosis module;

[0007] The drone control module is used to remotely control the drone;

[0008] The image acquisition module is installed on the UAV and is used to collect spectral images of crops in the target area;

[0009] The image acquisition module includes an image sensor, and the image sensor can control its window position and window size by programming, so as to select a spectral band;

[0010] The data transmission module is used for data transmission between the image acquisition module and the water shortage diagnosis module, including transmission of the acquired crop spectral images;

[0011] The water shortage diagnosis module is used to analyze the received spectral image to determine the degree of water shortage of crops in the target area and issue a water shortage warning.

[0012] Further, the drone control module includes a microcomputer and a depth camera arranged on the drone;

[0013] The depth camera is used to obtain a depth image in front of the flight path of the drone;

[0014] A pre-trained obstacle avoidance model is built on the microcomputer, and a safe flight window of the UAV on the depth image is obtained by inputting the depth image into the obstacle avoidance model, and the obstacle avoidance model is a deep neural network model;

[0015] The drone's control system adjusts the drone's horizontal, vertical and forward flight distance according to the safe flight window.

[0016] Furthermore, according to the safe flight window, the horizontal, vertical and forward flight distances of the drone are adjusted, including:

[0017] Assume the coordinates of the upper left corner of the depth image are , the coordinates of the lower right corner are , the coordinates of the upper left corner of the safe flight window are , the coordinates of the lower right corner are , the center point is ;

[0018] if and , the UAV moves horizontally and vertically to adjust the flight according to the center point of the safe flight window, and the forward flight distance of the UAV is:

[0019] ,

[0020] in, is the proportionality coefficient; is the attenuation coefficient; The minimum safe distance for drone flight;

[0021] Otherwise, the center point of the safe flight window is not inside the depth image, and the drone currently has no area to safely pass through. At this time, the drone uses radar for auxiliary navigation and obstacle avoidance.

[0022] Further, the data transmission module includes a ground server and a control panel installed on the drone;

[0023] The ground server transmits the control command to the control board in a programming manner, and the control board controls the image acquisition module based on the received control command.

[0024] Furthermore, the window position and window size of the image sensor are controlled programmatically, including:

[0025] Based on the required spectral band, the ground server sends control commands to the control board in a programming form, and the control board controls the window position and window size of the image sensor based on the received control commands.

[0026] Furthermore, the image acquisition module also includes a filter and a wedge-shaped film;

[0027] The filter and the wedge-shaped film are combined to form a spectroscopic device with a wedge-shaped FP resonant cavity. The spectroscopic device allows the input light source to pass through a spectrum range of 400nm to 1000nm with a spectral resolution of 5nm. The central wavelength of the filter is 500nm.

[0028] Furthermore, the image sensor is used to convert the received light signal into an electrical signal, and to pre-process the collected crop spectral image, including denoising, color correction and white balance adjustment.

[0029] Furthermore, the water shortage diagnosis module includes an image stitching unit, a crop water shortage diagnosis unit, a visualization unit, and a water shortage early warning unit;

[0030] The image stitching unit stitches the received crop spectral images based on the latitude and longitude coordinate information to integrate the crop spectral images of the target area;

[0031] The crop water shortage diagnosis unit analyzes the spliced ​​spectral images to determine the degree of water shortage of crops in the target area;

[0032] The visualization unit displays the degree of water shortage of crops through a bar graph and / or a line graph;

[0033] The water shortage warning unit performs a water shortage warning for crops in a target area based on the analysis result of the crop water shortage diagnosis unit, and indicates the water shortage location.

[0034] Furthermore, the water shortage diagnosis unit performs feature extraction on the spliced ​​spectral image based on the trained machine learning model to obtain the soil moisture content in the crop spectral image.

[0035] In another aspect, the present invention also provides a method for diagnosing water shortage in crops, comprising:

[0036] Determine the spectral band according to the application scenario;

[0037] According to the determined spectral band, the data transmission module controls the window position and window size of the image sensor by programming;

[0038] The drone flies according to the planned route and adjusts the flight route through the drone control module. The image acquisition module carried by the drone acquires images of the target area;

[0039] The collected crop spectral images are transmitted to the water shortage diagnosis module through the data transmission module;

[0040] The water shortage diagnosis module analyzes the crop spectral images, determines the degree of water shortage of crops in the target area, and issues water shortage warnings based on the degree of water shortage of crops.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] The invention provides a crop water shortage diagnosis system, which performs preliminary planning of the route of the drone through a drone control module, obtains the safe flight window of the drone by using a microcomputer and a depth camera carried on the drone during the flight of the drone, adjusts the flight route of the drone according to the safe flight window, and enables the drone to avoid obstacles; an image acquisition module is installed on the drone, and is used to collect crop images. The image acquisition module comprises an image sensor capable of controlling the window position and the window size by programming, so as to remotely control the spectral band of the image sensor, and the selectable spectral range can reach 400-1000 nm, and the spectral resolution reaches 5 nm, which meets the requirements of different multi-(high) spectral remote sensing application scenarios and different crop water shortage diagnosis; the data transmission module can quickly transmit the captured spectral image to a ground server, omitting the steps of exporting and importing the image externally, and is real-time, fast and reliable; the water shortage diagnosis module can quickly analyze the water shortage degree of the acquired crop spectral image through a machine learning model, complete the diagnosis of the water shortage degree of the crop and perform water shortage warning in a short time, and has a high degree of intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of the structure of a crop water shortage diagnosis system provided by an embodiment of the present invention;

[0044] Figure 2 is an example of an image captured by a depth camera in an embodiment of the present invention, wherein (a) is an original image and (b) is a depth image;

[0045] Figure 3 Schematic diagram of the structure of the RepVGG model in an embodiment of the present invention;

[0046] Figure 4A schematic flow chart of a method for diagnosing water shortage in crops provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. The embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0048] The term "and / or" herein is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " herein generally indicates that the front and back associated objects are in an "or" relationship. The orientation or position relationship indicated by the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "inside", "outside", etc. is based on the orientation or position relationship shown in the accompanying drawings, and is only used to explain the relative position relationship, movement, etc. between the components in a certain specific posture. If the specific posture changes, the directional indication will also change accordingly. It is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present disclosure / application.

[0049] In addition, the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", etc. may explicitly or implicitly include one or more of the feature. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0050] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood by specific circumstances.

[0051] Embodiment 1:

[0052] like Figure 1As shown, an embodiment of the present invention provides a crop water shortage diagnosis system, the system includes: an unmanned aerial vehicle control module, an image acquisition module, a data transmission module and a water shortage diagnosis module; the unmanned aerial vehicle control module is used to remotely control the unmanned aerial vehicle; the image acquisition module is installed on the unmanned aerial vehicle, and is used to collect crop spectral images in a target area; the image acquisition module includes an image sensor, and the image sensor can control its window position and window size by programming, so as to select a spectral band; the data transmission module is used for data transmission between the image acquisition module and the water shortage diagnosis module, including transmitting the collected crop spectral images; the water shortage diagnosis module is used to analyze the received spectral images to determine the degree of water shortage of the crops and issue a water shortage warning.

[0053] The drone control module includes a microcomputer and a depth camera installed on the drone; the depth camera is used to obtain a depth image in front of the drone's flight path; a pre-trained obstacle avoidance model is built on the microcomputer, which is used to obtain the drone's safe flight window on the depth image by inputting the depth image into the obstacle avoidance model, and the obstacle avoidance model is a deep neural network model. The drone's control system adjusts the drone's horizontal, vertical and forward flight distances in real time based on the obtained safe flight window.

[0054] When a drone performs a mission in a complex and ever-changing environment, an efficient autonomous obstacle avoidance strategy can help the drone avoid potential obstacles. Such a strategy can not only increase the autonomy of the drone, but also significantly reduce the risk of accidental collisions. Therefore, the present invention continuously adjusts the flight path of the drone by obtaining a safe flight window in the depth image ahead of the drone's flight path.

[0055] Examples of images captured by the depth camera are as follows: Figure 2 As shown in the figure, (a) is the original image and (b) is the depth image. Each pixel in the depth image represents an image at a specific (x, y) coordinate in the field of view of the depth sensor, using the depth image distance as the pixel value. The closer the distance, the darker the color representation visually.

[0056] In the embodiment of the present invention, the depth camera uses Intel's Realsense D435 sensor.

[0057] The present invention uses an unmanned aerial vehicle equipped with a microcomputer to perform the task of collecting crop spectral images. The limitation of onboard computing resources puts forward requirements on the computational efficiency and lightweight of the obstacle avoidance model. It is necessary to ensure that the unmanned aerial vehicle can process sensor data in real time and make obstacle avoidance decisions quickly without sacrificing the accuracy of safe flight window recognition.

[0058] In the embodiment of the present invention, the backbone network of the obstacle avoidance model uses the RepVGG model, and a regression model is added at the end of the RepVGG network to predict the safe flight window. Figure 3 As shown in the figure, the RepVGG model is constructed into a deep convolutional neural network by repeatedly stacking RepVGGBlocks. The core of the model is the 3×3 and 1×1 convolution operations performed in parallel inside each RepVGGBlock. The outputs of these two convolutions are added together and then processed by the ReLU activation function to map the input features to a nonlinear feature space.

[0059] When training the obstacle avoidance model, a self-built data set is used for training. The data set contains obstacles in different scenarios and labels marked according to the direction in which the drone controls the obstacle avoidance.

[0060] Adjust the drone's horizontal, vertical, and forward flight distance according to the safe flight window, including:

[0061] Assume that the coordinates of the upper left corner of the depth image are , the coordinates of the lower right corner are , the coordinates of the upper left corner of the safe flight window are , the coordinates of the lower right corner are , the center point is ;

[0062] if and , the drone moves horizontally and vertically to adjust its flight according to the center point of the safe flight window, and the forward flight distance of the drone is:

[0063] ,

[0064] in, is a proportional coefficient used to adjust the relationship between the safe flight window size and the forward flight distance; is the attenuation coefficient, which is used to control the sensitivity of the distance map; The minimum safe distance for the drone to fly, ensuring that the drone does not get too close to obstacles. Map the relationship between the size of the safe flight window and the distance between the drone and the obstacle. The larger the safe window, the closer the distance between the drone and the obstacle. The smaller the safe window, the farther the distance between the drone and the obstacle.

[0065] Otherwise, the center point of the safe flight window is not inside the depth image, and the drone currently has no area to safely pass through. At this time, the drone uses radar for auxiliary navigation and obstacle avoidance.

[0066] Image sensors are used to convert received light signals into electrical signals. Currently, most sensors use fixed spectral bands, which have great limitations on the analysis of different crops. The programmable image sensor used in the image acquisition module of the present invention can control its window position and window size by programming, so as to remotely control the spectral band of the image sensor according to the application scenario.

[0067] The image acquisition module also includes a camera, a filter and a wedge film.

[0068] The filter and the wedge-shaped film are combined to form a spectroscopic device with a wedge-shaped FP resonant cavity. The wedge-shaped FP resonant cavity forms a wedge-shaped structure between two flat plates instead of a traditional parallel structure. This structure causes the light to undergo angle changes during the reflection process, thereby affecting the light propagation path and interference effect. The wedge-shaped FP resonant cavity can adjust and change the transmission, reflection, polarization and phase state of the light wave by adjusting the wedge angle. The filter can switch between bandpass, high-pass and low-pass functions. The spectroscopic device allows the input light source to pass through a spectral range of 400nm to 1000nm, with a spectral resolution of 5nm. The central wavelength of the filter is 500nm, which can meet the needs of different multi-(high) spectral remote sensing application scenarios and different crop water shortage diagnosis.

[0069] The image sensor of the present invention has a size of 4.25 μm and a total number of pixels of 5056*2968, and can provide high-quality multi-spectral (high) spectral images. In addition to converting the received optical signal into an electrical signal and performing preliminary analog signal processing, such as denoising, color correction, and white balance adjustment of the spectral image, it can also be used as a spectral band control platform to control the window position and window size of the sensor by programming, so as to select the required band.

[0070] In the embodiment of the present invention, the camera has 10 lenses, with a full field MTF > 0.4 and a maximum distortion of less than 0.1%, reaching an advanced level. The weight of the entire image acquisition module does not exceed 2kg, which is suitable for most drones.

[0071] The data transmission module mainly includes a ground server and a control board installed on the UAV. The ground server transmits control commands to the control board through programming, and the control board controls the image acquisition module based on the received control commands.

[0072] The control board controls the image acquisition module based on the received control command, including controlling the window position and window size of the image sensor as mentioned above.

[0073] In the embodiment of the present invention, the data transmission system between the image acquisition module and the ground server is built based on the Netty framework. Netty is a network programming and high-performance network application framework based on Java NIO. It provides an asynchronous, event-driven network application framework and tools for quickly developing high-performance, high-reliability network servers and client programs, which can ensure that images are quickly and securely transmitted to the water shortage diagnosis module.

[0074] The water shortage diagnosis module includes an image stitching unit, a crop water shortage diagnosis unit, a visualization unit and a water shortage warning unit; the image stitching unit is used to stitch the received crop spectral images based on the latitude and longitude coordinate information, and integrate the crop spectral images of the target area; the crop water shortage diagnosis unit is used to analyze the stitched spectral images to determine the degree of water shortage of the crops in the target area; the visualization unit is used to display the degree of water shortage of the crops in the target area through a bar chart and / or a line chart; the water shortage warning unit is used to warn the crops of water shortage in the target area based on the analysis results of the crop water shortage diagnosis unit and indicate the water shortage location.

[0075] Among them, the crop water shortage diagnosis unit of the present invention performs feature extraction on the spliced ​​spectral image based on the trained machine learning model to obtain the soil moisture content in the crop spectral image.

[0076] In the embodiment of the present invention, the machine learning model uses a random forest model (RS), and the model is trained in advance through a large number of crop spectral images with known soil moisture content. Different machine learning models can be established for different types of crops.

[0077] The water shortage diagnosis module of the present invention can quickly analyze the acquired crop spectral images, realize water shortage diagnosis for various types of crops and timely reflect the degree and specific location of crop water shortage, which is beneficial to precision irrigation.

[0078] Embodiment 2:

[0079] like Figure 4 As shown, the embodiment of the present invention provides a method for diagnosing water shortage in crops. This method can be implemented based on the water shortage diagnosis system for crops described in Example 1. Therefore, the specific definitions of the drone control module, image acquisition module, data transmission module, and water shortage diagnosis module provided below can refer to the definitions of modules in the water shortage diagnosis system for crops described above, and will not be repeated here. This flowchart only shows the logical sequence of the method described in this embodiment. Under the premise of no conflict, in other possible embodiments of the present invention, different Figure 4 The steps shown or described are completed in the order shown. The method specifically comprises the following steps:

[0080] Determine the spectral band according to the application scenario;

[0081] According to the determined spectral band, the data transmission module controls the window position and window size of the image sensor by programming;

[0082] The drone flies according to the planned route and adjusts the flight route through the drone control module. The image acquisition module carried by the drone acquires images of the target area;

[0083] The collected crop spectral images are transmitted to the water shortage diagnosis module through the data transmission module;

[0084] The water shortage diagnosis module analyzes the crop spectral images, determines the degree of water shortage of crops in the target area, and issues water shortage warnings based on the degree of water shortage of crops.

[0085] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of this disclosure / application.

Claims

1. A crop water shortage diagnosis system, characterized in that: include: UAV control module, image acquisition module, data transmission module and water shortage diagnosis module; The drone control module is used to remotely control the drone; The image acquisition module is installed on the UAV and is used to collect spectral images of crops in the target area; The image acquisition module includes an image sensor, and the image sensor can control its window position and window size by programming, so as to select a spectral band; The data transmission module is used for data transmission between the image acquisition module and the water shortage diagnosis module, including transmission of the acquired crop spectral images; The water shortage diagnosis module is used to analyze the received spectral image to determine the degree of water shortage of crops in the target area and issue a water shortage warning.

2. The crop water shortage diagnosis system according to claim 1, characterized in that: The drone control module includes a microcomputer and a depth camera arranged on the drone; The depth camera is used to obtain a depth image in front of the flight path of the drone; A pre-trained obstacle avoidance model is built on the microcomputer, which is used to obtain a safe flight window for the UAV on the depth image by inputting the depth image into the obstacle avoidance model, wherein the obstacle avoidance model is a deep neural network model; The drone's control system adjusts the drone's horizontal, vertical and forward flight distance according to the safe flight window.

3. The crop water shortage diagnosis system according to claim 2, characterized in that: Adjust the drone's horizontal, vertical, and forward flight distance according to the safe flight window, including: Assume the coordinates of the upper left corner of the depth image are , the coordinates of the lower right corner are , the coordinates of the upper left corner of the safe flight window are , the coordinates of the lower right corner are , the center point is ; if and , the UAV moves horizontally and vertically to adjust the flight according to the center point of the safe flight window, and the forward flight distance of the UAV is: ， in, is the proportionality coefficient; is the attenuation coefficient; The minimum safe distance for drone flight; Otherwise, the center point of the safe flight window is not inside the depth image, and the drone currently has no area to safely pass through. At this time, the drone uses radar for auxiliary navigation and obstacle avoidance.

4. The crop water shortage diagnosis system according to claim 1, characterized in that: The data transmission module includes a ground server and a control panel installed on the drone; The ground server transmits the control command to the control board in a programming manner, and the control board controls the image acquisition module based on the received control command.

5. The crop water shortage diagnosis system according to claim 4, characterized in that: Programmatically control the image sensor window position and window size, including: Based on the required spectral band, the ground server sends control commands to the control board in a programming form, and the control board controls the window position and window size of the image sensor based on the received control commands.

6. The crop water shortage diagnosis system according to claim 1, characterized in that: The image acquisition module also includes a filter and a wedge-shaped film; The filter and the wedge-shaped film are combined to form a spectroscopic device with a wedge-shaped FP resonant cavity. The spectroscopic device allows the input light source to pass through a spectrum range of 400nm to 1000nm with a spectral resolution of 5nm. The central wavelength of the filter is 500nm.

7. The crop water shortage diagnosis system according to claim 1, characterized in that: The image sensor is used to convert the received light signal into an electrical signal, and to pre-process the collected crop spectral image, including denoising, color correction and white balance adjustment.

8. The crop water shortage diagnosis system according to claim 1, characterized in that: The water shortage diagnosis module includes an image stitching unit, a crop water shortage diagnosis unit, a visualization unit and a water shortage early warning unit; The image stitching unit stitches the received crop spectral images based on the latitude and longitude coordinate information to integrate the crop spectral images of the target area; The crop water shortage diagnosis unit analyzes the spliced ​​spectral images to determine the degree of water shortage of crops in the target area; The visualization unit displays the degree of water shortage of crops through a bar graph and / or a line graph; The water shortage warning unit performs a water shortage warning for crops in a target area based on the analysis result of the crop water shortage diagnosis unit, and indicates the water shortage location.

9. The crop water shortage diagnosis system according to claim 8, characterized in that: The water shortage diagnosis unit performs feature extraction on the spliced ​​spectral image based on the trained machine learning model to obtain the soil moisture content in the crop spectral image.

10. A method for diagnosing crop water shortage based on the crop water shortage diagnosis system according to any one of claims 1 to 9, comprising: Determine the spectral band according to the application scenario; According to the determined spectral band, the data transmission module controls the window position and window size of the image sensor by programming; The drone flies according to the planned route and adjusts the flight route through the drone control module. The image acquisition module carried by the drone acquires images of the target area; The collected crop spectral images are transmitted to the water shortage diagnosis module through the data transmission module; The water shortage diagnosis module analyzes the crop spectral images, determines the degree of water shortage of crops in the target area, and issues water shortage warnings based on the degree of water shortage of crops.

Images