Sugarcane growth monitoring and evaluating device based on multispectral imaging

Abstract

The invention discloses a sugarcane growth monitoring and evaluating device based on multispectral imaging in the technical field of modern farming. The sugarcane growth monitoring and evaluating device comprises an unmanned aerial vehicle, a multispectral acquisition module and a verification image acquisition module, the verification image acquisition module is used for carrying out verification image acquisition on the multispectral image acquisition area, extracting the swaying condition of the sugarcane from the verification image, judging whether the unmanned aerial vehicle needs to stay in the area or not based on the swaying condition of the sugarcane, and when judging that the unmanned aerial vehicle stays, sending the verification image to the unmanned aerial vehicle; and defining an area in which multispectral image acquisition is carried out as a verification area, controlling the multispectral acquisition module to repeatedly carry out multispectral image acquisition on the verification area, extracting the multispectral image of the blocked sugarcane leaf from the verification area, and carrying out position correction on the blocked multispectral image of the leaf. According to the method, the verification image is collected while the multispectral image is collected, the multispectral image is corrected based on the correction image, and the influence of sugarcane leaf shielding on the growth condition judgment and prediction precision is reduced.

Description

Technical Field

[0001] The invention belongs to the field of modern agricultural technology, and in particular is a sugarcane growth monitoring and evaluation device based on multispectral imaging. Background Art

[0002] Sugarcane is one of the main raw materials for the sugar industry. Through extraction and refining, various sugar products such as white sugar, brown sugar, and rock sugar can be obtained. Sugarcane sugar accounts for about 90% of my country's total sugar production. Sugarcane is also an important raw material for making fuel ethanol. 40% of the raw materials for fuel ethanol in the world come from sugarcane. Sugarcane can also be used to make beverages, candies, desserts and other delicacies. Sugarcane juice is a refreshing drink that quenches thirst, and sugarcane wine is a nutritious and unique alcoholic drink. In addition, the sugar substances in sugarcane can also be used to make paper, chemical products, biological products, etc.

[0003] Generally, the sugarcane planting process includes seedling raising, land preparation, planting, fertilization, and pest and disease control. Seedling raising is usually carried out in spring, and healthy seedlings are selected for cutting or sowing. When preparing the land, deep plowing and careful cultivation are required, and sufficient base fertilizer is applied to keep the soil loose. When planting, the seedlings are inserted into the soil, covered with a thin layer of soil, and then watered to keep them moist. During the growth period, topdressing and weeding are required in a timely manner, and attention should be paid to pest and disease control. With the development of science and technology, in order to improve the yield and quality of sugarcane, accurately prevent and control pests and diseases, and optimize the planting structure and management strategy, a variety of sugarcane growth monitoring and evaluation devices have emerged. The modern technologies they mainly rely on include satellite remote sensing technology and drone remote sensing technology. Among them, drone remote sensing technology, mainly combined with multispectral or hyperspectral sensors carried by drones, can monitor the moisture and temperature conditions of the soil in real time, provide a scientific basis for sugarcane field irrigation, and collect high-definition image data to construct a surface model of the sugarcane crop, and then estimate the growth parameters such as the leaf area index (LAI) of sugarcane, and then monitor and evaluate the growth of sugarcane.

[0004] When monitoring and evaluating the sugarcane growth area, the existing sugarcane growth monitoring and evaluation device directly collects the corresponding image information through the multispectral imaging instrument, directly analyzes the multispectral image information, and judges and predicts the growth of the sugarcane. However, in the sugarcane growing area, due to the high density of sugarcane planting, there are some sugarcane leaves that are blocked by adjacent sugarcanes, especially between sugarcanes with large growth differences. The growth and development status of the blocked leaves is difficult to be collected by the multispectral imaging instrument. For example, some sugarcane leaves with insect pests or poor growth and development are blocked by some sugarcane leaves with good growth and development. When the multispectral imaging instrument collects data, the multispectral image information of the leaves with good growth and development located on the surface will be collected; thus, errors will occur in the subsequent judgment and prediction of the sugarcane growth.

[0005] In summary, the present invention proposes a sugarcane growth monitoring and evaluation device based on multispectral imaging. When acquiring multispectral images, correction images are also acquired, and the multispectral images are corrected based on the correction images to reduce the impact of sugarcane leaf occlusion on the judgment and prediction accuracy of its growth conditions. Summary of the invention

[0006] In order to solve the above problems, the purpose of the present invention is to provide a sugarcane growth monitoring and evaluation device based on multispectral imaging. By collecting verification images at the same time as multispectral images, the multispectral images are corrected based on the correction images, thereby reducing the influence of sugarcane leaf occlusion on the judgment and prediction accuracy of its growth conditions.

[0007] In order to achieve the above-mentioned object, the technical solution of the present invention is as follows: a sugarcane growth monitoring and evaluation device based on multispectral imaging, comprising a drone, a multispectral acquisition module and a verification image acquisition module, wherein the multispectral acquisition module and the verification image acquisition module are both mounted on the drone;

[0008] Drones, which are used to fly along a preset path;

[0009] Multispectral acquisition module, used to collect multispectral images of sugarcane planting areas;

[0010] The verification image acquisition module is used to acquire verification images for the area where multispectral image acquisition is being performed, and extract the swaying condition of sugarcane from the verification image. Based on the swaying condition of sugarcane, it is determined whether the UAV needs to stay in the area and the UAV is controlled to continue flying. When it is determined that the UAV stays, the area where multispectral image acquisition is being performed is defined as the verification area, and the multispectral acquisition module is controlled to repeatedly acquire multispectral images of the verification area, and the multispectral image of the obscured sugarcane leaves is extracted therefrom, and the multispectral image of the obscured sugarcane leaves is corrected in position; when the UAV continues to fly, the multispectral acquisition module is controlled to resume acquiring multispectral images of the unacquired sugarcane area;

[0011] A processing module, used for extracting plant growth characteristics from the multispectral image and the corrected multispectral image, and generating corresponding characteristic indexes;

[0012] The evaluation module is used to receive the characteristic index and output the growth status and estimated yield in the sugarcane planting area.

[0013] Furthermore, the verification image acquisition module is used to extract the obscured sugarcane leaves from the verification image when the UAV is not stopping. When the range of the obscured sugarcane leaves exceeds the occlusion threshold, the corresponding area is marked as the area to be verified.

[0014] Furthermore, it also includes an environmental monitoring module for performing environmental monitoring of the sugarcane planting area, and the monitoring data includes wind direction and wind speed; when it is determined that the drone is stopping, the wind speed is collected, and the collected wind speed data is defined as a re-collected wind speed threshold; when the wind speed data reaches the re-collected wind speed threshold, a first control signal is sent, and when the wind direction causes the blocked leaves to be unblocked, a second control signal is sent.

[0015] Furthermore, the drone is used to fly to the area to be verified when it receives the first control signal and the second control signal, collect multispectral images, obtain the multispectral images of the area to be verified, and update the multispectral images of the area not to be verified. The processing module is also used to extract plant growth characteristics from the updated multispectral images of the area to be verified.

[0016] Furthermore, the processing module is used to obtain the sugarcane planting area and establish a virtual coordinate system in the sugarcane planting area; when the wind speed does not meet the re-collection wind speed threshold, each sugarcane is marked with a corresponding coordinate point in the virtual coordinate system, and the marked coordinate is defined as the initial coordinate of the sugarcane; based on the initial coordinate of the sugarcane, the multispectral image information collected after the sugarcane is swayed is corrected and reset.

[0017] Furthermore, the processing module is used to stitch and reflectance correct the collected or corrected sugarcane multispectral images, perform reflectance correction on each band based on a correction plate with similar reflectance of sugarcane, generate index maps and DSMs for different bands, and calculate sugarcane indices, including normalized sugarcane index and chlorophyll index.

[0018] Furthermore, the processing module is used to create mask grids in different sugarcane index maps and DSMs, and multiply the sugarcane index maps of the same batch with the created mask grids in sequence to generate sugarcane index maps with soil background removed.

[0019] Furthermore, the processing module is used to extract the sugarcane index of each area based on the sugarcane index map with the soil background removed.

[0020] Furthermore, the evaluation module is used to output the growth conditions and expected yields in the sugarcane planting area based on a machine learning regression method after receiving the normalized sugarcane index and the chlorophyll index.

[0021] Furthermore, the processing module is used to obtain the sugarcane planting area, perform multi-point position analysis on each sugarcane root in the sugarcane planting area, and obtain a number of offset points; when the variances of the coordinates of the several offset points and the initial coordinates of the sugarcane meet the error variance threshold, the area formed by the sugarcane in the multispectral image is corrected to the initial coordinates of the sugarcane.

[0022] The following beneficial effects were achieved by adopting the above scheme: (1) This scheme, through the multispectral acquisition module equipped on the drone, can efficiently and widely acquire multispectral images of the sugarcane planting area, which greatly improves the monitoring accuracy and efficiency compared with the traditional manual monitoring method; at the same time, by verifying that the image acquisition module can judge the swaying of the sugarcane and the obstruction of the leaves in real time, the accuracy of the multispectral image acquisition is ensured, and the image distortion caused by environmental factors is avoided. According to the swaying of the sugarcane and the obstruction of the leaves, the drone can intelligently determine whether it needs to stay in a specific area for repeated acquisition, and whether it needs to correct the obstructed multispectral image, thus realizing dynamic adjustment and optimization during the monitoring process.

[0023] (2) This solution introduces an environmental monitoring module that can obtain wind direction and wind speed data in real time, providing a more accurate control basis for the flight and image acquisition of the UAV.

[0024] (3) This solution can extract plant growth features from the collected multispectral images through the processing module and generate corresponding feature indexes to provide accurate data support for the subsequent evaluation module. The evaluation module is based on machine learning regression methods and can accurately predict the growth and expected yield in the sugarcane planting area, providing a scientific decision-making basis for agricultural production.

[0025] (4) In this scheme, the entire monitoring and evaluation process is highly automated, which reduces human intervention and errors and improves the objectivity and accuracy of the monitoring results. At the same time, by establishing a virtual coordinate system and marking the initial coordinates of the sugarcane, the growth changes and swaying of the sugarcane can be accurately tracked, reducing the monitoring errors caused by offset.

[0026] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Schematic diagram of the process of the sugarcane growth monitoring and evaluation device based on multispectral imaging in this embodiment. DETAILED DESCRIPTION

[0028] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] The following is further described in detail through specific implementation methods:

[0030] Implementation example Figure 1 As shown, a sugarcane growth monitoring and evaluation device based on multispectral imaging is mainly composed of a drone, a multispectral acquisition module and a verification image acquisition module, both of which are mounted on the drone; the multispectral acquisition module is a multispectral camera, and the image acquisition module is a drone camera. The drone is used to fly along a preset path; the multispectral acquisition module is used to collect multispectral images of the sugarcane planting area.

[0031] The verification image acquisition module is used to perform verification image acquisition on the area where multispectral image acquisition is being performed, and extract the swaying condition of sugarcane from the verification image. Based on the swaying condition of sugarcane, it is judged whether the UAV needs to stay in the area and control the UAV to continue flying. When it is judged that the UAV stays, the area where multispectral image acquisition is being performed is defined as the verification area, and the multispectral acquisition module is controlled to repeatedly perform multispectral image acquisition on the verification area. Based on the verification image, the obscured sugarcane leaves are identified, which is achieved through existing image recognition technology, such as: multi-scale convolutional neural network, YOLOV8, DEEPLABV3+ and other plant leaf image segmentation technologies, and the multispectral image of the obscured sugarcane leaves is extracted, and the multispectral image of the obscured sugarcane leaves is positionally corrected; when the UAV continues to fly, the multispectral acquisition module is controlled to resume multispectral image acquisition of the uncollected sugarcane area.

[0032] The processing module is used to extract plant growth characteristics from the multispectral image and the corrected multispectral image, and generate corresponding characteristic indexes.

[0033] The evaluation module is used to receive the characteristic index and output the growth conditions and estimated yields in the sugarcane planting area. The growth conditions include the pest conditions, growth height conditions and nutrient conditions of sugarcane in each area, and the estimated yields include the estimated yields and estimated losses of sugarcane in each area.

[0034] The verification image acquisition module is used to extract the obscured sugarcane leaves from the verification image when the UAV is not stopping. When the range of the obscured sugarcane leaves exceeds the occlusion threshold, the corresponding area is marked as the area to be verified.

[0035] It also includes an environmental monitoring module for environmental monitoring of the sugarcane planting area, and the monitoring data includes wind direction and wind speed; when it is determined that the drone is staying, the wind speed is collected, and the collected wind speed data is defined as the re-collection wind speed threshold; when the wind speed data reaches the re-collection wind speed threshold, a first control signal is sent, and when the wind direction causes the obstructed blades to be unobstructed, a second control signal is sent. The collected wind speed threshold corresponding to the first control signal is used to determine whether the current wind speed is sufficient to cause the obstructed blades to sway, and whether the wind direction corresponding to the second control signal is sufficient to cause the obstructed blades to be exposed.

[0036] The drone is used to fly to the area to be verified and collect multispectral images when receiving the first control signal and the second control signal (the blocked sugarcane leaves are exposed and can be collected by multispectral images), obtain the multispectral image of the area to be verified, and update the multispectral image of the area not to be verified. The processing module is also used to extract plant growth characteristics from the updated multispectral image of the area to be verified.

[0037] The processing module is used to obtain the sugarcane planting area and establish a virtual coordinate system in the sugarcane planting area; when the wind speed does not meet the re-collection wind speed threshold, each sugarcane is marked with the corresponding coordinate point in the virtual coordinate system and defined as the initial coordinate of the sugarcane; based on the initial coordinate of the sugarcane, the multispectral image information collected after the sugarcane swaying is corrected and reset.

[0038] The processing module is used to stitch and correct the reflectance of the collected or corrected sugarcane multispectral images, perform reflectance correction on each band based on the correction plate with similar reflectance of sugarcane, generate index maps and digital surface models (DSM) for different bands, and calculate sugarcane indices, including normalized sugarcane index and chlorophyll index.

[0039] The processing module is used to create mask grids in different sugarcane index maps and DSMs, and multiply the sugarcane index maps of the same batch with the created mask grids in sequence to generate sugarcane index maps with soil background removed.

[0040] The processing module is used for extracting the sugarcane index of each area based on the sugarcane index map with soil background removed.

[0041] The evaluation module is used to output the growth conditions and estimated yields in the sugarcane planting area based on the machine learning regression method after receiving the normalized sugarcane index and chlorophyll index. Common machine learning regression methods include BP neural network, support vector machine, random forest, long short-term memory network (LSTM), etc. The evaluation module of this embodiment is constructed based on LSTM. LSTM can better capture the long-term dependencies in the sequence data through the cell state and gating mechanism, and retain the more distant context information, which is suitable for predicting the growth conditions and estimated yields of sugarcane. By obtaining the normalized sugarcane index and chlorophyll index, the predicted growth conditions and estimated yields of sugarcane are output to perform LSTM training, and the trained LSTM is used as an evaluation module to evaluate and predict the growth conditions and estimated yields of sugarcane.

[0042] The processing module is used to obtain the sugarcane planting area, perform multi-point position analysis on each sugarcane root in the sugarcane planting area, and obtain a number of offset points; when the variances of the coordinates of the several offset points and the initial coordinates of the sugarcane meet the error variance threshold, the area formed by the sugarcane in the multispectral image is corrected to the initial coordinates of the sugarcane.

[0043] Specific implementation process: First, the drone flies over the sugarcane planting area according to the preset path, and collects real-time images of the sugarcane planting area through the multispectral acquisition module and verification image acquisition module carried by the drone to obtain the first multispectral image and the first verification image; at the same time, the verification module analyzes the collected verification image to determine the swaying of the sugarcane; when the swaying amplitude of the sugarcane is greater than or equal to the preset swaying value or the range of the obscured sugarcane leaves exceeds the occlusion threshold, the drone stays in the monitoring area, and controls the multispectral acquisition module to repeatedly collect the area (verification area), and extract the multispectral image of the obscured sugarcane leaves from it, and then splices and corrects it with the first multispectral image to improve the accuracy of the sugarcane growth after subsequent image processing; and when the drone continues to fly, the multispectral acquisition module is controlled to resume the multispectral image acquisition of the uncollected sugarcane area to ensure the continuity of the entire monitoring process. When the drone stops for the first time, the environmental detection module will detect the wind speed and wind direction at this time, and use the wind speed at this time as the wind speed threshold for subsequent judgment of sugarcane swaying.

[0044] In addition, the verification module will also analyze the occlusion of the sugarcane leaves in the first verification image. When the range of the obscured sugarcane leaves exceeds the occlusion threshold and the occlusion duration is greater than the preset time, the area will be marked as the area to be verified, and then the drone will detect the next sugarcane area; after completing the multispectral image acquisition of the entire sugarcane planting area, the area to be verified still needs to acquire multispectral images to further correct the first multispectral image, so that the environmental monitoring module will monitor the wind speed in the sugarcane planting area in real time, and the environmental monitoring module will also monitor the wind direction. When the real-time wind speed meets the wind speed threshold and the wind direction blows from one side of the obscured sugarcane leaves to the other side, the drone will fly to the area to be detected in turn to perform repeated multispectral image acquisition, and when the wind speed and wind direction do not meet any of the above two conditions, the drone will return to the origin and stand by to save energy.

[0045] At the same time, the processing module establishes a virtual coordinate system in the sugarcane planting area, and marks the coordinates of each sugarcane when it is not swaying, forming a two-dimensional image of the sugarcane coordinates; at the same time, multi-point analysis is performed on the swaying sugarcane to obtain a number of offset points, and then each offset point is compared with its initial coordinates in the two-dimensional image for variance analysis to determine the offset amplitude, and then the position of the sugarcane area in the multispectral image is corrected so that its position coincides with the initial coordinates, further improving the monitoring and recognition accuracy.

[0046] After completing the multispectral image acquisition and correction of the sugarcane planting area, the corrected multispectral image is transferred to the processing module for processing; the processing steps are as follows: 1. Use DJI Terra software to stitch and correct the reflectivity of the multispectral image, and select a correction plate with a reflectivity similar to that of sugarcane to correct the reflectivity of each band, and generate each band index map and DSM; 2. Import the obtained band image into ArcMap and generate the corresponding file; 3. Create a mask grid to remove the soil background in each sugarcane index map, and obtain the sugarcane index map without the soil background.

[0047] Finally, the obtained sugarcane index (normalized sugarcane index and chlorophyll index) is input into the evaluation module to obtain the data of sugarcane growth and expected yield; the evaluation module is established by regression analysis (a statistical analysis method to determine the quantitative relationship between the degree of mutual dependence of two or more variables); at the same time, the regression model mainly uses the determination coefficient R 2 The accuracy of the model is evaluated by using the root mean square error (RMSE). 2 Reflects the stability of model establishment and verification, R 2 The closer it is to 1, the better the stability of the model and the higher the degree of fit. RMSE is used to evaluate the error and estimation accuracy of the model. The smaller the RMSE value, the better the model estimation ability. The formula is as follows:

[0048]

[0049] Where: y i represents the measured value, represents the predicted value, represents the mean of the measured values, and n is the number of samples.

[0050] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A sugarcane growth monitoring and evaluation device based on multispectral imaging, characterized in that: It includes a drone, a multispectral acquisition module and a verification image acquisition module, and the multispectral acquisition module and the verification image acquisition module are both mounted on the drone; Drones, which are used to fly along a preset path; Multispectral acquisition module, used to collect multispectral images of sugarcane planting areas; The verification image acquisition module is used to acquire verification images of the area where multispectral image acquisition is being performed, and extract the swaying condition of sugarcane from the verification image. Based on the swaying condition of sugarcane, it is determined whether the UAV needs to stay in the area and the UAV is controlled to continue flying. When it is determined that the UAV stays, the area where multispectral image acquisition is being performed is defined as the verification area, and the multispectral acquisition module is controlled to repeatedly acquire multispectral images of the verification area, and the multispectral image of the obscured sugarcane leaves is extracted therefrom, and the multispectral image of the obscured sugarcane leaves is corrected in position; when the UAV continues to fly, the multispectral acquisition module is controlled to resume acquiring multispectral images of the unacquired sugarcane area; A processing module, used for extracting plant growth characteristics from the multispectral image and the corrected multispectral image, and generating corresponding characteristic indexes; The evaluation module is used to receive the characteristic index and output the growth status and estimated yield in the sugarcane planting area.

2. The sugarcane growth monitoring and evaluation device based on multispectral imaging according to claim 1, characterized in that: The verification image acquisition module is used to extract the obscured sugarcane leaves from the verification image when the UAV is not stopping. When the range of the obscured sugarcane leaves exceeds the occlusion threshold, the corresponding area is marked as the area to be verified.

3. The sugarcane growth monitoring and evaluation device based on multispectral imaging according to claim 2, characterized in that: It also includes an environmental monitoring module for performing environmental monitoring of the sugarcane planting area, and the monitoring data includes wind direction and wind speed; when it is determined that the drone is stopping, the wind speed is collected, and the collected wind speed is defined as a re-collected wind speed threshold; when the wind speed data reaches the re-collected wind speed threshold, a first control signal is sent, and when the wind direction causes the blocked leaves to be unblocked, a second control signal is sent.

4. The sugarcane growth monitoring and evaluation device based on multispectral imaging according to claim 3, characterized in that: The drone is used to fly to the area to be verified when receiving the first control signal and the second control signal, collect multispectral images, obtain the multispectral images of the area to be verified, and update the multispectral images of the area to be verified. The processing module is also used to extract plant growth characteristics from the updated multispectral images of the area to be verified.

5. The sugarcane growth monitoring and evaluation device based on multispectral imaging according to claim 4, characterized in that: The processing module is used to obtain the sugarcane planting area and establish a virtual coordinate system in the sugarcane planting area; when the wind speed does not meet the re-collection wind speed threshold, each sugarcane is marked with the corresponding coordinate point in the virtual coordinate system, and the marked coordinate is defined as the initial coordinate of the sugarcane; based on the initial coordinate of the sugarcane, the multispectral image information collected after the sugarcane swaying is corrected and reset.

6. The sugarcane growth monitoring and evaluation device based on multispectral imaging according to claim 5, characterized in that: The processing module is used to stitch and correct the reflectance of the collected or corrected sugarcane multispectral images, perform reflectance correction on each band based on the correction plate with similar reflectance of sugarcane, generate index maps and DSM for different bands, and calculate sugarcane index, including normalized sugarcane index and chlorophyll index.

7. The sugarcane growth monitoring and evaluation device based on multispectral imaging according to claim 6, characterized in that: The processing module is used to create mask grids in different sugarcane index maps and DSMs, and multiply the sugarcane index maps of the same batch with the created mask grids in sequence to generate sugarcane index maps with soil background removed.

8. The sugarcane growth monitoring and evaluation device based on multispectral imaging according to claim 7, characterized in that: The processing module is used for extracting the sugarcane index of each area based on the sugarcane index map with soil background removed.

9. The sugarcane growth monitoring and evaluation device based on multispectral imaging according to claim 8, characterized in that: The evaluation module is used to output the growth conditions and expected yield in the sugarcane planting area based on the machine learning regression method after receiving the normalized sugarcane index and chlorophyll index.

10. The sugarcane growth monitoring and evaluation device based on multispectral imaging according to claim 9, characterized in that: The processing module is used to obtain the sugarcane planting area, perform multi-point position analysis on each sugarcane root in the sugarcane planting area, and obtain a number of offset points; when the variances of the coordinates of the several offset points and the initial coordinates of the sugarcane meet the error variance threshold, the area formed by the sugarcane in the multispectral image is corrected to the initial coordinates of the sugarcane.

Images