A sugarcane growth monitoring and evaluation device based on multispectral imaging

Abstract

The present invention discloses a sugarcane growth monitoring and assessment device based on multispectral imaging in the field of modern agricultural technology. The device comprises a drone, a multispectral acquisition module, and a verification image acquisition module. The verification image acquisition module is configured to acquire verification images of the area being acquired for multispectral image acquisition, extract the sway of the sugarcane from the verification images, and determine whether the drone needs to stay in the area based on the sway of the sugarcane. If the drone is determined to be staying, the area being acquired for multispectral image acquisition is defined as the verification area. The multispectral acquisition module is controlled to repeatedly acquire multispectral images of the verification area, extract multispectral images of obscured sugarcane leaves, and perform position correction on the multispectral images of obscured leaves. The present invention simultaneously acquires verification images while acquiring multispectral images, and corrects the multispectral images based on the correction images, thereby reducing the impact of sugarcane leaf obstruction on the accuracy of growth judgment and prediction.

Description

Technical Field

[0001] The present invention belongs to the field of modern agricultural technology, and in particular is a sugarcane growth monitoring and evaluation device based on multispectral imaging. Background Art

[0002] Sugarcane is one of the main raw materials in the sugar industry. Through extraction and refining, it can be produced into a variety of sugar products, including white sugar, brown sugar, and crystal sugar. Sugarcane sugar accounts for approximately 90% of my country's total sugar production. Sugarcane is also a key raw material for the production of fuel ethanol, with 40% of the world's fuel ethanol raw materials coming from sugarcane. Sugarcane can also be used to make beverages, candies, desserts, and other delicious foods. Sugarcane juice is a refreshing and thirst-quenching drink, while sugarcane wine is a nutritious and uniquely flavored alcoholic beverage. Furthermore, the sugars in sugarcane can be used to make paper, chemical products, and biological products.

[0003] The typical sugarcane planting process includes seedling cultivation, land preparation, planting, fertilization, and pest and disease control. Seedling cultivation typically takes place in spring, with healthy seedlings selected for cuttings or sowing. Land preparation requires deep tillage, adequate base fertilizer, and loose soil. During planting, the seedlings are placed in the soil, covered with a thin layer of soil, and then watered to maintain moisture. During the growing season, appropriate topdressing and weeding are required, along with pest and disease control. Advances in technology have led to the development of a variety of sugarcane growth monitoring and assessment devices to improve sugarcane yield and quality, precisely control pests and diseases, and optimize planting structures and management strategies. These devices primarily rely on modern technologies, including satellite remote sensing and drone remote sensing. Drone remote sensing, primarily using drone-mounted multispectral or hyperspectral sensors, can monitor soil moisture and temperature in real time, providing a scientific basis for sugarcane field irrigation. The high-definition image data collected can be used to construct a surface model of the sugarcane crop, thereby estimating growth parameters such as the leaf area index (LAI), enabling monitoring and assessment of sugarcane growth.

[0004] Existing sugarcane growth monitoring and evaluation devices, when monitoring and evaluating the sugarcane growing area, directly collect corresponding image information through multispectral imaging instruments, and directly analyze the multispectral image information to judge and predict the sugarcane growth situation. However, in the sugarcane growing area, due to the high density of sugarcane planting, there are some sugarcane leaves that are blocked by adjacent sugarcanes, especially those with large growth differences. The growth and development status of the blocked leaves is difficult to be collected by the multispectral imaging instrument. For example, some sugarcane leaves with insect pests or poor growth and development are blocked by some sugarcane leaves with good growth and development. When the multispectral imaging instrument collects data, the multispectral image information of the leaves with good growth and development on the surface will be collected; thus, errors will occur in the subsequent judgment and prediction of the sugarcane growth situation.

[0005] In summary, the present invention proposes a sugarcane growth monitoring and evaluation device based on multispectral imaging. When acquiring multispectral images, a correction image is also acquired. The multispectral image is corrected based on the correction image to reduce the impact of sugarcane leaf occlusion on the judgment and prediction accuracy of its growth condition. Summary of the Invention

[0006] In order to solve the above problems, the purpose of the present invention is to provide a sugarcane growth monitoring and evaluation device based on multispectral imaging. By collecting verification images at the same time as multispectral images, the multispectral images are corrected based on the correction images, thereby reducing the impact of sugarcane leaf occlusion on the growth judgment and prediction accuracy.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows: a sugarcane growth monitoring and evaluation device based on multispectral imaging, comprising a drone, a multispectral acquisition module, and a verification image acquisition module, both of which are mounted on the drone;

[0008] drones, which are used to fly along a preset path;

[0009] Multispectral acquisition module, used to collect multispectral images of sugarcane planting areas;

[0010] The verification image acquisition module is used to acquire verification images of the area where multispectral image acquisition is being performed, extract the swaying condition of the sugarcane from the verification images, determine whether the drone needs to stay in the area based on the swaying condition of the sugarcane, and control the drone to continue flying. When it is determined that the drone needs to stay, the area where multispectral image acquisition is being performed is defined as the verification area, and the multispectral acquisition module is controlled to repeatedly acquire multispectral images of the verification area, extract the multispectral images of the obscured sugarcane leaves from the verification area, and perform position correction on the multispectral images of the obscured sugarcane leaves; when the drone continues to fly, the multispectral acquisition module is controlled to resume multispectral image acquisition of the unacquired sugarcane area;

[0011] A processing module, configured to extract plant growth features from the multispectral image and the corrected multispectral image, and generate corresponding feature indices;

[0012] The evaluation module is used to receive characteristic indices and output the growth status and estimated yield of sugarcane in the sugarcane planting area.

[0013] Furthermore, the verification image acquisition module is used to extract the obscured sugarcane leaves from the verification image when the drone is not stopping. When the range of the obscured sugarcane leaves exceeds the occlusion threshold, the corresponding area is marked as the area to be verified.

[0014] Furthermore, it also includes an environmental monitoring module for performing environmental monitoring of the sugarcane planting area, and the monitoring data includes wind direction and wind speed; when it is determined that the drone is stopping, the wind speed is collected, and the collected wind speed data is defined as the re-collection wind speed threshold; when the wind speed data reaches the re-collection wind speed threshold, a first control signal is sent, and when the wind direction causes the blocked leaves to be unblocked, a second control signal is sent.

[0015] Furthermore, the drone is used to fly to the area to be verified when it receives the first control signal and the second control signal, collect multispectral images, obtain multispectral images of the area to be verified, and update the multispectral images of the area not to be verified. The processing module is also used to extract plant growth characteristics from the updated multispectral images of the area to be verified.

[0016] Furthermore, the processing module is used to obtain the sugarcane planting area and establish a virtual coordinate system within the sugarcane planting area; when the wind speed does not meet the wind speed threshold for re-collection, the corresponding coordinate point of each sugarcane is marked in the virtual coordinate system, and the marked coordinate is defined as the initial coordinate of the sugarcane; based on the initial coordinate of the sugarcane, the multispectral image information collected after the sugarcane swaying is corrected and reset.

[0017] Furthermore, the processing module is used to stitch and perform reflectance correction on the collected or corrected sugarcane multispectral images, perform reflectance correction on each band based on a correction plate with similar sugarcane reflectance, generate index maps and DSMs for different bands, and calculate sugarcane indices, including the normalized sugarcane index and chlorophyll index.

[0018] Furthermore, the processing module is used to create mask grids in different sugarcane index maps and DSMs, and multiply the sugarcane index maps of the same batch with the created mask grids in sequence to generate sugarcane index maps with the soil background removed.

[0019] Furthermore, the processing module is used to extract the sugarcane index of each area based on the sugarcane index map with the soil background removed.

[0020] Furthermore, the evaluation module is used to output the growth status and expected yield in the sugarcane planting area based on a machine learning regression method after receiving the normalized sugarcane index and chlorophyll index.

[0021] Furthermore, the processing module is used to obtain the sugarcane planting area, perform multi-point position analysis on each sugarcane root in the sugarcane planting area, and obtain a number of offset points; when the variance of the coordinates of the several offset points and the initial coordinates of the sugarcane meets the error variance threshold, the area formed by the sugarcane in the multispectral image is corrected to the initial coordinates of the sugarcane.

[0022] The following beneficial effects were achieved by adopting the above scheme: (1) This scheme, through the use of a multispectral acquisition module equipped on a drone, can efficiently and extensively acquire multispectral images of the sugarcane planting area. Compared with the traditional manual monitoring method, it greatly improves the accuracy and efficiency of monitoring. At the same time, by verifying that the image acquisition module can judge the swaying of the sugarcane and the obstruction of the leaves in real time, it ensures the accuracy of the multispectral image acquisition and avoids image distortion caused by environmental factors. Based on the swaying of the sugarcane and the obstruction of the leaves, the drone can intelligently determine whether it needs to stay in a specific area for repeated acquisition and whether it needs to correct the obstructed multispectral image, thus achieving dynamic adjustment and optimization during the monitoring process.

[0023] (2) This solution introduces an environmental monitoring module that can obtain wind direction and wind speed data in real time, providing a more accurate control basis for the UAV's flight and image acquisition.

[0024] (3) This solution uses a processing module to extract plant growth characteristics from collected multispectral images and generate corresponding characteristic indices, providing accurate data support for the subsequent evaluation module. The evaluation module, based on machine learning regression methods, can accurately predict growth conditions and expected yields within sugarcane planting areas, providing a scientific basis for decision-making in agricultural production.

[0025] (4) This solution makes the entire monitoring and evaluation process highly automated, reducing human intervention and errors, and improving the objectivity and accuracy of the monitoring results. Furthermore, by establishing a virtual coordinate system and marking the initial coordinates of the sugarcane, the growth and sway of the sugarcane can be accurately tracked, reducing monitoring errors caused by offset.

[0026] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Schematic diagram of the process of the sugarcane growth monitoring and evaluation device based on multispectral imaging in this embodiment. DETAILED DESCRIPTION

[0028] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0029] The following is further described in detail through specific implementation methods:

[0030] Implementation example Figure 1 The device, shown here, uses multispectral imaging to monitor and evaluate sugarcane growth. It consists of a drone, a multispectral acquisition module, and a verification image acquisition module, both of which are mounted on the drone. The multispectral acquisition module is a multispectral camera, and the image acquisition module is a drone camera. The drone is designed to fly along a pre-set path, while the multispectral acquisition module is used to capture multispectral images of the sugarcane planting area.

[0031] The verification image acquisition module is used to perform verification image acquisition on the area where multispectral image acquisition is being performed, and extract the swaying condition of sugarcane from the verification image. Based on the swaying condition of sugarcane, it is judged whether the drone needs to stay in the area and the drone is controlled to continue flying. When it is judged that the drone is staying, the area where multispectral image acquisition is being performed is defined as the verification area, and the multispectral acquisition module is controlled to repeatedly perform multispectral image acquisition on the verification area. The obscured sugarcane leaves are identified based on the verification image, which is achieved through existing image recognition technology, such as: multi-scale convolutional neural network, YOLOV8, DEEPLABV3+ and other plant leaf image segmentation technologies, and the multispectral image of the obscured sugarcane leaves is extracted from it, and the multispectral image of the obscured sugarcane leaves is positionally corrected; when the drone continues to fly, the multispectral acquisition module is controlled to resume multispectral image acquisition of the uncollected sugarcane area.

[0032] The processing module is used to extract plant growth features from the multispectral image and the corrected multispectral image, and generate corresponding feature indexes.

[0033] The evaluation module receives characteristic indices and outputs growth conditions and estimated yields within the sugarcane planting area. Growth conditions include pest infestation, growth height, and nutrient availability for each sugarcane area. Yield estimates include estimated yield and estimated sugarcane loss for each area.

[0034] The verification image acquisition module is used to extract the obscured sugarcane leaves from the verification image when the drone is not stopping. When the range of the obscured sugarcane leaves exceeds the occlusion threshold, the corresponding area is marked as the area to be verified.

[0035] The system also includes an environmental monitoring module for monitoring the environment of the sugarcane planting area. The monitoring data includes wind direction and wind speed. When the drone is determined to be stationary, the wind speed is collected and the collected wind speed data is defined as the re-collection wind speed threshold. When the wind speed data reaches the re-collection wind speed threshold, a first control signal is transmitted. When the wind direction causes the obscured leaves to be unobstructed, a second control signal is transmitted. The collected wind speed threshold corresponding to the first control signal is used to determine whether the current wind speed is sufficient to cause the obscured leaves to sway, and whether the wind direction corresponding to the second control signal is sufficient to expose the obscured leaves.

[0036] When the drone receives the first control signal and the second control signal (the obscured sugarcane leaves are exposed and can be captured by multispectral images), it flies to the area to be verified, collects multispectral images, obtains multispectral images of the area to be verified, and updates the multispectral images of the area not to be verified. The processing module is also used to extract plant growth characteristics from the updated multispectral images of the area to be verified.

[0037] The processing module is used to obtain the sugarcane planting area and establish a virtual coordinate system within the sugarcane planting area; when the wind speed does not meet the re-collection wind speed threshold, the corresponding coordinate point of each sugarcane is marked in the virtual coordinate system and defined as the initial coordinate of the sugarcane; based on the initial coordinate of the sugarcane, the multispectral image information collected after the sugarcane swaying is corrected and reset.

[0038] The processing module is used to stitch and perform reflectance correction on the collected or corrected sugarcane multispectral images, perform reflectance correction on each band based on a correction plate with similar sugarcane reflectance, generate index maps and digital surface models (DSM) for different bands, and calculate sugarcane indices, including the normalized sugarcane index and chlorophyll index.

[0039] The processing module is used to create mask grids in different sugarcane index maps and DSMs, and multiply the sugarcane index maps of the same batch with the created mask grids in sequence to generate sugarcane index maps with soil background removed.

[0040] The processing module is used to extract the sugarcane index of each area based on the sugarcane index map with the soil background removed.

[0041] The evaluation module is used to output the growth status and estimated yield of the sugarcane planting area based on machine learning regression methods after receiving the normalized sugarcane index and chlorophyll index. Common machine learning regression methods include BP neural networks, support vector machines, random forests, long short-term memory networks (LSTMs), etc. The evaluation module of this embodiment is constructed based on LSTM. LSTM, through cell states and gating mechanisms, can better capture long-term dependencies in sequence data and retain more distant contextual information, making it suitable for predicting sugarcane growth and estimated yield. By obtaining the normalized sugarcane index and chlorophyll index, the predicted sugarcane growth and estimated yield are output and LSTM training is performed. The trained LSTM is used as the evaluation module to evaluate and predict sugarcane growth and estimated yield.

[0042] The processing module is used to obtain the sugarcane planting area and perform multi-point position analysis on each sugarcane root in the sugarcane planting area to obtain several offset points. When the variance between the coordinates of the several offset points and the initial coordinates of the sugarcane meets the error variance threshold, the area formed by the sugarcane in the multispectral image is corrected to the initial coordinates of the sugarcane.

[0043] The specific implementation process is as follows: First, the drone flies over the sugarcane planting area according to a preset path. The drone uses its multispectral acquisition module and verification image acquisition module to collect real-time images of the sugarcane planting area, generating a first multispectral image and a first verification image. The verification module then analyzes the verification image to determine the sway of the sugarcane. If the sugarcane sway amplitude is greater than or equal to a preset sway value or the range of obscured sugarcane leaves exceeds an occlusion threshold, the drone stops within the monitoring area and controls the multispectral acquisition module to repeatedly acquire images in the same area (the verification area). The multispectral image of the obscured sugarcane leaves is then stitched and corrected with the first multispectral image to improve the accuracy of the sugarcane growth information obtained after subsequent image processing. Furthermore, when the drone resumes flight, the multispectral acquisition module is controlled to resume multispectral image acquisition of the unacquired sugarcane areas, ensuring the continuity of the entire monitoring process. During the drone's first stop, the environmental detection module detects the wind speed and direction at that time and uses the wind speed as the wind speed threshold for subsequent sugarcane sway determination.

[0044] In addition, the verification module will also analyze the occlusion of the sugarcane leaves in the first verification image. When the range of the obscured sugarcane leaves exceeds the occlusion threshold and the occlusion duration is greater than the predetermined time, the area will be marked as the area to be verified, and then the drone will proceed to detect the next sugarcane area. After completing the multispectral image acquisition of the entire sugarcane planting area, the area to be verified still needs to acquire multispectral images to further correct the first multispectral image. Therefore, the environmental monitoring module will monitor the wind speed in the sugarcane planting area in real time. At the same time, the environmental monitoring module will also monitor the wind direction. When the real-time wind speed meets the wind speed threshold and the wind direction blows from one side of the obscured sugarcane leaves to the other side, the drone will fly to the area to be detected in turn to perform repeated multispectral image acquisition. When the wind speed and wind direction do not meet any of the above two conditions, the drone will return to the origin and stand by to save energy.

[0045] At the same time, the processing module establishes a virtual coordinate system in the sugarcane planting area and marks the coordinates of each sugarcane when it is not swaying, forming a two-dimensional image of the sugarcane coordinates; at the same time, multi-point position analysis is performed on the swaying sugarcane to obtain several offset points, and then each offset point is compared with its initial coordinates in the two-dimensional image for variance analysis to determine its offset amplitude, and then the position of the sugarcane area in the multispectral image is corrected so that its position coincides with the initial coordinates, further improving the monitoring and recognition accuracy.

[0046] After completing the multispectral image acquisition and correction of the sugarcane planting area, the corrected multispectral images were transferred to the processing module for processing. The processing steps are as follows: First, the multispectral images were stitched and reflectance corrected using DJI Terra software, and a correction plate with a reflectance close to that of sugarcane was selected to correct the reflectance of each band to generate index maps for each band and DSM; Second, the obtained band images were imported into ArcMap and the corresponding files were generated; Third, a mask raster was created to remove the soil background from each sugarcane index map to obtain the sugarcane index map without the soil background.

[0047] Finally, the obtained sugarcane index (normalized sugarcane index and chlorophyll index) is input into the evaluation module to obtain the data of sugarcane growth and expected yield; the evaluation module is established by regression analysis (a statistical analysis method that determines the quantitative relationship between the degree of mutual dependence of two or more variables); and the regression model mainly uses the coefficient of determination R 2 The accuracy of the model is evaluated by the root mean square error (RMSE). 2 Reflects the stability of model establishment and verification, R 2 The closer it is to 1, the better the stability of the model and the higher the degree of fit. RMSE is used to evaluate the error and estimation accuracy of the model. The smaller the RMSE value, the better the model estimation ability. The formula is as follows:

[0048]

[0049] Where: y i represents the measured value, represents the predicted value, represents the mean of the measured values, and n is the number of samples.

[0050] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will readily appreciate that other variations or modifications based on the above descriptions are possible. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A sugarcane growth monitoring and evaluation device based on multispectral imaging, characterized in that: It includes a drone, a multispectral acquisition module and a verification image acquisition module, and the multispectral acquisition module and the verification image acquisition module are both carried on the drone; drones, which are used to fly along a preset path; Multispectral acquisition module, used to collect multispectral images of sugarcane planting areas; The verification image acquisition module is used to acquire verification images of the area where multispectral image acquisition is being performed, extract the swaying condition of the sugarcane from the verification images, determine whether the drone needs to stay in the area based on the swaying condition of the sugarcane, and control the drone to continue flying. When it is determined that the drone needs to stay, the area where multispectral image acquisition is being performed is defined as the verification area, and the multispectral acquisition module is controlled to repeatedly acquire multispectral images of the verification area, extract the multispectral images of the obscured sugarcane leaves from the verification area, and perform position correction on the multispectral images of the obscured sugarcane leaves; when the drone continues to fly, the multispectral acquisition module is controlled to resume acquiring multispectral images of the unacquired sugarcane area; A processing module, for extracting plant growth features from the multispectral image and the corrected multispectral image, and generating corresponding feature indices; The evaluation module is used to receive characteristic indices and output the growth status and estimated yield of sugarcane in the sugarcane planting area.

2. The sugarcane growth monitoring and evaluation device based on multispectral imaging according to claim 1, characterized in that: The verification image acquisition module is used to extract the obscured sugarcane leaves from the verification image when the drone is not stopping. When the range of the obscured sugarcane leaves exceeds the occlusion threshold, the corresponding area is marked as the area to be verified.

3. The sugarcane growth monitoring and evaluation device based on multispectral imaging according to claim 2, characterized in that: It also includes an environmental monitoring module for performing environmental monitoring of the sugarcane planting area. The monitoring data includes wind direction and wind speed. When it is determined that the drone is stopping, the wind speed is collected, and the collected wind speed is defined as the re-collection wind speed threshold. When the wind speed data reaches the re-collection wind speed threshold, a first control signal is sent. When the wind direction causes the blocked leaves to be unblocked, a second control signal is sent.

4. The sugarcane growth monitoring and evaluation device based on multispectral imaging according to claim 3, characterized in that: When receiving the first control signal and the second control signal, the drone is used to fly to the area to be verified, collect multispectral images, obtain the multispectral images of the area to be verified, and update the multispectral images of the area to be verified. The processing module is also used to extract plant growth characteristics from the updated multispectral images of the area to be verified.

5. The sugarcane growth monitoring and evaluation device based on multispectral imaging according to claim 4, characterized in that: The processing module is used to obtain the sugarcane planting area and establish a virtual coordinate system within the sugarcane planting area; when the wind speed does not meet the re-collection wind speed threshold, the corresponding coordinate point of each sugarcane is marked in the virtual coordinate system, and the marked coordinates are defined as the initial coordinates of the sugarcane; based on the initial coordinates of the sugarcane, the multispectral image information collected after the sugarcane swaying is corrected and reset.

6. The sugarcane growth monitoring and evaluation device based on multispectral imaging according to claim 5, characterized in that: The processing module is used to stitch and perform reflectance correction on the collected or corrected sugarcane multispectral images, perform reflectance correction on each band based on a correction plate with similar sugarcane reflectance, generate index maps and DSMs for different bands, and calculate sugarcane indices, including the normalized sugarcane index and chlorophyll index.

7. The sugarcane growth monitoring and evaluation device based on multispectral imaging according to claim 6, characterized in that: The processing module is used to create mask grids in different sugarcane index maps and DSMs, and multiply the sugarcane index maps of the same batch with the created mask grids in sequence to generate sugarcane index maps with soil background removed.

8. The sugarcane growth monitoring and evaluation device based on multispectral imaging according to claim 7, characterized in that: The processing module is used to extract the sugarcane index of each area based on the sugarcane index map with the soil background removed.

9. The sugarcane growth monitoring and evaluation device based on multispectral imaging according to claim 8, characterized in that: The evaluation module is used to output the growth status and expected yield in the sugarcane planting area based on the machine learning regression method after receiving the normalized sugarcane index and chlorophyll index.

10. The sugarcane growth monitoring and evaluation device based on multispectral imaging according to claim 9, characterized in that: The processing module is used to obtain the sugarcane planting area and perform multi-point position analysis on each sugarcane root in the sugarcane planting area to obtain several offset points. When the variance between the coordinates of the several offset points and the initial coordinates of the sugarcane meets the error variance threshold, the area formed by the sugarcane in the multispectral image is corrected to the initial coordinates of the sugarcane.

Images