Method for evaluating sugarcane smut based on unmanned aerial vehicle image

Abstract

The invention relates to the field of agriculture, and discloses a method for evaluating sugarcane smut based on an unmanned aerial vehicle image, which comprises the following steps: data acquisition: planning a route by using an unmanned aerial vehicle carrying a multispectral camera, a hyperspectral camera and an RGB (Red, Green and Blue) camera, so that the unmanned aerial vehicle acquires image data of sugarcane; preprocessing: sequentially carrying out geometric correction, radiation correction and image fusion processing on the image data to obtain preprocessed data; feature fusion: constructing a fusion model including spectral feature extraction, visual feature extraction and spatial-temporal feature fusion according to the feature expression of the sugarcane smut; and model generalization: collecting multi-source images and disease investigation data according to the actual condition of the sugarcane. The method comprises the following steps: performing data acquisition through an unmanned aerial vehicle, obtaining multi-dimensional image data in a sugarcane growth critical period, performing preprocessing, constructing a fusion model including spectral feature extraction, visual feature extraction and spatial-temporal feature fusion, performing fine adjustment and optimization in combination with a transfer learning strategy, and performing real-time analysis on a disease detection result.

Description

Technical Field

[0001] The present invention relates to the field of agricultural technology, and in particular to a method for evaluating sugarcane smut based on unmanned aerial vehicle (UAV) images. Background Art

[0002] In modern agricultural production, the development of precision agriculture technologies plays a key role in the monitoring and prevention of crop pests and diseases. With the increasing scale and intensification of agriculture, the rapid detection and effective prevention of crop pests and diseases have become crucial for ensuring agricultural production safety and improving the quality of agricultural products. Unmanned aerial vehicle (UAV) remote sensing technology, with its advantages of high maneuverability, high resolution, low cost, and repeatable operation, is increasingly being used in agriculture, providing a new technical means for monitoring crop growth and detecting pests and diseases.

[0003] Currently, sugarcane smut is one of the major diseases in sugarcane cultivation, severely impacting both yield and quality. Traditional methods for detecting sugarcane smut rely primarily on manual field inspections. Technicians visually inspect sugarcane plants for external characteristics such as morphology and color to determine if they are diseased. However, manual inspections are inefficient, making rapid testing difficult across large sugarcane planting areas and failing to meet the real-time monitoring needs of these areas. Summary of the Invention

[0004] In response to the shortcomings of the existing technology, the present invention provides a method for assessing sugarcane smut based on drone images, which solves the problem that traditional sugarcane smut detection methods cannot meet the real-time monitoring needs of large sugarcane planting areas.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: A method for assessing sugarcane smut based on drone images, comprising the following steps:

[0006] Data collection: Using drones equipped with multispectral cameras, hyperspectral cameras, and RGB cameras, we planned routes to collect image data of sugarcane.

[0007] Preprocessing: Perform geometric correction, radiation correction and image fusion processing on the image data in sequence to obtain preprocessed data;

[0008] Feature fusion: Based on the characteristic manifestations of sugarcane smut, a fusion model is constructed that includes spectral feature extraction, visual feature extraction, and spatiotemporal feature fusion;

[0009] Model generalization: Based on the actual situation of sugarcane, we collected multi-source images and disease survey data to construct a multi-source dataset. We then used a transfer learning strategy to fine-tune and optimize the fusion model to obtain a detection model.

[0010] Extraction and analysis: The detection model is used to extract features and perform fusion analysis on the preprocessed data to obtain disease detection results containing multimodal features;

[0011] Real-time early warning: Through the air-ground collaborative computing architecture, disease detection results are analyzed in real time, and early warnings are triggered and prevention and control decisions are generated based on set conditions.

[0012] By adopting the above technical solutions, we can leverage the advantages of multi-source data and multimodal fusion to comprehensively and accurately extract the spectral, visual, and time series characteristic information of sugarcane smut. With the help of transfer learning, the model can be adapted to different production environments. The air-ground collaborative computing architecture is used to achieve rapid data processing and analysis, realizing high-precision, real-time monitoring of large sugarcane planting areas. At the same time, it improves detection efficiency and accuracy, and solves the problem that traditional sugarcane smut detection methods cannot meet the real-time monitoring needs of large sugarcane planting areas, providing reliable technical support for the scientific prevention and control of sugarcane smut.

[0013] Preferably, the flight altitude of the planned route is 30-50 meters, the heading overlap is ≥80%, and the lateral overlap is ≥60%. The image data includes multispectral image data, hyperspectral image data, and RGB image data. The multispectral image data includes spectral information of blue light, green light, red light, red edge, and near-infrared specific bands. The spectral coverage range of the hyperspectral image data is 400-1000nm. The RGB image data includes visual characteristics of the texture, shape, and color of sugarcane.

[0014] Preferably, in the radiation correction, the reflectance calculation formula of the multispectral data is Among them, DN is the pixel gray value, Gain and Bias are camera calibration parameters, E solar is the solar irradiance, θ is the solar zenith angle, and the image fusion processing is to perform spectral resampling on the hyperspectral image after radiation correction so that its spectral range is consistent with the RGB image data, perform principal component analysis on the RGB image data to obtain the principal component, perform spectral stretching on the hyperspectral image data so that its mean and variance are consistent with the principal component, and then replace and fuse the stretched hyperspectral image data with the principal component to obtain the preprocessed data.

[0015] Preferably, the characteristic manifestations include the characteristic manifestations of sugarcane smut in spectrum, vision, and visual sequence. In the spectral feature extraction, a continuous projection algorithm and a competitive adaptive reweighting algorithm are used to screen sensitive bands in hyperspectral data. A regression model of spectrum and disease severity is established based on partial least squares regression. The regression model is expressed as Y=XB+E, wherein Y is the disease severity variable, X is the spectral variable matrix, B is the regression coefficient matrix, and E is the residual matrix. In the visual feature extraction, EfficientNet is used as the backbone network, and a spatial pyramid pooling module and a bidirectional feature pyramid network are embedded, and channel attention, spatial attention and self-attention mechanisms are combined. In the spatiotemporal feature fusion, a gated recurrent unit is used to construct a temporal series model.

[0016] Preferably, the channel attention learns the dependencies between channels through global average pooling and two fully connected layers to calculate the channel weight w c The formula is wc=σ(W2δ(W1GAP(X))), where GAP is the global average pooling, W1 and W2 are the weights of the fully connected layer, δ is the activation function, and σ is the Sigmoid function.

[0017] Preferably, the actual conditions include the actual conditions of different ecological environments, planting patterns, and sugarcane varieties of sugarcane; the multi-source images and disease survey data include sugarcane image data of different production areas, different growth stages, and different disease severity, as well as corresponding disease levels, disease locations, and sugarcane variety survey data; and constructing the multi-source data set includes pixel-level annotation of the multi-source images and disease survey data using Labelme software, wherein the annotation content includes diseased plant location, disease location, disease level, and sugarcane variety information.

[0018] Preferably, the fine-tuning is to select a fusion model pre-trained on ImageNet, and globally fine-tune the fusion model using multi-source data sets to learn the common characteristics of sugarcane plants and the common characteristics of diseases in different production areas, and then perform secondary fine-tuning for the target production area using local labeled data to optimize the characteristics related to the local environment and variety. The optimization is to use indicators to evaluate the performance of the fusion model in different production areas, and to fuse several fusion models trained on different data sets through an ensemble learning method. The indicators include accuracy, recall rate, F1 value, and average precision mean. The ensemble learning method includes bagging and boosting.

[0019] Preferably, the feature extraction is to extract spectral features from hyperspectral data through spectral feature extraction in the detection model, including sensitive bands and calculating related vegetation indices, extract visual features of sugarcane plants through visual feature extraction, and extract spatiotemporal variation characteristics of sugarcane smut in time and space through spatiotemporal feature fusion, wherein the visual features include texture, shape, and color. The fusion analysis is to fuse spectral features, visual features, and spatiotemporal variation features to comprehensively judge whether sugarcane is infected with smut and the severity of the disease.

[0020] Preferably, the air-ground collaborative computing architecture includes an edge computing module and a cloud server. The edge computing module is integrated on the drone and deploys a lightweight detection model to perform real-time processing and preliminary analysis of image data, locate suspected diseased plants, and transmit their information to the cloud server through the network. After receiving the data transmitted by the edge end, the cloud server uses a distributed computing framework to store, manage and deeply analyze large-scale data, construct a spatiotemporal distribution model of the disease, and predict the spread trend of the disease.

[0021] Preferably, the real-time analysis is based on the spatiotemporal distribution model and prediction results of the disease constructed by the cloud server, combined with the current disease detection results, to evaluate the occurrence and development trend of sugarcane smut in real time. The set conditions include multi-level early warning conditions and relevant conditions for predicting future disease outbreak risks based on time-sensitive early warnings combined with meteorological data.

[0022] The present invention provides a method for assessing sugarcane smut based on drone images. It has the following beneficial effects:

[0023] 1. The present invention uses drones to collect data, acquire multi-dimensional image data during the critical growth period of sugarcane, and performs preprocessing. It then constructs a fusion model that includes spectral feature extraction, visual feature extraction, and spatiotemporal feature fusion. Fine-tuning and optimization are combined with a transfer learning strategy. Using an air-ground collaborative computing architecture, the system performs real-time analysis of disease detection results, thereby extracting the characteristics of sugarcane smut in the target production area. This enables rapid and accurate real-time monitoring of large sugarcane planting areas, addressing the problem that traditional sugarcane smut detection methods cannot meet the real-time monitoring needs of large sugarcane planting areas.

[0024] 2. This invention eliminates image distortion caused by factors such as drone flight through geometric correction, ensuring accurate image positioning. Radiometric correction converts pixel grayscale values ​​into actual reflectance through a specific formula, taking into account influencing factors such as solar irradiance and zenith angle, making data collected under different conditions comparable and truly reflecting the spectral characteristics of sugarcane. Image fusion combines the advantages of hyperspectral and RGB images to generate data with both high spectral and high spatial resolution, providing high-quality data for subsequent feature extraction and analysis, effectively improving the accuracy and stability of sugarcane smut assessment.

[0025] 3. The present invention screens sensitive bands through spectral feature extraction and establishes a regression model to explore the relationship between the spectrum and the degree of disease. Visual feature extraction uses advanced network architecture and attention mechanism to enhance the capture of visual features of diseased plants. Spatiotemporal feature fusion uses a specific model to learn the spatiotemporal variation patterns of diseases. Multimodal feature fusion enables the model to comprehensively analyze disease characteristics. Compared with single-modal detection methods, it can not only identify diseases more accurately, but also distinguish different degrees of severity, effectively improving detection accuracy. At the same time, it enhances the model's adaptability to complex disease conditions and different environmental conditions, reducing missed and misjudgment.

[0026] 4. The present invention constructs a multi-source dataset based on actual sugarcane planting conditions, covering data from different ecological environments, planting patterns, and sugarcane varieties. It also uses a transfer learning strategy to fine-tune and optimize the fusion model. Global fine-tuning allows the model to learn universal characteristics of sugarcane and common diseases. Secondary fine-tuning optimizes the local environment and variety. Combined with ensemble learning, multiple models are integrated to enable the detection model to adapt to the differences between different production areas. It can be quickly applied in new environments without extensive retraining, reducing model application costs and time, improving model versatility and practicality, and providing stable and effective technical support for sugarcane smut prevention and control in different regions, thereby expanding the scope of application of the technology.

[0027] 5. The present invention processes image data through edge computing, locates suspected diseased plants and transmits key information. After receiving the data, the cloud server conducts in-depth analysis, constructs a spatiotemporal distribution model of the disease to predict trends, combines multi-level warning conditions and time-sensitive warnings, assesses disease risks in real time based on multi-source data, sends warning information, and generates personalized prevention and control plans. This realizes real-time monitoring of sugarcane smut, helps farmers and management departments take timely measures in the early stages of the disease, effectively controls the spread of the disease, reduces sugarcane yield losses, improves prevention and control efficiency and economic benefits, and ensures the healthy and sustainable development of the sugarcane industry.

[0028] 6. The present invention utilizes drones equipped with multispectral cameras, hyperspectral cameras, and RGB cameras for data collection and plans specific flight path parameters to acquire multidimensional image data during the critical growth period of sugarcane. The multispectral camera provides spectral information in specific bands, the hyperspectral camera covers a continuous spectral range, and the RGB camera records visual features. The combination of these three enables the collected data to cover both spectral and visual information. Compared with traditional manual detection or single-sensor detection, this improves data collection efficiency and can quickly cover large sugarcane planting areas. It also provides rich and comprehensive data for subsequent analysis, reduces detection errors caused by missing data, and lays a solid foundation for the accurate assessment of sugarcane smut. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1This is a flow chart of the method for assessing sugarcane smut based on drone images proposed in the present invention. DETAILED DESCRIPTION

[0030] The following will clearly and completely describe the technical solution of the present invention in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0031] Please see the attached Figure 1 The embodiment of the present invention provides a method for assessing sugarcane smut based on drone images, comprising the following steps:

[0032] Data collection: Using drones equipped with multispectral cameras, hyperspectral cameras, and RGB cameras, routes are planned to collect image data of sugarcane. The planned flight altitude is 30-50 meters, the heading overlap is ≥80%, and the lateral overlap is ≥60%. The image data includes multispectral image data, hyperspectral image data, and RGB image data. The multispectral image data includes spectral information of specific bands such as blue light, green light, red light, red edge, and near-infrared. The spectral coverage range of the hyperspectral image data is 400-1000nm. The RGB image data includes the visual characteristics of the texture, shape, and color of the sugarcane.

[0033] Specifically, during the data collection phase, the DJI Matrice 350 RTK drone was selected as the data collection platform. This drone is equipped with a Micasense RedEdge-MX multispectral camera, a Headwall Photonics Nano-Hyperspec hyperspectral camera, and an RGB camera with over 20 megapixels. During key sugarcane growth stages (such as tillering, elongation, and maturity), flight paths were planned using DJI Terra software. The drone flew in a grid pattern at an altitude of 30-50 meters, maintaining a heading overlap of ≥80% and a lateral overlap of ≥60%, completing image data collection.

[0034] The multispectral camera captures multispectral image data covering five specific wavelength bands: blue (475nm), green (560nm), red (668nm), red edge (717nm), and near-infrared (842nm). The hyperspectral camera captures hyperspectral image data from 400-1000nm, providing continuous spectral information. The RGB camera captures visual characteristics such as texture, shape, and color of the sugarcane plants. This multi-camera configuration and acquisition strategy enables the acquisition of sugarcane growth status data from multiple spectral and visual dimensions, providing a rich data foundation for subsequent disease assessment.

[0035] Preprocessing: The image data is processed in sequence with geometric correction, radiation correction and image fusion to obtain preprocessed data; in radiation correction, the reflectance calculation formula of multispectral data is: Among them, DN is the pixel gray value, Gain and Bias are camera calibration parameters, E solar is the solar irradiance, θ is the solar zenith angle, and the image fusion processing is to spectrally resample the hyperspectral image after radiation correction to make its spectral range consistent with the RGB image data, and perform principal component analysis on the RGB image data to obtain the principal component, and spectrally stretch the hyperspectral image data to make its mean and variance consistent with the principal component, and then replace and fuse the stretched hyperspectral image data with the principal component to obtain the preprocessed data.

[0036] Specifically, the high-precision positioning data obtained by the RTK module on the drone is used, and a polynomial transformation model is adopted in the ENVI software. Combined with the ground control points, the coordinate transformation relationship is constructed to improve the image positioning accuracy to the sub-meter level, eliminating the geometric distortion caused by factors such as the drone's flight attitude and terrain undulations.

[0037] For multispectral data, according to the formula Calculate the reflectivity. DN is the pixel grayscale value, which reflects the original signal strength received by the sensor; Gain and Bias are camera calibration parameters, which are fixed parameters obtained through precise calibration when the camera leaves the factory and are used to adjust the gain and bias of the original signal; E solar ρ is the solar irradiance, whose value directly affects the intensity of light reflected from ground objects. The higher the solar irradiance, the stronger the light reflected from the same object. θ is the solar zenith angle. Changes in this angle alter the path and intensity of light reaching the ground, thereby affecting reflectivity. This formula converts pixel grayscale values ​​into actual reflectivity, ρ, making data collected at different times and under different weather conditions comparable. Accurate reflectivity calculation is crucial in sugarcane smut assessment, as healthy and diseased sugarcane differ in reflectivity across different wavelengths. Precise radiometric correction can highlight these differences. For example, the chlorophyll content in diseased sugarcane leaves is reduced, resulting in noticeable changes in reflectivity in the red and near-infrared bands. Accurate radiometric correction can more clearly capture these changes, providing reliable spectral data support for subsequent disease detection.

[0038] The radiometrically corrected hyperspectral image is spectrally resampled to make its spectral range consistent with the RGB image data; principal component analysis is performed on the RGB image data to obtain the principal components; the hyperspectral image data is spectrally stretched to make its mean and variance consistent with the principal components; finally, the stretched hyperspectral image data is replaced and fused with the principal components to obtain preprocessed data with both high spectral resolution and high spatial resolution, which facilitates subsequent more comprehensive feature analysis of sugarcane plants.

[0039] Feature fusion: Based on the characteristic manifestations of sugarcane smut, a fusion model is constructed, including spectral feature extraction, visual feature extraction, and spatiotemporal feature fusion. The characteristic manifestations include the characteristic manifestations of sugarcane smut in the spectrum, vision, and visual sequence. In spectral feature extraction, the continuous projection algorithm and the competitive adaptive reweighting algorithm are used to screen the sensitive bands in the hyperspectral data. Based on partial least squares regression, a regression model of spectrum and disease severity is established. The regression model is expressed as Y = XB + E, where Y is the disease severity variable, X is the spectral variable matrix, B is the regression coefficient matrix, and E is the residual matrix. In visual feature extraction, EfficientNet is used as the backbone network, embedded in the spatial pyramid pooling module and the bidirectional feature pyramid network, and combined with the channel attention, spatial attention and self-attention mechanisms. In spatiotemporal feature fusion, the gated recurrent unit is used to construct a time series model.

[0040] Channel attention learns the dependencies between channels through global average pooling and two fully connected layers, and calculates the channel weight w c The formula is wc=σ(W2δ(W1GAP(X))), where GAP is the global average pooling, W1 and W2 are the weights of the fully connected layer, δ is the activation function, and σ is the Sigmoid function.

[0041] Specifically, the successive projections algorithm (SPA) and the competitive adaptive reweighting algorithm (CARS) were used to screen sensitive bands in hyperspectral data. A regression model Y = XB + E was established based on partial least squares regression to link spectrum and disease severity. Here, Y represents the disease severity variable, X is the spectral variable matrix, B is the regression coefficient matrix, and E is the residual matrix. By analyzing the spectral data of a large number of sugarcane samples with known disease severity, the relationship between sensitive bands and disease severity was determined, enabling accurate assessment of sugarcane disease status and severity based on spectral data.

[0042] Using EfficientNet as the backbone network, it embeds the spatial pyramid pooling (SPP) module and the bidirectional feature pyramid network (BiFPN), and combines the channel attention (SE module), spatial attention (CBAM module) and self-attention mechanism. Among them, the channel attention learns the dependency between channels through global average pooling and two fully connected layers to calculate the channel weight w cThe formula is wc = σ(W2δ(W1GAP(X))). Through this mechanism, the model can automatically learn the importance of different channel features for disease detection, assigning higher weights to key channels and enhancing the ability to extract diseased plant features. For example, when identifying sugarcane smut, channel features related to lesion color and texture may be more critical. The channel attention mechanism can strengthen the feature representation of these channels and improve detection accuracy.

[0043] A timing model was constructed using the gated recurrent unit (GRU) to capture the development dynamics of sugarcane smut in the temporal dimension. A 3D-CNN was combined with the spatial dimension to extract the changing characteristics of lesions, achieving a comprehensive study of the spatiotemporal variation patterns of the disease.

[0044] Model generalization: Based on the actual situation of sugarcane, multi-source images and disease survey data are collected to construct a multi-source dataset. The transfer learning strategy is used to fine-tune and optimize the fusion model to obtain a detection model. The actual situation includes the different ecological environments, planting patterns, and actual conditions of sugarcane varieties. The multi-source images and disease survey data include sugarcane image data from different production areas, different growth stages, and different disease severity, as well as the corresponding disease grade, disease location, and sugarcane variety survey data. The construction of the multi-source dataset includes pixel-level annotation of the multi-source images and disease survey data using Labelme software. The annotation content includes the location of the diseased plant, the location of the disease, the disease grade, and the sugarcane variety information.

[0045] Fine-tuning involves selecting a fusion model pre-trained on ImageNet, globally fine-tuning the fusion model using multi-source datasets, learning the common characteristics of sugarcane plants and the common characteristics of diseases in different production areas, and then performing secondary fine-tuning for the target production area using local annotated data to optimize features related to the local environment and variety. Optimization involves using indicators to evaluate the performance of the fusion model in different production areas, and fusing several fusion models trained on different datasets through ensemble learning methods. The indicators include accuracy, recall rate, F1 value, and average precision mean. Ensemble learning methods include bagging and boosting.

[0046] Specifically, based on the specific ecological environments (e.g., plains, hills, and mountains) of sugarcane cultivation, cropping patterns (monocropping, intercropping), and sugarcane varieties (e.g., Yuetang, Guitang, and Taiwan Sugar), we collected image data of sugarcane from different production areas, growth stages, and disease severity levels, along with corresponding survey data on disease grade, disease location, and sugarcane variety. Using Labelme software, we annotated these data pixel-by-pixel, including diseased plant location, disease location, disease grade, and sugarcane variety, to construct a multi-source dataset.

[0047] A fusion model pre-trained on ImageNet was first fine-tuned globally using multiple datasets, enabling the model to learn universal characteristics of sugarcane plants and common features of diseases across different production areas. Secondary fine-tuning was then performed for the target production area using a small amount of locally annotated data to optimize the model's features specific to the local environment and variety. The performance of the fusion model in different production areas was evaluated using metrics such as accuracy, recall, F1 score, and mean average precision (mAP). Ensemble learning methods such as bagging and boosting were used to fuse multiple fusion models trained on different datasets to improve the model's generalization and detection stability across diverse environments.

[0048] Extraction and analysis: The preprocessed data is subjected to feature extraction and fusion analysis through the detection model to obtain disease detection results containing multimodal features; feature extraction is to extract spectral features from hyperspectral data through spectral feature extraction in the detection model, including sensitive bands and calculation of related vegetation indices, extract visual features of sugarcane plants through visual feature extraction, and extract spatiotemporal variation characteristics of sugarcane smut in time and space through spatiotemporal feature fusion. Visual features include texture, shape, and color. Fusion analysis is to fuse spectral features, visual features, and spatiotemporal variation features to comprehensively judge whether sugarcane is infected with smut and the severity of the disease.

[0049] Specifically, the detection model performs feature extraction and fusion analysis on the preprocessed data. The spectral feature extraction module extracts sensitive bands from the hyperspectral data and calculates relevant vegetation indices (such as NDVI and NDSI). The visual feature extraction module extracts visual features such as texture, shape, and color of the sugarcane plants. The spatiotemporal feature fusion module extracts temporal and spatial variations in sugarcane smut. Finally, the spectral, visual, and spatiotemporal variation features are fused to comprehensively determine the presence of smut infection in sugarcane and the severity of the disease, resulting in a disease detection result that incorporates multimodal features.

[0050] Real-time early warning: Through the air-ground collaborative computing architecture, disease detection results are analyzed in real time, and early warnings are triggered and prevention and control decisions are generated based on set conditions.

[0051] The air-ground collaborative computing architecture includes edge computing modules and cloud servers. The edge computing module is integrated on the drone and deploys a lightweight detection model to perform real-time processing and preliminary analysis of image data, locate suspected diseased plants, and transmit their information to the cloud server through the network. After the cloud server receives the data transmitted by the edge end, it uses a distributed computing framework to store, manage and conduct in-depth analysis of large-scale data, build a spatiotemporal distribution model of the disease, and predict the spread trend of the disease.

[0052] Real-time analysis is based on the spatiotemporal distribution model and prediction results of the disease built on the cloud server, combined with the current disease detection results, to evaluate the occurrence and development trend of sugarcane smut in real time. The set conditions include multi-level early warning conditions and related conditions for predicting future disease outbreak risks based on time-sensitive early warnings combined with meteorological data.

[0053] Specifically, a collaborative air-ground computing architecture consisting of edge computing modules and cloud servers enables real-time analysis of disease detection results. By integrating high-performance edge computing modules such as the NVIDIA Jetson AGX Xavier into drones and deploying lightweight detection models, collected image data is processed and initially analyzed in real time, rapidly locating suspected diseased plants. Key information, such as plant coordinates, disease level, and image slices, is then transmitted to cloud servers via 4G / 5G networks.

[0054] After receiving the data, the cloud server utilizes distributed computing frameworks like Apache Spark to store, manage, and perform in-depth analysis of the large-scale data. This allows the server to construct a spatiotemporal disease distribution model and predict disease spread trends. This model and prediction results, combined with current disease detection results, allow for real-time assessment of sugarcane smut occurrence and trends.

[0055] Multiple levels of early warning conditions are set. For example, the first-level condition is a diseased plant rate ≥5% in a single field and favorable weather conditions for disease spread (temperature 25-30°C, relative humidity ≥80%) within the next 48 hours; the second-level condition is the simultaneous occurrence of an outbreak in three or more adjacent fields and a rapid increase in the diseased plant rate. Time-sensitive early warning conditions are also set based on meteorological data to predict the risk of future disease outbreaks. When these conditions are met, early warning information is sent to farmers, agricultural technicians, and agricultural management departments via various channels, including text messages, mobile apps, and WeChat public accounts. Using expert systems and machine learning algorithms, combined with planting data (such as sugarcane variety, planting density, and fertilization records) and environmental data (soil fertility and meteorological conditions), a personalized prevention and control plan is generated, including recommended pesticides, dosages, application times, and methods, to achieve scientific prevention and control of sugarcane smut.

[0056] By using drones equipped with multispectral, hyperspectral, and RGB cameras for data collection, they obtained multidimensional image data from the critical growth period of sugarcane. The data was then preprocessed with geometric correction, radiometric correction, and image fusion. A fusion model was constructed, combining spectral feature extraction, visual feature extraction, and spatiotemporal feature fusion. This model was fine-tuned and optimized using a transfer learning strategy. Using an air-ground collaborative computing architecture, disease detection results were analyzed in real time, triggering warnings based on set conditions and generating prevention and control decision-making solutions. This approach comprehensively and accurately extracted sugarcane smut characteristics, improving the generalization and efficiency of the detection model. This enabled rapid, accurate, and real-time monitoring of large sugarcane planting areas, addressing the problem that traditional sugarcane smut detection methods were unable to meet the real-time monitoring needs of large sugarcane planting areas.

[0057] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for assessing sugarcane smut based on drone images, characterized in that: The following steps are involved: Data collection: Using drones equipped with multispectral cameras, hyperspectral cameras, and RGB cameras, we planned routes to collect image data of sugarcane. Preprocessing: Perform geometric correction, radiation correction and image fusion processing on the image data in sequence to obtain preprocessed data; Feature fusion: Based on the characteristic manifestations of sugarcane smut, a fusion model is constructed that includes spectral feature extraction, visual feature extraction, and spatiotemporal feature fusion; Model generalization: Based on the actual situation of sugarcane, we collected multi-source images and disease survey data to construct a multi-source dataset. We then used a transfer learning strategy to fine-tune and optimize the fusion model to obtain a detection model. Extraction and analysis: The detection model is used to extract features and perform fusion analysis on the preprocessed data to obtain disease detection results containing multimodal features; Real-time early warning: Through the air-ground collaborative computing architecture, disease detection results are analyzed in real time, and early warnings are triggered and prevention and control decisions are generated based on set conditions.

2. The method for assessing sugarcane smut based on drone images according to claim 1, characterized in that: The flight altitude of the planned route is 30-50 meters, the heading overlap is ≥80%, and the lateral overlap is ≥60%. The image data includes multispectral image data, hyperspectral image data, and RGB image data. The multispectral image data includes spectral information of specific bands of blue light, green light, red light, red edge, and near-infrared. The spectral coverage range of the hyperspectral image data is 400-1000nm. The RGB image data includes visual characteristics of the texture, shape, and color of sugarcane.

3. The method for assessing sugarcane smut based on drone images according to claim 1, characterized in that: In the radiation correction, the reflectance calculation formula of multispectral data is: Among them, DN is the pixel gray value, Gain and Bias are camera calibration parameters, E solar is the solar irradiance, θ is the solar zenith angle, and the image fusion processing is to perform spectral resampling on the hyperspectral image after radiation correction so that its spectral range is consistent with the RGB image data, perform principal component analysis on the RGB image data to obtain the principal component, perform spectral stretching on the hyperspectral image data so that its mean and variance are consistent with the principal component, and then replace and fuse the stretched hyperspectral image data with the principal component to obtain the preprocessed data.

4. The method for assessing sugarcane smut based on drone images according to claim 1, characterized in that: The characteristic manifestations include the characteristic manifestations of sugarcane smut in spectrum, vision, and visual sequence. In the spectral feature extraction, a continuous projection algorithm and a competitive adaptive reweighting algorithm are used to screen sensitive bands in hyperspectral data. A regression model of spectrum and disease severity is established based on partial least squares regression. The regression model is expressed as Y=XB+E, where Y is the disease severity variable, X is the spectral variable matrix, B is the regression coefficient matrix, and E is the residual matrix. In the visual feature extraction, EfficientNet is used as the backbone network, embedded in the spatial pyramid pooling module and the bidirectional feature pyramid network, and combined with the channel attention, spatial attention and self-attention mechanisms. In the spatiotemporal feature fusion, a gated recurrent unit is used to construct a temporal series model.

5. The method for assessing sugarcane smut based on drone images according to claim 4, characterized in that: The channel attention learns the dependencies between channels through global average pooling and two fully connected layers, and calculates the channel weight w c The formula is wc=σ(W2δ(W1GAP(X))), where GAP is the global average pooling, W1 and W2 are the weights of the fully connected layer, δ is the activation function, and σ is the Sigmoid function.

6. The method for assessing sugarcane smut based on drone images according to claim 1, characterized in that: The actual conditions include different ecological environments, planting patterns, and actual conditions of sugarcane varieties. The multi-source images and disease survey data include image data of sugarcane from different production areas, different growth stages, and different disease severity levels, as well as corresponding survey data on disease levels, disease locations, and sugarcane varieties. Constructing the multi-source dataset includes pixel-level annotation of the multi-source images and disease survey data using Labelme software. The annotation content includes information on the location of the diseased plant, the disease location, the disease level, and the sugarcane variety.

7. The method for assessing sugarcane smut based on drone images according to claim 1, characterized in that: The fine-tuning involves selecting a fusion model pre-trained on ImageNet, globally fine-tuning the fusion model using multi-source datasets, learning the common characteristics of sugarcane plants and the common characteristics of diseases in different production areas, and then performing secondary fine-tuning for the target production area using local annotated data to optimize features related to the local environment and variety. The optimization involves using indicators to evaluate the performance of the fusion model in different production areas, and fusing several fusion models trained on different datasets through an ensemble learning method. The indicators include accuracy, recall rate, F1 value, and average precision mean. The ensemble learning method includes bagging and boosting.

8. The method for assessing sugarcane smut based on drone images according to claim 1, characterized in that: The feature extraction method involves extracting spectral features from hyperspectral data through spectral feature extraction in a detection model, including sensitive bands and calculating related vegetation indices. Visual feature extraction is used to extract visual features of sugarcane plants. Spatiotemporal variation characteristics of sugarcane smut in time and space are extracted through spatiotemporal feature fusion. The visual features include texture, shape, and color. The fusion analysis method involves fusing spectral features, visual features, and spatiotemporal variation characteristics to comprehensively determine whether sugarcane is infected with smut and the severity of the disease.

9. The method for assessing sugarcane smut based on drone images according to claim 1, characterized in that: The air-ground collaborative computing architecture includes an edge computing module and a cloud server. The edge computing module is integrated on the drone and deploys a lightweight detection model to perform real-time processing and preliminary analysis of image data, locate suspected diseased plants, and transmit their information to the cloud server via the network. After receiving the data transmitted by the edge end, the cloud server uses a distributed computing framework to store, manage and deeply analyze large-scale data, build a spatiotemporal distribution model of the disease, and predict the spread trend of the disease.

10. The method for assessing sugarcane smut based on drone images according to claim 1, characterized in that: The real-time analysis is based on the spatiotemporal distribution model and prediction results of the disease constructed by the cloud server, combined with the current disease detection results, to evaluate the occurrence and development trend of sugarcane smut in real time. The set conditions include multi-level early warning conditions and relevant conditions for predicting future disease outbreak risks based on time-sensitive early warnings combined with meteorological data.

Images