Crop disease and pest real-time identification, prevention and control decision-making method and system based on multi-modal edge calculation

Abstract

The invention relates to the field of computer vision, in particular to a crop disease and insect pest real-time identification, prevention and control decision-making method and system based on multi-modal edge calculation. Comprising the following steps: acquiring an unmanned aerial vehicle multispectral image, an infrared thermal image and an NDVI index, performing preprocessing in combination with field Internet of Things sensing data, and generating a multi-modal feature input sequence; constructing a spectrum-environment fusion feature matrix on the basis of a Transform attention mechanism, and performing disease and pest detection by using a YOLOv7-Spectral model to generate a disease and pest distribution heat map; calculating a disease risk index (BRI) and a pesticide application priority map based on historical data; planning a pesticide application path of the unmanned aerial vehicle by adopting an ant colony optimization algorithm, and optimizing a pesticide application scheme in combination with wind speed and humidity parameters; the unmanned aerial vehicle performs precise pesticide application according to the optimized path and monitors the disease change trend; adjusting the model through pesticide application feedback data, and optimizing a disease and pest prevention and control strategy by adopting federal learning. According to the invention, the identification precision is improved, the pesticide use is reduced, and accurate, efficient and intelligent prevention and control are realized.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and particularly to a real-time recognition and prevention and control decision-making method and system for crop pests and diseases based on multi-modal edge computing. Background Art

[0002] Agricultural pest control is an important link in ensuring food security. Traditional crop pest and disease monitoring mainly relies on manual inspections and single image recognition methods, which have problems such as low monitoring efficiency, large influence of the recognition accuracy by the environment, and excessive use of pesticides. In addition, due to data transmission delays, the remote pest and disease recognition method based on cloud computing is difficult to meet the real-time prevention and control needs of large-scale farmland. Therefore, how to efficiently fuse multi-modal data on edge computing devices, improve the real-time recognition accuracy of pests and diseases, and optimize the precise pesticide application strategy of drones is a key technical problem in current intelligent agricultural prevention and control.

[0003] Most existing pest and disease monitoring methods are based on single RGB images or hyperspectral data, which are greatly affected by environmental factors such as light and occlusion, and it is difficult to ensure the stability of pest and disease recognition in complex farmland environments. At the same time, traditional pesticide application methods use fixed or empirical drone spraying paths, and do not optimize by combining disease risk assessment and meteorological parameters, resulting in pesticide waste, environmental pollution, and reduced prevention and control efficiency.

[0004] In view of the above problems, the present invention fuses drone multi-spectral images, infrared thermal imaging, and field Internet of Things sensing data, extracts spectral reflectance, leaf surface temperature, NDVI index, and environmental factors, constructs a spectral-environment fusion feature matrix using the Transformer attention mechanism, and realizes efficient pest and disease target detection on edge computing devices based on the lightweight YOLOv7-Spectral model. In addition, the present invention evaluates the severity of diseases through the disease risk index (BRI), generates a precise drone pesticide application path by combining the ant colony optimization algorithm (ACO), and at the same time adopts a federated learning mechanism to optimize the pest and disease recognition model, realizing cross-farm data sharing and the intelligent upgrade of disease prevention and control strategies. Summary of the Invention

[0005] The present invention provides a real-time recognition and prevention and control decision-making method and system for crop pests and diseases based on multi-modal edge computing, so as to solve the problem of how to fuse spectral reflectance, leaf surface temperature, NDVI index, and environmental factors based on drone multi-spectral images, infrared thermal imaging data, and field Internet of Things sensing data, construct a lightweight YOLOv7-Spectral model, realize the real-time recognition of pest and disease targets on edge computing devices, and plan a precise drone pesticide application path by combining the ant colony optimization algorithm, so as to improve the pest and disease recognition accuracy, reduce the amount of pesticide used, and optimize the prevention and control model through federated learning, realizing cross-farm data sharing and the intelligent upgrade of disease prevention and control strategies.

[0006] To solve the above technical problems, the present invention provides a real-time identification, prevention and control decision-making method for crop pests and diseases based on multimodal edge computing, including: A real-time identification, prevention and control decision-making method for crop pests and diseases based on multimodal edge computing, characterized in that the method includes the following steps: Obtain multispectral image data, infrared thermal imaging data and NDVI index collected by an unmanned aerial vehicle (UAV), and obtain temperature, humidity, wind speed and soil moisture data from field Internet of Things sensors, and synchronously preprocess the data to generate a multimodal feature input sequence; Perform normalization processing on the multimodal feature input sequence, and perform multi-dimensional feature mapping based on the Transformer attention mechanism to generate a spectral-environment fusion feature matrix; Based on the spectral-environment fusion feature matrix and the pre-trained YOLOv7-Spectral model, perform pest and disease target detection, generate a pest and disease distribution heat map, and perform confidence screening on the recognition results to extract pest and disease feature regions; Extract spatial position features from the pest and disease distribution heat map and the data obtained from field Internet of Things sensors, and combine historical disease trend data to evaluate the disease severity, and generate a disease risk index (BRI) and a pesticide application priority map; Based on the pesticide application priority map and the UAV operation parameters, construct an ant colony optimization path planning model, and combine wind speed and humidity parameters to generate an optimized UAV pesticide application path sequence; Obtain the optimized UAV pesticide application path sequence, control the UAV to perform precise pesticide application according to the optimized path, and simultaneously monitor the meteorological data and real-time images of the pesticide application area to update the disease change trend sequence; Based on the disease change trend sequence and the feedback data after pesticide application, adjust the parameter weights of the YOLOv7-Spectral model, and optimize the disease recognition and pesticide application path planning model through federated learning to generate an updated intelligent prevention and control model; Among them, the disease risk index (BRI) is calculated by the following formula: ; Among them, is the disease risk index at grid ; is the disease severity index at grid ; is the environmental data at grid ; is the weight of the disease and environmental impact factors.

[0007] Furthermore, the preprocessing of the multimodal feature input sequence comprises the following steps: Interpolation and noise removal are performed on the multispectral image data, infrared thermal imaging data and NDVI index to generate a standardized multispectral image sequence and infrared thermal imaging sequence; Perform time alignment and spatial registration on field IoT sensor data, and perform data normalization to generate standardized environmental data sequences; The standardized multi-spectral image sequence, the infrared thermal imaging sequence and the environmental data sequence are fused to generate a preliminarily processed multi-modal feature input sequence.

[0008] Furthermore, the construction of the multidimensional feature map comprises the following steps: Performing feature decomposition on the multimodal feature input sequence to extract spectral reflectance, leaf surface temperature, NDVI index and environmental factors; The Transformer attention mechanism is used to perform feature mapping on the spectral reflectance, leaf surface temperature, NDVI index and environmental factors to generate a spectral-environmental fusion feature matrix.

[0009] Furthermore, the pest target detection includes the following steps: Input the spectrum-environment fusion feature matrix into the pre-trained YOLOv7-Spectral model to perform pest and disease target detection and generate pest and disease target detection results; Based on the pest and disease target detection results, distribution mapping of the detection targets is performed according to spatial positions to generate a pest and disease distribution heat map; The pest and disease distribution heat map is screened for confidence, low-confidence targets are eliminated, and pest and disease characteristic area data are extracted.

[0010] Furthermore, the disease severity assessment comprises the following steps: Based on the pest and disease characteristic area data and the data obtained by the field IoT sensors, the spatial location characteristics of the target crop area are extracted to obtain the crop spatial distribution sequence; Based on the crop spatial distribution sequence and combined with historical disease trend data, the disease severity index is calculated and a disease risk index (BRI) is generated.

[0011] Furthermore, the optimized path sequence for drone pesticide application includes the following steps: Based on the pesticide application priority map and the UAV operation parameters, an ant colony optimization path planning model is constructed to generate an initial UAV pesticide application path plan; The initial UAV pesticide application path plan is optimized by combining wind speed and humidity parameters, and the spraying angle is adjusted to generate an optimized UAV pesticide application path sequence; Smooth the optimized UAV spraying path sequence and adjust the spraying height to generate the final precise spraying path.

[0012] Further, controlling the UAV to perform precise spraying according to the optimized path includes the following steps: Based on the UAV operation parameters, control the UAV to perform precise spraying according to the optimized UAV spraying path, and record the spraying position and pesticide dosage data at the same time; During the spraying process, obtain the meteorological data and real-time images of the spraying area, and compare the data with historical data to obtain the disease change trend sequence; Based on the disease change trend sequence, predict the disease change and update the disease severity assessment data.

[0013] Further, adjusting the parameter weights of the YOLOv7-Spectral model includes the following steps: Obtain the feedback data after spraying, including UAV operation data, crop health status and farmland environment data, and form a spraying effect feedback sequence; Based on the spraying effect feedback sequence, adjust the parameter weights of the multi-modal feature mapping model and the YOLOv7-Spectral model to generate an optimized pest and disease identification model.

[0014] Further, the federated learning optimized prevention and control model includes the following steps: Based on the pest and disease data of multiple farms, calculate the local update parameters of each sub-model and independently train the model on each end device; Based on the local update parameters, use the federated averaging algorithm to calculate the global model parameters, perform model aggregation, and generate an updated intelligent prevention and control model.

[0015] A real-time identification and prevention and control decision-making system for crop pests and diseases based on multi-modal edge computing, which is applied to the real-time identification and prevention and control decision-making method for crop pests and diseases based on multi-modal edge computing as described in any one of the above, includes: A data acquisition module for obtaining monitoring data related to crop growth and pests and diseases from multi-modal data sources; A data processing and fusion module for preprocessing the collected multi-modal data and using the Transformer attention mechanism to perform feature mapping on the multi-modal data to generate a spectral-environment fusion feature matrix; A pest and disease identification module for performing real-time target detection of crop pests and diseases based on the YOLOv7-Spectral deep learning model; The disease risk assessment module is used to calculate the disease severity index based on the pest and disease distribution heat map, field environment sensing data, and historical disease trend data, and further generate the disease risk index (BRI). The UAV spraying path optimization module is used to plan the optimal spraying path by using the ant colony optimization algorithm based on the disease risk index and UAV operation parameters. The UAV spraying execution and data monitoring module is used to control the UAV to perform precise spraying according to the optimized path, and record the spraying position and pesticide dosage in real time. The federated learning optimization and model update module is used to optimize and adjust the multi-modal feature mapping model and the YOLOv7-Spectral pest and disease identification model based on the spraying feedback data, and integrate cross-farm data through the federated learning mechanism to improve the generalization ability of the model.

[0016] The following are its main beneficial effects: (1) Improve the accuracy of pest and disease identification. By using the multi-modal data fusion technology, the UAV multi-spectral images, infrared thermal imaging, NDVI index, and field environment data are subjected to Transformer feature mapping to construct a spectral-environment fusion feature matrix, effectively reducing the influence of single spectral data by factors such as illumination changes and background interference, and improving the accuracy of pest and disease identification in complex farmland environments. Compared with the traditional RGB image recognition method, the present invention realizes more stable and higher-precision pest and disease target detection on edge computing devices.

[0017] (2) Reduce the amount of pesticide used and improve the effect of precise spraying. By calculating the disease risk index (BRI), combining historical disease trend data and field environment data, quantitatively evaluating the disease severity, and generating a spraying priority map to ensure precise spraying by the UAV. The ant colony optimization algorithm (ACO) is used to plan the UAV spraying path, and the spraying angle and flow rate are adjusted in combination with meteorological parameters such as wind speed and humidity, reducing the unnecessary amount of pesticide used. Compared with the traditional uniform spraying method, the pesticide consumption is reduced by 30%, effectively reducing agricultural pollution and improving the precision of spraying.

[0018] (3) Improve the intelligent level of disease prevention and control and realize cross-farm data sharing. The federated learning (FL) is used to optimize the pest and disease identification and prevention and control model, and the cross-regional model update is carried out by using the data of different farms to improve the generalization ability of disease identification. The present invention can be distributedly deployed in multiple farms, without the need to centrally upload data to the cloud, avoiding data privacy leakage, and at the same time being able to adapt to different climate and soil environments, improving the intelligent level of large-scale agricultural prevention and control.

[0019] (4) Empowered by edge computing, millisecond-level real-time recognition is achieved. The traditional cloud computing-based pest and disease recognition method has data upload latency and cannot meet the requirements of large-scale real-time monitoring of farmland. The present invention adopts a lightweight YOLOv7-Spectral model, optimizes the neural network parameters, makes it applicable to edge computing devices such as Jetson AGX Xavier, realizes millisecond-level pest and disease detection, improves the response speed of real-time prevention and control in farmland, and avoids the spread of diseases caused by latency. Description of the Drawings

[0020] Figure 1 It is a schematic flowchart of the real-time recognition, prevention and control decision-making method for crop pests and diseases based on multi-modal edge computing provided by the embodiment of the present application; Figure 2 It is a structural block diagram of the real-time recognition, prevention and control decision-making system for crop pests and diseases based on multi-modal edge computing provided by the embodiment of the present application. Specific Embodiments

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs; the terms used in the description of the present application in the specification are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the description and claims of the present application and the above description of the drawings are intended to cover non-exclusive inclusion. The terms "first", "second", etc. in the description and claims of the present application or the above drawings are used to distinguish different objects and not to describe a specific order.

[0022] Mentioning "embodiment" herein means that the specific features, structures or characteristics described in connection with the embodiment may be included in at least one embodiment of the present application. The phrase appears at various positions in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein may be combined with other embodiments.

[0023] Embodiment 1: Refer to Figure 1 It is a schematic flowchart of the real-time recognition, prevention and control decision-making method for crop pests and diseases based on multi-modal edge computing provided by the embodiment of the present invention. The process may at least include steps S100-S700: S100. Obtain multi-spectral image data, infrared thermal imaging data and NDVI index collected by the unmanned aerial vehicle, and obtain temperature, humidity, wind speed and soil moisture data from the field Internet of Things sensors, and synchronously preprocess the data to generate a multi-modal feature input sequence; S200. Normalize the multi-modal feature input sequence and perform multi-dimensional feature mapping based on the Transformer attention mechanism to generate a spectral-environment fusion feature matrix; S300. Based on the spectral-environment fusion feature matrix and the pre-trained YOLOv7-Spectral model, perform pest and disease target detection, generate a pest and disease distribution heat map, and perform confidence screening on the recognition results to extract the pest and disease feature regions; S400. Extract spatial location features from the pest and disease distribution heat map and the data obtained from the field Internet of Things sensors, and combine historical disease trend data to evaluate the disease severity, generate a disease risk index (BRI) and a pesticide application priority map; S500. Based on the pesticide application priority map and the UAV operation parameters, construct an ant colony optimization path planning model, and combine the wind speed and humidity parameters to generate an optimized UAV pesticide application path sequence; S600. Obtain the optimized UAV pesticide application path sequence, control the UAV to perform precise pesticide application according to the optimized path, and simultaneously monitor the meteorological data and real-time images of the pesticide application area to update the disease change trend sequence; S700. Based on the disease change trend sequence and the feedback data after pesticide application, adjust the parameter weights of the YOLOv7-Spectral model, and optimize the disease recognition and pesticide application path planning model through federated learning to generate an updated intelligent prevention and control model.

[0024] Step S100 at least includes steps S110 - S130: S110. UAV multi-spectral and infrared thermal imaging data acquisition; Specifically, call the UAV-borne hyperspectral imaging device to obtain the spectral reflectance data of the crop canopy. The data covers the 400 - 1000 nm band, and collect multi-spectral images according to the set spatial resolution to generate a multi-spectral image dataset . At the same time, use the infrared thermal imaging device carried by the UAV to measure the surface temperature of the crop leaves to generate an infrared thermal imaging dataset .

[0025] The spectral reflectance of the multi-spectral image dataset is calculated as follows: ; where: ; is the radiance of the crop canopy at wavelength ; is the blackboard reference luminance (used to eliminate device noise); is the reference brightness of the whiteboard (for illumination normalization).

[0026] The infrared thermal imaging dataset Measure the temperature of the crop leaf surface by an unmanned aerial vehicle thermal imager, and calculate the crop leaf temperature , and its calculation formula is as follows: ; Where: is the infrared thermal radiation energy; is the Stefan-Boltzmann constant ( ).

[0027] Furthermore, the and are geographically registered to generate a spatially aligned crop multi-spectral - thermal imaging dataset , as the input for subsequent data fusion.

[0028] S120. Obtain field Internet of Things sensing data; Specifically, collect environmental data from Internet of Things sensors deployed in the field, including air temperature , air humidity , soil moisture and wind speed , and generate an environmental sensing dataset .

[0029] The environmental data is transmitted in real time through LoRa or 5G network, and the timestamp is synchronized to ensure that all data is calculated within the same time window.

[0030] The environmental data normalization process is as follows: ; Where: is the normalized environmental sensing data; and are the minimum and maximum values of the environmental data respectively.

[0031] Furthermore, synchronize the environmental data with the and register them according to the crop spatial coordinates to generate a fused input dataset , which is used for subsequent multi-modal feature extraction.

[0032] S130. Data preprocessing and feature extraction; Specifically, perform interpolation completion, noise removal, and normalization on the fused input dataset to generate a preliminarily processed multi-modal feature input sequence 。

[0033] For the data missing caused by acquisition errors, bilinear interpolation method is used to complete it, so that all data are consistent in the time dimension and space dimension.

[0034] The interpolation and completion calculation is as follows: ; Where: is the interpolation value of the current pixel point; , , , are the values of adjacent pixel points respectively.

[0035] For the dataset after the interpolation and completion noise removal is performed. Wavelet transform is used to remove high-frequency noise and reduce data acquisition errors. The wavelet transform denoising process is as follows: ; Where: is the data after denoising; is the wavelet coefficient; is the wavelet basis function.

[0036] For the dataset after denoising normalization processing is performed to ensure that different modality data are within the same numerical range, and a standardized feature dataset is generated. The calculation is as follows: ; Where: is the mean value of the dataset; is the standard deviation of the dataset.

[0037] Furthermore, the is used as input data and transmitted to the S200 multi-modal data fusion module, and is used for feature correlation modeling based on the Transformer-based multi-dimensional feature mapping in S220.

[0038] Step S200 at least includes steps S210 - S230: S210, multi-modal feature normalization; Specifically, the multi-modal feature input sequence generated by calling the S130 is used to perform normalization processing on the data to eliminate the numerical scale differences of different data modalities. The normalization processing includes global normalization and crop category adaptive weighted normalization.

[0039] For the spectral reflectance calculated by the S130, leaf surface temperature , NDVI index and environmental factors are standardized, and the normalization calculation is as follows: ; Where: is the normalized feature; is the original feature value; is the mean value of this feature; is the standard deviation of this feature.

[0040] The above standardization process ensures that different data modalities are in the same numerical range, improving the stability of subsequent model learning.

[0041] Furthermore, according to the crop type and growth stage, the normalized data is weighted and adjusted to generate the normalized features of the crop growth stage: ; Where: is the adaptively adjusted feature value; is the normalized weight corresponding to the crop type; is the bias parameter corresponding to the crop growth stage.

[0042] The above adaptive weighted normalization is dynamically adjusted according to the spectral feature differences of different crop varieties to improve the accuracy of feature fusion.

[0043] Based on the above normalization steps, a standardized feature data sequence is generated, and the data is used as input and transmitted to S220 for multi-dimensional feature mapping based on Transformer.

[0044] S220, multi-dimensional feature mapping based on Transformer; Specifically, the spectral reflectance, leaf surface temperature, NDVI index and environmental factors are extracted from the standardized feature data sequence generated by S210, a multi-dimensional feature mapping relationship is constructed, and a spectral-environment fusion feature matrix is generated based on the Transformer attention mechanism.

[0045] The standardized feature data sequence is decomposed into feature subsets: ; Where: is the spectral reflectance matrix; is the leaf surface temperature matrix; is the normalized NDVI index matrix; is the environmental factor matrix.

[0046] Based on the said feature subset, the Transformer self-attention mechanism is adopted to calculate the weight distribution of multi-modal features, and the calculation formula is as follows: ; where: is the fused feature after attention weighting; are the query, key, and value matrices respectively, and the calculation methods are as follows: ; where are the weight matrices of the query, key, and value; is the feature dimension normalization factor.

[0047] The calculation process of the said Transformer can automatically adjust the influence weights of different data modalities, so that the features related to pests and diseases receive higher attention and enhance the recognition ability of the model.

[0048] The fused feature matrix after weighting by the said attention mechanism is transformed to generate the spectral-environmental fused feature matrix : ; where: is the linear transformation weight matrix; is the bias parameter.

[0049] Furthermore, the calculated by S220 is used as the input and transmitted to S230 for dimensionality reduction and feature extraction.

[0050] S230, multi-modal data fusion output; Specifically, the spectral-environmental fused feature matrix generated by calling S220 is used to perform dimensionality reduction on the said data, and the core features are extracted through the feature clustering algorithm to generate the fused feature data for pest and disease recognition.

[0051] Perform principal component analysis (PCA) on the spectral-environmental fused feature matrix calculated by S220 to reduce the feature dimension and improve the calculation efficiency. The calculation formula is as follows: ; where: is the feature matrix after dimensionality reduction; is the principal component transformation matrix of the feature matrix, which is calculated by PCA.

[0052] The dimensionality reduction of the feature reduces redundant information and improves the computational efficiency of subsequent pest and disease detection.

[0053] Furthermore, for the feature matrix after dimensionality reduction perform feature clustering analysis, calculate the similarity of different feature subsets, and the calculation is as follows: ; Where: is the similarity between the th and the th samples; is the similarity measurement parameter.

[0054] The feature clustering analysis can effectively identify the key features related to pests and diseases, eliminate irrelevant information, and improve the detection accuracy.

[0055] Finally, the data matrix after feature clustering analysis is used as the output and transmitted to the S300 pest and disease target detection module for the pest and disease detection task of the YOLOv7-Spectral model.

[0056] Step S300 at least includes steps S310 - S330: S310, input the feature matrix into the pre-trained YOLOv7-Spectral model; Specifically, call the fused feature data generated by S230 , input the data into the pre-trained YOLOv7-Spectral model for target detection to identify the pest and disease targets on the crops and generate the pest and disease target detection results .

[0057] Perform format conversion on the fused feature data calculated by S230 to ensure that it meets the input requirements of the YOLOv7-Spectral target detection model, and the calculation is as follows: ; Where: is the input data before being processed by YOLOv7-Spectral; is the input image size of the target detection network.

[0058] Input the data after format conversion into the pre-trained YOLOv7-Spectral model for forward inference to generate the target bounding box and its confidence score , and the calculation is as follows: ; Wherein: are the bounding box coordinates of the pest and disease target; is the confidence score of the corresponding target; represents the detected target index.

[0059] Furthermore, based on the inference result of the YOLOv7-Spectral model, generate pest and disease target detection result data , which contains the positions, categories, and confidence scores of all detected targets, and transmit the data to S320 for generating the pest and disease distribution heat map.

[0060] S320, generating the pest and disease distribution heat map; Specifically, call the pest and disease target detection result calculated by S310 , perform distribution mapping according to the target spatial position, and generate the pest and disease distribution heat map .

[0061] Normalize the target bounding box in the pest and disease target detection result to adapt to the pixel coordinate system for heat map generation, and the calculation is as follows: ; Wherein: is the normalized target bounding box; are the width and height of the input image.

[0062] Based on the normalized target coordinates , calculate the spatial density distribution of the pest and disease target , and the calculation is as follows: ; Wherein: represents the pest and disease density at the pixel coordinate ; is the confidence score of the pest and disease target; is the density distribution smoothing parameter.

[0063] Furthermore, perform visualization processing on the pest and disease density distribution data to generate the pest and disease distribution heat map , and transmit the data to S330 for confidence score screening and feature extraction.

[0064] S330, confidence score screening and feature extraction; Specifically, call the pest and disease distribution heat map calculated by S320 , perform confidence screening on the data, extract the pest and disease characteristic regions, and obtain the pest and disease characteristic region data .

[0065] For the pest and disease distribution heat map perform confidence screening, eliminate the low-confidence targets, and calculate as follows: ; where: is the pest and disease distribution heat map after screening; is the confidence threshold.

[0066] Based on the pest and disease distribution heat map after screening calculated from S330 , perform connected component analysis, extract the pest and disease characteristic regions, and calculate as follows: ; where: is the pest and disease characteristic region data; is the connected component extraction function.

[0067] Finally, use the pest and disease characteristic region data as the output and transmit it to the S400 pest and disease risk assessment module for subsequent assessment of disease severity and calculation of pesticide application optimization.

[0068] Step S400 includes at least steps S410 - S430: S410, extract the spatial location features; Specifically, call the pest and disease characteristic region data calculated from S330 , and combine it with the environmental sensing data sequence generated by S120 , extract the spatial location features of the target crop region, and obtain the crop spatial distribution sequence .

[0069] Perform spatial coordinate transformation on the pest and disease characteristic region data to ensure that the target location matches the farmland grid area, and calculate as follows: ; where: is the transformed target spatial coordinate; is the original target coordinate; is the geospatial coordinate transformation matrix.

[0070] Based on the field Internet of Things sensing data calculated from S120 , perform crop area index matching to obtain the geographical grid number corresponding to the target crop , calculate as follows: ; Where: is the geographical grid number corresponding to the target crop; is the grid index function.

[0071] Generate a crop spatial distribution sequence based on the target location mapping and crop area index matching , and transmit the data to S420 for disease severity assessment.

[0072] S420, disease severity assessment; Specifically, call the crop spatial distribution sequence calculated by S410 , combined with historical disease trend data , calculate the disease severity index , and further generate a disease risk index (BRI).

[0073] Based on the crop spatial distribution sequence and historical disease trend data , calculate the disease severity index , calculate as follows: ; Where: is the disease severity corresponding to the grid ; is the historical disease index of the grid ; is the confidence score of the pest and disease target; is the target to the grid Euclidean distance; is the spatial influence scale parameter; is the weight parameter.

[0074] Furthermore, based on the disease severity index calculate the disease risk index (BRI), calculate as follows: ; Where: is the disease risk index of the grid ; is the environmental factor data; is the weight of the disease and environmental impact factors.

[0075] Finally, the disease risk index (BRI) data is generated, and the data is transmitted to S430 for generating a pesticide application priority map.

[0076] S430, Generating a pesticide application priority map; Specifically, the disease risk index (BRI) data calculated by calling S420 is used to divide the data into regions, and a pesticide application priority map is generated based on the disease distribution trend .

[0077] Based on the disease risk index (BRI) calculated by S420, the farmland area is graded to generate different risk level areas, and the calculation is as follows: ; Where: is the grid corresponding disease risk level; is the risk index threshold.

[0078] Based on the disease risk area division result, the pesticide application priority is calculated , and the calculation is as follows: ; Where: is the grid pesticide application priority; is the weighting parameter.

[0079] Finally, a pesticide application priority map is generated based on the pesticide application priority calculation result , and the data is transmitted to the S500 UAV pesticide application path optimization module for optimizing the UAV pesticide application operation.

[0080] Step S500 includes at least steps S510 - S530: S510, Constructing an ant colony optimization path planning model; Specifically, the pesticide application priority map calculated by calling S430 is , and combined with the UAV operation parameters , an ant colony optimization path planning model is constructed to generate an initial UAV pesticide application path plan .

[0081] Based on the pesticide application priority map , a farmland grid path network is constructed , where: represents the grid set, and each grid represents an area in the farmland; represents the connected edge between adjacent grids; each edge The path cost is calculated as follows: ; Where: is the path cost from grid to grid ; is the Euclidean distance between two grids; is the disease risk index of grid ; is the path cost weight parameter.

[0082] Initialize the ant colony optimization (ACO) model, set the number of ants and the maximum number of iterations , define the pheromone matrix and the heuristic information : ; Where: is the pheromone concentration on path ; is the heuristic information; is a very small positive number to prevent division by zero.

[0083] Based on the ant colony optimization model, calculate the initial spraying path plan of the drone , and calculate the path selection probability as follows: ; Where: is the probability that the ant selects from node to node ; is the weight of the pheromone and the heuristic information.

[0084] After the calculation, transmit the initial spraying path plan of the drone to S520 for path optimization.

[0085] S520 optimizes the path by combining the wind speed and humidity parameters; Specifically, call the initial spraying path plan of the drone calculated by S510 , and combine the wind speed and humidity parameters generated by S120 , optimize the spraying path, and generate an optimized spraying path sequence of the drone .

[0086] Based on the wind speed parameter , adjust the spraying angle of the drone at grid , the calculation is as follows: ; Where: is the spraying angle of the UAV at grid . are the components of the wind speed in the and directions respectively.

[0087] Based on the humidity parameter , adjust the pesticide application flow rate , the calculation is as follows: ; Where: is the pesticide application flow rate of the UAV at grid ; is the reference pesticide application flow rate.

[0088] Based on the corrected spraying angle and the pesticide application flow rate , update the UAV pesticide application path to generate an optimized UAV pesticide application path sequence , and transmit it to S530 for final path optimization.

[0089] S530 outputs the final pesticide application path; Specifically, call the optimized UAV pesticide application path sequence calculated by S520, smooth the path, and adjust the spraying height to generate the final precise pesticide application path .

[0090] Perform Bezier curve fitting on the optimized UAV pesticide application path sequence to ensure the smoothness of the flight trajectory, the calculation is as follows: ; Where: is the smoothed path point; are three adjacent points on the path; is the interpolation parameter, and the value range is .

[0091] Based on the crop height , adjust the flight height of the UAV at grid , the calculation is as follows: ; Where: is the spraying height of the UAV; is the reference spraying height; is the crop height adjustment factor.

[0092] Finally, generate the final precise pesticide application path , and transmit the data to the S600 UAV pesticide application execution and data monitoring module for precise pesticide application tasks.

[0093] Step S600 includes at least steps S610 - S630: S610. Execute precise pesticide application; Specifically, call the final precise pesticide application path calculated by S530 , and based on the UAV operation parameters , control the UAV to execute precise pesticide application according to the path, and record the pesticide application position data and pesticide dosage data .

[0094] Based on the final precise pesticide application path , control the UAV to fly to each target grid in sequence , execute precise pesticide application in the corresponding grid, and calculate the current position of the UAV , as follows: ; Where: is the time the actual position of the UAV at the moment; is the target position in the path sequence; is the path interpolation coefficient.

[0095] Based on the spraying height calculated by S530 and the wind speed adjustment parameter , dynamically adjust the pesticide flow rate , calculate as follows: ; Where: is the pesticide spraying flow rate at grid ; is the reference spraying flow rate; is the maximum spraying height of the UAV.

[0096] Furthermore, record the UAV GPS position information, real-time pesticide consumption, and pesticide application duration during the pesticide application process, generate the pesticide application position data and pesticide dosage data , and transmit the data to S620 for pesticide application area data monitoring.

[0097] S620, Monitoring of data in the pesticide application area; Specifically, call the pesticide application location data generated by S610 and the pesticide dosage data , and combine with the meteorological data of the pesticide application area collected by S120 and the real-time image data obtained by the UAV camera equipment to monitor the pesticide application area and calculate the disease change trend sequence .

[0098] Based on the pesticide application location data and the meteorological data , match the wind speed, humidity and temperature parameters at the corresponding pesticide application time, and calculate as follows: ; Where: is the wind speed at the grid at the pesticide application time; is the air humidity at the pesticide application time; is the air temperature at the pesticide application time.

[0099] Based on the disease change trend sequence calculated by S620 and the UAV real-time image data , calculate the change in the health status of the crops in the target area after pesticide application, and calculate as follows: ; Where: is the crop health recovery rate at the grid ; is the disease distribution heat map before pesticide application.

[0100] Finally, generate the pesticide application effect feedback sequence , and transmit the data to S720 for model optimization and weight adjustment.

[0101] S720, Model optimization and weight adjustment; Specifically, call the pesticide application effect feedback sequence calculated by S710 , and combine with the multi-modal feature mapping model generated by S220 and the YOLOv7-Spectral model pre-trained by S310 , optimize and adjust the model parameters to generate an optimized pest and disease identification model .

[0102] Based on the pesticide application effect feedback sequence , calculate the loss function of the model , and calculate as follows: ; Among them: is the error loss of the model; is the actual disease label; is the disease label predicted by the model.

[0103] Based on the loss function , the model parameters are updated using the Adam optimization algorithm, and the calculation is as follows: ; Among them: is the parameter of the model at time ; is the learning rate.

[0104] Finally, an optimized pest and disease identification model is generated, and the model is transmitted to S730 for federated learning training.

[0105] S730, federated learning optimized prevention and control model; Specifically, the optimized pest and disease identification model calculated by calling S720 is combined with cross-farm data for federated learning training to achieve model generalization optimization and generate an updated intelligent prevention and control model .

[0106] Based on the cross-farm data , the parameter update of each farm sub-model is calculated , and the calculation is as follows: ; Among them: is the model update parameter at farm ; is the model loss at farm .

[0107] Based on the sub-model updates of all farms , the global model parameters are calculated using the federated averaging algorithm (FedAvg), and the calculation is as follows: ; Among them: is the data weight at farm .

[0108] Finally, an updated intelligent prevention and control model is generated, and the model is transmitted to S310 for pest and disease target detection tasks.

[0109] Example 2: Figure 2 The structure block diagram of the real-time identification, prevention and control decision-making system for crop diseases and pests based on multi-modal edge computing according to an embodiment of the present invention is shown. As Figure 2 shown, the structure may include: The data acquisition module 10 is used to obtain monitoring data related to crop growth and diseases and pests from multi-modal data sources, including multi-spectral image data, infrared thermal imaging data, NDVI index carried by unmanned aerial vehicles, and environmental data (temperature, humidity, wind speed, soil moisture, etc.) collected by field Internet of Things sensors. This module ensures the diversity and high spatio-temporal resolution of the collected data, providing accurate original input for subsequent pest identification.

[0110] The data processing and fusion module 20 is used to preprocess the collected multi-modal data, including data denoising, missing value filling, normalization processing, and use the Transformer attention mechanism to perform feature mapping on the multi-modal data to generate a spectral-environment fusion feature matrix. This module ensures the alignment of data from different data sources, improving the generalization ability of the model and the data fusion accuracy.

[0111] The pest and disease identification module 30 is used to perform real-time object detection on crop diseases and pests based on the YOLOv7-Spectral deep learning model. This module uses the spectral-environment fusion feature matrix as input, extracts the bounding boxes and confidence levels of pest and disease targets through the object detection model, generates a pest and disease distribution heat map, and eliminates low-confidence targets, improving the accuracy and robustness of identification.

[0112] The disease risk assessment module 40 is used to calculate the disease severity index based on the pest and disease distribution heat map, field environmental sensing data, and historical disease trend data, and further generate a disease risk index (BRI). This module classifies the disease-affected areas through the disease risk index, providing data support for precise pesticide application.

[0113] The UAV pesticide application path optimization module 50 is used to plan the optimal pesticide application path based on the disease risk index and UAV operation parameters using the ant colony optimization algorithm. This module optimizes the pesticide application angle and flight trajectory of the UAV in combination with wind speed and humidity parameters, reduces the amount of pesticide used, improves the operation efficiency of the UAV, and generates a precise pesticide application path.

[0114] The UAV pesticide application execution and data monitoring module 60 is used to control the UAV to perform precise pesticide application according to the optimized path and record the pesticide application position and pesticide dosage in real time. At the same time, this module obtains the meteorological data and real-time images of the UAV in the pesticide application area, and dynamically monitors the pesticide application effect based on the disease change trend sequence, providing feedback data for subsequent model optimization.

[0115] The Federated Learning Optimization and Model Update Module 70 is used to optimize and adjust the multi-modal feature mapping model and the YOLOv7-Spectral pest and disease identification model based on the application feedback data, and integrate cross-farm data through the federated learning mechanism to improve the generalization ability of the model. This module uses the Federated Averaging algorithm (FedAvg) to update the global model parameters to ensure the high efficiency and adaptability of the pest and disease identification and application path optimization model during long-term operation.

[0116] Obviously, the embodiments described above are only a part of the embodiments of the present application, rather than all the embodiments. The accompanying drawings give preferred embodiments of the present application, but do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosed content of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or perform equivalent replacements on some of the technical features. Any equivalent structure directly or indirectly using the content of the specification and drawings of the present application in other related technical fields shall be within the scope of the patent protection of the present application by the same token.

Claims

1. A real-time recognition, prevention and control decision-making method for crop diseases and pests based on multi-modal edge computing, characterized in that, The method includes the following steps: Obtain multispectral image data, infrared thermal imaging data, and NDVI index collected by the drone, and obtain temperature, humidity, wind speed, and soil moisture data from field Internet of Things sensors. Synchronously preprocess the data to generate a multimodal feature input sequence; Perform normalization processing on the multimodal feature input sequence, and perform multi-dimensional feature mapping based on the Transformer attention mechanism to generate a spectral-environment fusion feature matrix; Based on the spectral-environment fusion feature matrix and the pre-trained YOLOv7-Spectral model, perform pest and disease target detection, generate a pest and disease distribution heat map, and perform confidence screening on the recognition results to extract pest and disease feature regions; Extract spatial position features from the pest and disease distribution heat map and the data obtained from field Internet of Things sensors, and combine historical disease trend data to evaluate the severity of the disease, and generate a disease risk index (BRI) and a pesticide application priority map; Based on the pesticide application priority map and the drone operation parameters, construct an ant colony optimization path planning model, and combine the wind speed and humidity parameters to generate a drone pesticide application optimization path sequence; Obtain the drone pesticide application optimization path sequence, and control the drone to perform precise pesticide application according to the optimized path. At the same time, monitor the meteorological data and real-time images of the pesticide application area, and update the disease change trend sequence; Based on the disease change trend sequence and the feedback data after pesticide application, adjust the parameter weights of the YOLOv7-Spectral model, and optimize the disease recognition and pesticide application path planning model through federated learning to generate an updated intelligent prevention and control model; Among them, the disease risk index (BRI) is calculated by the following formula: ； Among them, is the disease risk index at the grid is the disease severity index at the grid ; is the grid environmental data at; is the weight of the disease and environmental impact factors.

2. The method according to claim 1, wherein The preprocessing of the multimodal feature input sequence includes the following steps: Perform interpolation completion and noise removal on the multispectral image data, infrared thermal imaging data, and NDVI index to generate a standardized multispectral image sequence and an infrared thermal imaging sequence; Perform time alignment and spatial registration on the field Internet of Things sensing data, and perform data normalization to generate a standardized environmental data sequence; Perform data fusion on the standardized multispectral image sequence, infrared thermal imaging sequence, and environmental data sequence to generate a preliminarily processed multimodal feature input sequence.

3. The method according to claim 1, wherein The construction of the multi-dimensional feature mapping includes the following steps: Perform feature decomposition on the multimodal feature input sequence to extract spectral reflectance, leaf surface temperature, NDVI index, and environmental factors; Use the Transformer attention mechanism to perform feature mapping on the spectral reflectance, leaf surface temperature, NDVI index, and environmental factors to generate a spectral-environment fusion feature matrix.

4. The method according to claim 1, wherein The pest and disease target detection includes the following steps: Input the spectral-environment fusion feature matrix into the pre-trained YOLOv7-Spectral model to perform pest and disease target detection, and generate a pest and disease target detection result; Based on the pest and disease target detection result, perform distribution mapping on the detection targets according to the spatial position to generate a pest and disease distribution heat map; Perform confidence screening on the pest and disease distribution heat map, eliminate low-confidence targets, and extract data on the characteristic regions of pests and diseases.

5. The method according to claim 1, characterized in that, The evaluation of the disease severity includes the following steps: Based on the data of the characteristic regions of pests and diseases and the data obtained by the field Internet of Things sensors, extract the spatial location characteristics of the target crop regions to obtain the crop spatial distribution sequence; Based on the crop spatial distribution sequence, combine the historical disease trend data, calculate the disease severity index, and generate the disease risk index (BRI).

6. The method according to claim 1, characterized in that, The optimized path sequence for drone spraying includes the following steps: Based on the spraying priority map and the drone operation parameters, construct an ant colony optimization path planning model to generate an initial drone spraying path plan; Combine the wind speed and humidity parameters to optimize the path of the initial drone spraying path plan and adjust the spraying angle to generate an optimized drone spraying path sequence; Smooth the optimized drone spraying path sequence and adjust the spraying height to generate the final precise spraying path.

7. The method according to claim 1, characterized in that, The control of the drone to perform precise spraying according to the optimized path includes the following steps: Based on the drone operation parameters, control the drone to perform precise spraying according to the optimized drone spraying path, and record the spraying position and pesticide dosage data at the same time; During the spraying process, obtain the meteorological data and real-time images of the spraying area, and compare the data with the historical data to obtain the disease change trend sequence; Based on the disease change trend sequence, predict the disease change and update the disease severity evaluation data.

8. The method according to claim 1, wherein The adjustment of the parameter weights of the YOLOv7-Spectral model includes the following steps: Obtain the feedback data after spraying, including drone operation data, crop health status, and farmland environment data, to form a spraying effect feedback sequence; Based on the spraying effect feedback sequence, adjust the parameter weights of the multi-modal feature mapping model and the YOLOv7-Spectral model to generate an optimized pest and disease recognition model.

9. The method according to claim 1, wherein The federated learning optimized prevention and control model includes the following steps: Based on the pest and disease data of multiple farms, calculate the local update parameters of each sub-model and independently train the model on each end device; Based on the local update parameters, use the federated average algorithm to calculate the global model parameters, perform model aggregation, and generate an updated intelligent prevention and control model.

10. A real-time identification, prevention and control decision-making system for crop diseases and pests based on multimodal edge computing, which is applied to the real-time identification, prevention and control decision-making method for crop diseases and pests based on multimodal edge computing according to any one of claims 1-9, characterized in that, Including: A data acquisition module for obtaining monitoring data related to crop growth and pests and diseases from multi-modal data sources; A data processing and fusion module for preprocessing the collected multi-modal data and using the Transformer attention mechanism to perform feature mapping on the multi-modal data to generate a spectral-environment fusion feature matrix; A pest and disease identification module for performing real-time target detection of crop pests and diseases based on the YOLOv7-Spectral deep learning model; A disease risk assessment module for calculating the disease severity index based on the pest and disease distribution heat map, field environment sensing data, and historical disease trend data, and further generating the disease risk index (BRI); The UAV spraying path optimization module is used to plan the optimal spraying path based on the disease risk index and UAV operation parameters by using the ant colony optimization algorithm; The UAV spraying execution and data monitoring module is used to control the UAV to perform precise spraying according to the optimized path and record the spraying position and pesticide dosage in real time; The federated learning optimization and model update module is used to optimize and adjust the multi-modal feature mapping model and the YOLOv7-Spectral pest and disease identification model based on the spraying feedback data, and integrate cross-farm data through the federated learning mechanism to improve the generalization ability of the model.

Images