Mangrove forest disease and insect pest monitoring system based on multispectral feature fusion and edge calculation

Abstract

The invention discloses a mangrove forest disease and insect pest monitoring system based on multispectral feature fusion and edge calculation. The mangrove forest disease and insect pest monitoring system comprises an unmanned aerial vehicle flying platform, a multispectral image acquisition module, an edge calculation module and a ground control end. The multi-spectral image acquisition module synchronously acquires image data of six wave bands, an improved YOLOv8 model is built in the edge calculation module, the detection precision and generalization ability are improved through a double-branch network, feature fusion, an attention mechanism and a domain adaptive algorithm, and real-time reasoning is realized through lightweight deployment; and the ground control end realizes data storage, visual display, automatic early warning and decision support. The system is high in detection precision, the tiny pest detection rate is greater than or equal to 92%, and the positioning error is less than or equal to 5m; the real-time performance is high, and the reasoning speed is smaller than or equal to 200 ms / frame; the environment adaptability is good, and the full-light condition and the cross-region monitoring requirement are covered; the monitoring efficiency is improved by more than three times, the cost is reduced by 60%, high-precision, all-weather and real-time monitoring of mangrove forest diseases and insect pests can be realized, and technical support is provided for ecological protection.

Description

Technical Field

[0001] This invention belongs to the field of forestry pest and disease monitoring and computer vision technology. Specifically, it relates to a monitoring system that uses a drone equipped with a multispectral camera and deep learning algorithms to achieve real-time identification and analysis of mangrove pests and diseases. It is applicable to pest and disease control in various mangrove reserves, wetland ecological monitoring areas and other scenarios. Background Technology

[0002] Mangroves, as important wetland ecosystems, play crucial ecological roles such as windbreak and sand fixation, water purification, and biodiversity maintenance. However, their unique growing environment makes them susceptible to pests and diseases. Common diseases include anthracnose, and pests mainly include aphids and water fleas. Currently, there are many problems with the monitoring methods for mangrove pests and diseases, which seriously affect the timeliness and effectiveness of prevention and control efforts.

[0003] Current monitoring methods rely primarily on manual inspections. This approach is severely limited by the tidal changes and complex terrain of mangrove forests, resulting in a monitoring coverage rate of less than 30%. It is not only inefficient but also requires significant investment of manpower and resources, leading to high monitoring costs. Furthermore, manual inspections struggle to detect early-stage micro-pests and hidden diseases, resulting in a high rate of missed detections. Often, by the time pests and diseases are discovered, substantial ecological damage has already occurred.

[0004] In terms of remote sensing monitoring technology, although satellite remote sensing can achieve wide-area coverage, its resolution is low, typically between 10-30 meters. This makes it unable to accurately identify specific pests and diseases, and can only provide a preliminary assessment of vegetation abnormalities, failing to meet the needs of precise prevention and control. Ordinary UAV visible light (RGB) monitoring technology is significantly affected by lighting conditions in practical applications. In complex lighting environments such as low light at dawn and dusk, and cloudy skies, its ability to extract features of early-stage anthracnose and microaphids is insufficient. It is also easily interfered with by environmental factors such as background vegetation and soil, resulting in low identification accuracy.

[0005] Furthermore, existing AI recognition models lack specific optimization for mangrove multispectral data, exhibiting poor generalization ability in cross-regional monitoring scenarios such as different mangrove reserves, leading to a significant drop in recognition accuracy when the models are transferred for application. Additionally, most of these models rely on cloud-based data processing; images collected by drones need to be transmitted to the cloud before analysis, resulting in high data transmission latency and hindering real-time early warning, thus delaying the optimal time for pest and disease control.

[0006] The existing technologies mentioned above suffer from problems such as low efficiency, difficulty in identifying minor pests and diseases, poor environmental adaptability, and lagging data processing, which seriously restrict the monitoring and control of mangrove pests and diseases. Therefore, there is an urgent need for a monitoring system that can solve the above problems. Summary of the Invention

[0007] This invention aims to overcome the technical shortcomings of existing mangrove pest and disease monitoring methods, such as low efficiency, difficulty in identifying minor pests and diseases, poor environmental adaptability, and lagging data processing. It provides a mangrove pest and disease monitoring system based on multispectral feature fusion and edge computing. Through multimodal data acquisition and enhancement, improved deep learning algorithms, and lightweight deployment of edge computing, it achieves high-precision, all-weather, and real-time monitoring of mangrove pests and diseases, providing technical support for mangrove ecological protection.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] A mangrove pest and disease monitoring system based on multispectral feature fusion and edge computing includes an unmanned aerial vehicle (UAV) flight platform, a multispectral image acquisition module, an edge computing module, and a ground control terminal. Data transmission and command interaction between the modules are achieved through wireless communication. The specific structure and connection relationships are as follows:

[0010] 1. Unmanned Aerial Vehicle (UAV) Flight Platform

[0011] The drone flight platform is designed to withstand winds up to level 7, equipped with an RTK positioning system achieving a positioning accuracy of ±10cm. It has a flight time of 60 minutes and can stably fly at an altitude of 30-50 meters, capturing images with a resolution of 0.5cm / pixel. It is also capable of operating in complex environments such as dawn, dusk, and low light conditions. The drone flight platform has a preset flight path planning function, allowing it to fly automatically along a pre-defined route while maintaining an 80% image overlap rate during flight, ensuring complete coverage of the monitored area.

[0012] 2. Multispectral Image Acquisition Module

[0013] The multispectral image acquisition module uses a RedEdge-PMX multispectral camera, which supports 5-band imaging with an imaging wavelength range of 400-900nm. It can simultaneously acquire images in the visible (RGB), blue (475nm), green (560nm), red (668nm), red-edge (717nm), and near-infrared (842nm) bands, and supports vegetation index calculations such as NDVI and PRI. The multispectral image acquisition module is fixedly connected to the UAV flight platform and moves synchronously with it, acquiring multispectral image data of the monitored area in real time and transmitting the acquired data to the edge computing module in real time.

[0014] 3. Edge computing module

[0015] The edge computing module uses the NVIDIA Jetson AGX Orin edge computing box, which has an improved YOLOv8 model built in. This model uses knowledge distillation technology to compress parameters by 40% and combines TensorRT quantization technology to achieve lightweight deployment, enabling local real-time inference on the edge computing module.

[0016] The improved YOLOv8 model constructs a dual-branch network structure of "multispectral feature extraction + RGB image enhancement", in which:

[0017] The multispectral feature extraction branch extracts features from multispectral band data through FPN (Feature Pyramid Network), and captures early disease characteristics by utilizing the high sensitivity of the red edge (717nm) and near-infrared (842nm) bands to vegetation health.

[0018] The RGB image enhancement branch preprocesses visible light images to improve the clarity of image texture features;

[0019] The features extracted by the two branches are fused through FPN (Feature Pyramid Network) to enhance the distinguishability of pest and disease features;

[0020] The model incorporates a Transformer attention mechanism to enhance pixel-level correlations in feature maps, and performs feature enhancement for tiny targets such as aphids (0.5-2mm) and water fleas.

[0021] A Domain Adaptive Neural Network (DANN) algorithm is introduced to extract common features of various pests and diseases across regions through adversarial learning, thereby improving the model's cross-regional generalization ability.

[0022] The edge computing module also integrates a data augmentation unit, which uses spectral distortion simulation technology to generate simulated data under complex lighting conditions such as dawn / dusk and cloudy conditions by adding sensor noise, etc., for model training and to improve the robustness of the model.

[0023] 4. Ground control terminal

[0024] The ground control terminal includes a data storage unit, a visualization unit, and an early warning and decision-making unit. The ground control terminal achieves wireless communication with the edge computing module through a 4G / 5G network.

[0025] The data storage unit stores the raw multispectral image data and pest detection results transmitted by the edge computing module, and constructs a hierarchical dynamic pest database. The visualization unit uses a GIS heat map to intuitively display the location and density distribution of pests and diseases, and can display information such as the types and severity of detected pests and diseases. The early warning decision unit presets a pest and disease density threshold. When the pest and disease density detected by the edge computing module exceeds the threshold (e.g., 50 pests / plant), it automatically triggers an early warning mechanism, sends early warning information to relevant personnel, and provides preliminary prevention and control decision suggestions.

[0026] 5. Data transmission link

[0027] The UAV flight platform and the multispectral image acquisition module are connected via a wired data interface to ensure the stability and real-time performance of image data transmission; the multispectral image acquisition module and the edge computing module are connected via a high-speed data transmission bus to achieve real-time transmission of acquired data; the edge computing module and the ground control terminal are connected via a 4G / 5G wireless communication network to achieve the transmission of detection results, early warning information and other data, as well as the issuance of commands from the ground control terminal to the UAV flight platform and the edge computing module.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] 1. High detection accuracy

[0030] This invention simultaneously acquires image data across six bands using a multispectral image acquisition module. Utilizing the high sensitivity of the red-edge and near-infrared bands to vegetation health, it effectively captures early disease characteristics. The improved YOLOv8 model employs a dual-branch network structure and FPN feature fusion technology, significantly enhancing the spectral feature discrimination of pests and diseases. Combined with the Transformer attention mechanism, it specifically enhances features for small targets, increasing the detection rate of small pests and diseases such as aphids from the traditional 75% to over 92%. The model's mAP@0.5 reaches 92.3%, with a positioning error ≤5 meters, demonstrating significantly superior detection accuracy compared to existing technologies.

[0031] 2. High real-time performance

[0032] The edge computing module employs knowledge distillation and TensorRT quantization technologies to achieve lightweight model deployment. It can perform local real-time inference on the onboard edge computing unit with an inference speed of ≤200ms / frame. In practical applications, the average inference speed is 150ms / frame. It eliminates the need to transmit data to the cloud for processing, greatly reducing data processing latency and enabling real-time detection and early warning of pests and diseases. This solves the problem of lagging data processing in existing technologies.

[0033] 3. Good environmental adaptability

[0034] The multispectral image acquisition module of this invention supports image acquisition in complex environments such as dawn and dusk with low light. The data enhancement unit generates simulated data under complex lighting conditions through spectral distortion simulation technology, which improves the robustness of the model. The improved YOLOv8 model introduces a domain adaptation algorithm, which enhances the generalization ability of cross-regional monitoring. This improves the accuracy of the model on the validation set by 8% in complex environments such as dawn and dusk with low light and cloudy conditions. It can adapt to the monitoring needs of different mangrove reserves, and its environmental adaptability is significantly better than that of existing monitoring technologies.

[0035] 4. High monitoring efficiency and low cost

[0036] The drone flight platform can fly automatically along a preset route, achieving 100% monitoring coverage. Compared to manual inspection, the monitoring efficiency is increased by more than 3 times. It requires no large-scale human intervention, and the drone has a long flight time, allowing it to complete a large-scale monitoring task in a single flight. Monitoring costs are reduced by 60%, effectively solving the problems of low monitoring efficiency and high cost of existing technologies.

[0037] 5. Comprehensive functions and strong practicality

[0038] This invention constructs a closed-loop business process of "data collection - intelligent analysis - hierarchical early warning - decision support". The ground control terminal can realize functions such as data storage, visualization, automatic early warning and decision suggestions, providing staff with comprehensive pest and disease monitoring information and prevention and control guidance. It is highly practical and can effectively support the precise prevention and control of mangrove pests and diseases, and reduce the damage of pests and diseases to the mangrove ecosystem. Attached Figure Description

[0039] In the attached diagram:

[0040] Figure 1 This is the flow control logic diagram of the present invention;

[0041] Figure 2 This is a schematic diagram of the user interface of the present invention. Detailed Implementation

[0042] Example 1

[0043] This embodiment provides a mangrove pest and disease monitoring system based on multispectral feature fusion and edge computing, with the following specific structure:

[0044] The drone flight platform uses the DJI Matrice 350 drone, which is resistant to level 7 winds, equipped with an RTK positioning system with a positioning accuracy of ±10cm, a flight time of 60 minutes, a maximum flight altitude of 50 meters, a minimum flight altitude of 30 meters, and a flight speed that can be adjusted between 5-15m / s. It supports preset flight routes and can set a flight overlap rate of 80%.

[0045] The multispectral image acquisition module uses a RedEdge-PMX multispectral camera, which is fixedly mounted on the bottom of the UAV flight platform and connected to the UAV flight platform via a wired data interface. The specific parameters of the camera are as follows: imaging bands include visible light (RGB), blue light (475nm), green light (560nm), red light (668nm), red edge (717nm), and near-infrared (842nm), with a band width of ±10nm, an image resolution of 12 million pixels, support for NDVI and PRI vegetation index calculation, and a data transmission rate of 100Mbps.

[0046] The edge computing module uses the NVIDIA Jetson AGX Orin edge computing box, which is connected to the multispectral image acquisition module through a high-speed data transmission bus. The hardware parameters of the edge computing box are: 12 CPU cores, 200 TOPS GPU computing power, 32GB memory, 1TB storage capacity, and support for TensorRT 8.0 and above, which can meet the real-time inference requirements of lightweight models.

[0047] The improved YOLOv8 model built into the edge computing module is trained in the following way:

[0048] 1. Data collection: Multispectral image data of 100,000 samples were collected in different mangrove reserves, including healthy vegetation, anthrax-infected vegetation, aphid-infested vegetation, and katydid-infested vegetation, covering different lighting conditions such as dawn, dusk, cloudy, and sunny days.

[0049] 2. Data Augmentation: Utilizing the spectral distortion simulation technology of the data augmentation unit, sensor noise is added, brightness and contrast are adjusted, etc., to generate 200,000 enhanced sample data.

[0050] 3. Model Training: The original sample data and augmented sample data are divided into training set and validation set in a 7:3 ratio. The Adam optimizer is used with a learning rate of 0.001 and 100 training rounds. The trained model parameters are compressed by 40% using knowledge distillation technology, and then the model is quantized using TensorRT quantization technology to obtain a lightweight model.

[0051] The ground control unit uses an industrial-grade tablet computer equipped with a 4G / 5G communication module, enabling wireless communication with the edge computing module. The data storage unit uses a 2TB solid-state drive to store raw image data and detection results. The visualization unit uses a GIS map engine to generate heat maps of pest and disease distribution, displaying information such as pest and disease types, locations, densities, and severity. The early warning decision unit presets aphid density thresholds of 50 insects / plant, water flea density thresholds of 30 insects / plant, and anthracnose incidence area thresholds of 10%. When the detection results exceed the corresponding thresholds, it automatically sends early warning information to staff via SMS, APP push notifications, and other means.

[0052] The working process of this embodiment is as follows:

[0053] 1. Flight route planning: Staff send flight route planning instructions to the UAV flight platform through the ground control terminal, setting the boundary of the monitoring area, the flight altitude of 40 meters, the flight speed of 10 m / s, and the overlap rate of 80%.

[0054] 2. Image Acquisition: The UAV flight platform flies along a preset route, and the multispectral image acquisition module simultaneously acquires image data in 6 bands at a frequency of 10 frames / second, and transmits the acquired data to the edge computing module in real time;

[0055] 3. Real-time inference: After receiving image data, the edge computing module performs real-time inference using the improved YOLOv8 model. First, features are extracted by the multispectral feature extraction branch and the RGB image enhancement branch, respectively. Then, the features are fused by the FPN feature fusion network. The features of small targets are enhanced by the Transformer attention mechanism module. Finally, the detection results such as the type, location, and density of pests and diseases are output by the domain adaptive algorithm module.

[0056] 4. Data transmission and display: The edge computing module transmits the detection results to the ground control terminal in real time, the data storage unit stores the detection results and raw image data, and the visualization display unit displays the distribution of pests and diseases in the form of a GIS heat map;

[0057] 5. Early warning and decision-making: The early warning and decision-making unit judges the detection results. When the aphid density in a certain area is 60 aphids / plant, exceeding the preset threshold, it automatically sends an early warning message to the staff, indicating that there is a risk of aphid infestation in the area, and recommends taking control measures by spraying biological pesticides.

[0058] 6. Follow-up monitoring: Based on the early warning information and decision-making suggestions, staff can issue key monitoring instructions to the UAV flight platform through the ground control terminal. The UAV flight platform will then go to the area to conduct close-range, high-frequency monitoring and continuously report on changes in pests and diseases.

[0059] Example 2

[0060] The difference between this embodiment and Embodiment 1 is that the UAV flight platform adopts a customized long-endurance UAV, with an endurance time extended to 90 minutes, a maximum flight altitude of 60 meters, and support for automatic obstacle avoidance, enabling it to adapt to more complex terrain environments; the multispectral image acquisition module uses a RedEdge-PRO multispectral camera, adding a short-wave infrared band (1650nm), which can better capture deep-seated disease characteristics; the GPU computing power of the edge computing module is increased to 300 TOPS, and the model inference speed is increased to 120ms / frame; the ground control terminal adds a data export function, which can export the detection results and raw data to Excel, PDF and other formats, facilitating subsequent data analysis and report generation.

[0061] The working process of this embodiment is basically the same as that of Embodiment 1. The newly added shortwave infrared band can effectively identify early diseases inside vegetation, enabling the detection of diseases such as anthracnose to be advanced by 3-5 days, further improving the timeliness of pest and disease monitoring; the automatic obstacle avoidance function enables the drone to fly safely in dense mangrove areas, expanding the monitoring range; the faster inference speed and data export function improve work efficiency and are more suitable for monitoring work in large-scale mangrove reserves.

[0062] Example 3

[0063] The difference between this embodiment and Embodiment 1 is that the improved YOLOv8 model introduces a bidirectional attention mechanism, which further enhances the feature extraction capability of small targets; the multispectral image acquisition module supports real-time video stream acquisition with a resolution of 4K, which can more clearly display the morphological characteristics of pests and diseases; the ground control terminal adds a remote control function, which allows staff to remotely adjust the flight parameters of the UAV flight platform, the acquisition parameters of the multispectral image acquisition module, and the detection threshold of the edge computing module through the ground control terminal.

[0064] In the operation of this embodiment, 4K video stream acquisition can provide staff with a more intuitive view of the pest and disease situation on site. The two-way attention mechanism increases the detection rate of small pests and diseases such as aphids to over 95%. The remote control function allows staff to flexibly adjust equipment parameters according to the actual monitoring situation, improving the system's flexibility and adaptability. It is suitable for scenarios with complex pest and disease situations that require dynamic adjustment of monitoring strategies.

[0065] Example 4

[0066] The difference between this embodiment and Embodiment 1 is that the system adds a drone cluster collaborative monitoring function, which can deploy 3-5 drone flight platforms simultaneously for collaborative monitoring. The flight paths and data sharing between drones are realized through the ground control terminal to avoid monitoring overlap and omissions. The edge computing module supports 5G edge cloud collaboration. When the processing capacity of the edge computing module of a single drone is insufficient, some data can be transmitted to the edge cloud for collaborative processing to ensure the real-time performance and accuracy of the detection.

[0067] This embodiment is applicable to the monitoring of ultra-large-scale mangrove reserves. The collaborative monitoring of drone swarms can improve the monitoring efficiency by 5-8 times. Three drones working together can complete the monitoring task of 1,000 acres of mangroves within 2 hours. The 5G edge cloud collaborative function ensures the stable operation of the system in complex monitoring scenarios, further improving the practicality and reliability of the system.

Claims

1. A mangrove pest and disease monitoring system based on multispectral feature fusion and edge computing, comprising an unmanned aerial vehicle (UAV) flight platform, a multispectral image acquisition module, an edge computing module, and a ground control terminal, characterized in that: The multispectral image acquisition module is fixedly connected to the UAV flight platform. The edge computing module is connected to the multispectral image acquisition module via a high-speed data transmission bus. The ground control terminal is connected to the edge computing module via a 4G / 5G wireless communication network. The UAV flight platform has RTK positioning and preset flight path functions, with a positioning accuracy of ±10cm, a flight time of ≥60 minutes, a flight altitude of 30-50 meters, and an image acquisition overlap rate of ≥80%. The multispectral image acquisition module can simultaneously acquire images in the visible light (RGB), blue light (475nm), green light (560nm), red light (668nm), red edge (717nm), and near-infrared (842nm) bands. The edge computing module has a built-in improved YOLOv8 model and uses knowledge distillation and TensorRT quantization technology to achieve lightweight deployment with an inference speed of ≤200ms / frame. The ground control terminal includes a data storage unit, a visualization display unit, and an early warning and decision-making unit, which can realize data storage, visualization display, automatic early warning, and decision-making suggestions.

2. The mangrove pest and disease monitoring system based on multispectral feature fusion and edge computing according to claim 1, characterized in that: The drone flight platform is designed to withstand winds up to level 7, has a flight speed of 5-15 m / s, supports automatic obstacle avoidance, and can adapt to complex environments such as dawn, dusk, and low light.

3. The mangrove pest and disease monitoring system based on multispectral feature fusion and edge computing according to claim 1, characterized in that: The multispectral image acquisition module is a RedEdge-PMX multispectral camera with an image resolution of ≥12 million pixels, supports NDVI and PRI vegetation index calculation, and has a data transmission rate of ≥100Mbps.

4. The mangrove pest and disease monitoring system based on multispectral feature fusion and edge computing according to claim 1, characterized in that: The improved YOLOv8 model constructs a dual-branch network structure of "multispectral feature extraction + RGB image enhancement", achieves feature fusion through FPN feature pyramid network, introduces Transformer attention mechanism and domain adaptive algorithm (DANN), the model mAP@0.5≥92.3%, and the positioning error ≤5 meters.

5. The mangrove pest and disease monitoring system based on multispectral feature fusion and edge computing according to claim 1, characterized in that: The edge computing module uses the NVIDIA Jetson AGX Orin edge computing box, with a CPU core count of ≥12 cores, GPU computing power of ≥200 TOPS, memory of ≥32GB, storage capacity of ≥1TB, and supports TensorRT 8.0 and above.

6. The mangrove pest and disease monitoring system based on multispectral feature fusion and edge computing according to claim 1, characterized in that: The edge computing module also integrates a data augmentation unit, which uses spectral distortion simulation technology to generate simulated data under complex lighting conditions for model training to improve robustness.

7. The mangrove pest and disease monitoring system based on multispectral feature fusion and edge computing according to claim 1, characterized in that: The data storage unit uses a solid-state drive with a capacity of ≥2TB to store raw multispectral image data and pest detection results data; the visualization unit uses a GIS map engine to generate a heat map of pest distribution; the early warning decision unit presets pest density thresholds and disease area percentage thresholds, and automatically sends early warning information when the thresholds are exceeded.

8. The mangrove pest and disease monitoring system based on multispectral feature fusion and edge computing according to claim 1, characterized in that: The ground control terminal uses an industrial-grade tablet computer equipped with a 4G / 5G communication module and supports data export function, which can export the test results and raw data to Excel and PDF formats.

9. The mangrove pest and disease monitoring system based on multispectral feature fusion and edge computing according to any one of claims 1-8, characterized in that: The drone flight platform can achieve cluster collaborative monitoring. When 3-5 drones work together, the monitoring efficiency is improved by 5-8 times, and it supports edge cloud collaborative data processing.

Images