Termite intelligent monitoring method based on multi-modal perception and edge computing

Abstract

A termite intelligent monitoring method based on multimodal perception and edge computing includes the following steps: Step 1: Data is collected using hardware equipped with multimodal sensors to obtain termite activity images and environmental data of the target area according to a planned path; Step 2: An improved target detection model YOLOv5n-CBAM-BiFPN is built and deployed at the edge to perform real-time target detection on the collected image data, identify termite activity areas, and transmit the detection results and key data to the cloud; Step 3: After receiving the data, the cloud triggers multi-model collaborative analysis, uses the MiDaS depth estimation model to construct the three-dimensional structure of termite nests, and uses the MaxEnt-Gaussian kernel coupling model to predict the spatiotemporal spread trend of termites; Step 4: Based on the analysis results of Step 3, a comprehensive inference is performed through the RAGFlow and DeepSeek dual-engine decision system to automatically generate a comprehensive control plan that includes specific control measures and operational guidance; Step 5: Termite infestation early warning information, three-dimensional heat maps, and the generated control plan are pushed to the user interaction terminal, and the data collection terminal is scheduled to perform precise application or verification according to the plan instructions.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to termite control technology based on computer technology, specifically to an intelligent termite monitoring method based on multimodal perception and edge computing. Background Technology

[0002] Current termite control methods mainly rely on manual inspections, which suffer from low efficiency, high missed detection rates, and slow response times. With the development of information technology, some intelligent monitoring platforms have begun to emerge. For example, patent document with application publication number CN113873440A describes a termite control platform.

[0003] However, existing technologies, including this public platform, still face several multi-dimensional technical bottlenecks that urgently need to be overcome when facing the complex environments in the wild and the practical needs of precise prevention and control: Firstly, regarding the detection of tiny targets and edge deployment: Termites are small and often hide in complex outdoor environments such as vegetation and soil, highly overlapping with background features. Existing target detection algorithms often struggle to preserve shallow details and extract deep semantics when extracting features; relying solely on traditional attention mechanisms or feature splicing can easily lead to the loss of scale variation information or the amplification of shallow background noise, which is then transmitted to deeper networks, resulting in missed or false detections of tiny termite targets. Simultaneously, high-precision, complex models often have a large number of parameters, making direct deployment on computationally limited edge devices in the field difficult; while transmitting all massive amounts of multimodal raw data back to the cloud for processing would face enormous network bandwidth pressure and communication latency, making it impossible to achieve highly timely closed-loop control. Secondly, regarding the deep fusion of multimodal data: existing multi-source data fusion typically employs traditional algorithms based on feature stitching or simple overlay. This fusion method represents a shallow fusion at the data or decision level, failing to fully consider the inherent correlation and complementarity at the feature level between different modal data (such as the high-resolution geometric features of visible light images, the temperature anomaly features of thermal infrared images, and the macroscopic situation of environmental sensor data), resulting in insufficient feature-level fusion. This deficiency leads to poor robustness of the platform when facing complex terrain and variable environmental conditions in the field. Thirdly, regarding three-dimensional spatial reconstruction and dynamic diffusion prediction: current intelligent monitoring methods mostly rely on two-dimensional planar identification of surface termite activity, lacking the ability to estimate the depth of the three-dimensional structure of hidden nests from two-dimensional images. This makes it difficult to accurately locate the main termite chamber buried deep underground in actual control efforts. Furthermore, in predicting termite population diffusion, existing platforms largely depend on static species distribution assessments, lacking dynamic spatiotemporal evolution simulations that incorporate a time dimension. This makes it impossible to accurately predict future termite infestation diffusion paths and outbreak hotspots in response to changes in environmental factors. Fourth, regarding intelligent decision-making and the closed-loop control mechanism: most existing monitoring platforms can only complete the front-end "detection and alarm" tasks. Subsequent control measures (such as pesticide selection, concentration ratio, and physical barrier setup) still heavily rely on the personal experience of plant protection experts for manual intervention. The system lacks an internal reasoning mechanism that deeply couples real-time monitoring multidimensional data, external environmental data, and a massive professional pest and disease knowledge base. This makes it impossible to automatically generate accurate and scientific operational guidance plans, and thus difficult to truly achieve a fully intelligent closed loop from "ubiquitous application" to "precision targeting."

[0004] Therefore, there is an urgent need to develop a termite intelligent monitoring and positioning platform and method that can deeply integrate multimodal data and achieve high precision and high robustness in complex environments. Summary of the Invention

[0005] To address the shortcomings of existing termite control and monitoring methods, such as the ease with which small targets are missed or misdetected, the shallow fusion of multimodal data leading to a lack of three-dimensional spatial positioning and dynamic prediction capabilities, and the high dependence of control plan generation on human experience, this invention aims to provide a termite intelligent monitoring platform and method based on multimodal perception and edge computing, so as to achieve high-precision identification of termite activity, three-dimensional prediction, and intelligent closed-loop control across the entire chain.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A termite intelligent monitoring method based on multimodal sensing and edge computing includes the following steps: Step 1: Collect data using hardware equipped with multimodal sensors, and obtain termite activity images and environmental data of the target area according to the planned path; Step 2: Build and deploy the improved target detection model YOLOv5n-CBAM-BiFPN at the edge to perform real-time target detection on the collected image data, identify termite activity areas, and transmit the detection results and key data to the cloud; Step 3: After receiving data in the cloud, trigger multi-model collaborative analysis, use the MiDaS depth estimation model to construct the three-dimensional structure of termite nests, and use the MaxEnt-Gaussian kernel coupling model to predict the spatiotemporal spread trend of termites. Step 4: Based on the analysis results of Step 3, the RAGFlow and DeepSeek dual-engine decision-making system is used for comprehensive reasoning to automatically generate a comprehensive prevention and control plan that includes specific prevention and control measures and operational guidance. Step 5: Push the ant infestation early warning information, 3D heat map and generated prevention and control plan to the user interaction terminal, and schedule the data collection terminal to perform precise application or verification according to the plan instructions.

[0007] In step 2, before inputting the data into the improved target detection model, the specific procedure is as follows: preprocessing is performed through a three-level serial image filtering mechanism; The first stage calculates the image gradient magnitude based on the Sobel edge detection operator and removes image frames lacking effective information due to motion blur or defocus. The second stage converts the preserved image to the HSV color space, and uses a lightweight UNet+ network for adaptive noise reduction and contrast enhancement to suppress thermal noise in thermal imaging images and uneven illumination in visible light images. The third stage involves inputting the high-quality image processed in the first two stages into the improved YOLOv5n-CBAM-BiFPN model to perform proficient termite target detection.

[0008] In step 2, the construction and operation mechanism of the improved target detection model YOLOv5n-CBAM-BiFPN is as follows: After the outputs of stages 3, 4, and 5 of the YOLOv5n backbone network, a CBAM attention mechanism is embedded. This mechanism enhances the model's response to small termite targets through dual-path adaptive recalibration in both channel and spatial dimensions, and avoids the premature loss of shallow detail features. Simultaneously, the original PANet is replaced with a BiFPN weighted bidirectional feature pyramid. Skip residual paths are introduced into the cross-scale connections of BiFPN, allowing the CBAM-enhanced features to retain the original scale information for BiFPN to adaptively select. This forms a non-linear multi-path interaction between shallow detail features and deep semantic features, overcoming interference from complex backgrounds and improving the detection recall rate of multi-scale termite targets.

[0009] In step 3, the specific implementation of the multi-model collaborative analysis is as follows: First, the cloud-based system performs ROI cropping on the termite activity areas detected at the edge. Then, SIFT and RANSAC algorithms are used to achieve pixel-level registration of visible light and thermal imaging images. The registered images are input into a lightweight modified MiDaS model to output a high-resolution depth map. By fusing depth maps from multiple perspectives, point cloud data is generated, automatically constructing a three-dimensional model containing structures such as the main termite chamber and termite tunnels. Subsequently, the collected environmental data on temperature, humidity, soil type, and human activity factors are fused to drive the MaxEnt-Gaussian kernel coupling model for termite spatiotemporal diffusion prediction. The core formula is expressed as: P(x, y, t) = G(Δx, Δy, t)·D(θ)dΔxdΔy; where P(x, y, t) represents the termite's time... t The probability of occurrence at spatial coordinates (x, y), G(Δx, Δy, t) is the Gaussian kernel function simulating spatial diffusion, and D(θ) is the environmental dependence matrix dynamically adjusted according to real-time environmental factors, to output a heat map of future temporal and spatial distribution.

[0010] In step 4, the specific operating strategy of the RAGFlow and DeepSeek dual-engine decision system is as follows: Upon receiving reported features including location, ant species, and severity of damage, the RAGFlow engine first retrieves the most relevant similar cases from a pre-built professional knowledge base containing a large number of papers, cases, and procedures based on the problem features, using a formula to calculate cosine similarity. Subsequently, the DeepSeek large language model further integrates real-time 3D nest models, soil parameter monitoring data, weather forecasts, and external data from ecological red lines based on the retrieved cases, performing multi-parameter comprehensive reasoning and decision-making to dynamically generate a detailed prevention and control plan that includes pesticide type, concentration ratio, precise application point map, and physical barrier blocking suggestions.

[0011] The improved target detection model YOLOv5n-CBAM-BiFPN consists of a backbone network, a feature fusion network, and a detection head. The backbone network is used to extract multi-level features of termite targets from complex backgrounds, the feature fusion network is used to perform cross-scale weighted concatenation of features to enhance the representation of small targets, and the detection head is used to output the final termite location and category information.

[0012] The data processing and input / output process of the backbone network is as follows: The preprocessed three-channel termite image matrix is ​​input to the first Focus module for slicing to obtain feature F1; Feature F1 is input to the first Conv module for processing and then to the first C3 module for feature extraction, and then to the first CBAM module for processing to obtain feature F2; Feature F2 is input to the second Conv module for processing and then to the second C3 module, and then to the second CBAM module for recalibration, outputting a first-scale termite feature map P3, which is used to focus on the local texture features of tiny termite targets; The first-scale termite feature map P3 is input to the third Conv module for processing and then to the third C3 module, and then to the third CBAM module for recalibration, outputting a second-scale termite feature map P4; The second-scale termite feature map P4 is input to the fourth Conv module for processing and then to the SPP spatial pyramid pooling module to increase the receptive field, then to the fourth C3 module, and finally to the fourth CBAM module for recalibration, outputting a third-scale termite feature map P5, which is used to extract deep semantic features of termite nests or high-density termite colonies.

[0013] The first to fourth CBAM modules mentioned above adopt the same internal network architecture. The specific processing procedure is as follows: the features output by the previous C3 module are input to the channel attention module. The spatial dimension is compressed by the global max pooling layer and the average pooling layer, respectively. After calculation and merging by the multilayer perceptron, the channel attention weight tensor is output. The channel attention weight tensor is multiplied element-wise with the original input features to obtain the channel refined features, thereby effectively suppressing the invalid channel noise interference caused by soil texture, vegetation occlusion and other factors in complex field environments. Subsequently, the channel refined features are input to the spatial attention module. After pooling and concatenation along the channel dimension, the spatial attention weight tensor is calculated by the spatial convolution layer and output. The spatial attention weight tensor is multiplied element-wise with the channel refined features to output the final recalibrated features, thereby accurately highlighting the geometric outline and physical location of the termite individual in the spatial dimension.

[0014] The aforementioned feature fusion network adopts a BiFPN structure. Its specific processing steps are as follows: The third-scale termite feature map P5 is input to the fifth Conv module for dimensionality reduction, then input to the first Upsample module for upsampling, and fused with the second-scale termite feature map P4 by the first Concat module. This is then processed sequentially by the fifth C3 module and the fifth CBAM module to obtain intermediate feature M4. The intermediate feature M4 is then input to the sixth Conv module for dimensionality reduction, then input to the second Upsample module for upsampling, and fused with the first-scale termite feature map P3 by the second Concat module. This is then processed sequentially by the sixth C3 module and the sixth CBAM module to obtain the fused first-scale output feature O3. This cross-scale skip connection preserves the shallow details of a single termite's tiny target to the maximum extent. The fused first-scale output feature O3 is then input to the seventh Conv module for downsampling, and fused with the intermediate feature M4 by the third Concat module. This is then processed sequentially by the seventh C3 module and the seventh CBAM module to obtain the fused second-scale output feature O4. The fused second-scale output feature O4 is input to the eighth Conv module for downsampling, and then input to the fourth Concat module for fusion with the feature processed by the fifth Conv module. The fused third-scale output feature O5 is obtained by sequentially processing by the eighth C3 module and the eighth CBAM module. Through the above top-down and bottom-up multi-path interaction, the problem of abrupt scale change of termite targets at different physical distances is effectively overcome.

[0015] The specific processing procedure of the above detection head is as follows: the fused first-scale output feature O3, second-scale output feature O4, and third-scale output feature O5 are input to the ninth, tenth, and eleventh Conv modules respectively for channel dimension adjustment; the adjusted feature matrices of the three dimensions are input to the three corresponding branches of the Detect module for single forward inference; the Detect module directly outputs a prediction tensor containing the predicted anchor box bounding box coordinate offset, target confidence, and termite category probability in three dimensions. After filtering by the non-maximum suppression algorithm, it outputs accurate two-dimensional physical coordinates and category information of termites, providing a high-confidence spatial positioning basis for subsequent cloud-based construction of a three-dimensional nest model and generation of precise pesticide application schemes.

[0016] Compared with existing technologies, this invention provides a termite intelligent monitoring method based on multimodal perception and edge computing. By constructing a three-level collaborative architecture of "end-edge-cloud," it achieves end-to-end intelligent operation from front-end multimodal data acquisition and lightweight real-time inference at the edge, to deep spatial analysis and large-model decision-making in the cloud. Compared with existing technologies, this invention has the following significant advantages: 1) Significantly improves the detection accuracy and robustness of small targets in complex field environments: This invention innovatively constructs an improved YOLOv5n-CBAM-BiFPN lightweight target detection model on the edge computing side. By selectively embedding CBAM channel and spatial dual-path attention mechanisms deep into the backbone network, and introducing a BiFPN bidirectional feature pyramid with skip residual paths in the feature fusion stage, the model can effectively suppress complex background noise such as soil texture and vegetation occlusion while preserving the shallow detail features of a single termite target to the greatest extent. This design effectively overcomes the technical bottleneck of single improvements being unable to simultaneously achieve "anti-background interference" and "multi-scale adaptation," significantly reducing the false negative and false positive rates of termite targets. At the same time, the lightweight edge deployment combined with a three-level preprocessing filtering mechanism greatly reduces the bandwidth pressure of uploading invalid data, ensuring the system's real-time response capability under high-latency network conditions in the field. 2) Achieving high-precision 3D spatial localization of concealed nests and prediction of population dynamics and spatiotemporal evolution: This invention overcomes the limitations of traditional systems that only perform shallow fusion of two-dimensional images. Instead, it achieves deep spatial correlation of multimodal data in the cloud. On one hand, by introducing the MiDaS depth estimation model combined with the SIFT+RANSAC image registration algorithm, a millimeter-level 3D model of the nest, including structures such as the main ant chamber and ant tunnels, can be automatically constructed from 2D thermal imaging and visible light images, solving the problem of locating hidden underground hazards. On the other hand, the system innovatively extends the static MaxEnt model to include a time dimension. tThe Gaussian kernel coupled with the diffusion coefficient D dynamic propagation model can combine real-time meteorological and soil data to accurately predict the diffusion heat map of termite populations in the future time and space, realizing the leap from "post-event alarm" to "advanced spatiotemporal early warning"; 3) Constructing a dual-engine system of "large model + knowledge base" to achieve intelligent closed-loop decision-making from "ubiquitous application" to "precise targeting": This invention breaks through the bottleneck of traditional prevention and control relying heavily on human experience, pioneering a dual-engine decision-making system integrating the RAGFlow retrieval enhancement framework and the DeepSeek large language model. This system can perform deep cross-reasoning between cloud-generated 3D situation maps, real-time environmental parameters, and a massive professional pest and disease knowledge base, automatically and dynamically generating detailed prevention and control plans that include precise pesticide types, concentration ratios, precise application 3D coordinates, and physical blocking suggestions. This not only greatly improves the scientific rigor and effectiveness of prevention and control but also, through plan distribution and scheduling of front-end equipment, truly establishes a closed-loop process of "intelligent monitoring—3D deduction—scientific decision-making—precise execution," significantly reducing the impact of chemical pesticide abuse on the ecological environment. Attached Figure Description

[0017] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a schematic diagram of the overall process of the present invention; Figure 2 This is a schematic diagram of the improved YOLOv5n-CBAM-BiFPN network architecture in this invention; Figure 3 This is a schematic diagram illustrating the principle of the MaxEnt-Gaussian kernel coupling model in this invention; Figure 4 This is a schematic diagram of the workflow of the RAGFlow+DeepSeek decision engine in this invention; Figure 5 This is a schematic diagram illustrating the process of applying the MiDaS depth estimation model to the 3D reconstruction of a nest in this invention. Figure 6 This is a flowchart illustrating the application of the SORT target tracking algorithm in the analysis of individual termite behavior in this invention. Figure 7 This is a comparison chart of the detection results of the model of this invention on a self-built termite dataset. Detailed Implementation

[0018] like Figure 1 As shown, a termite intelligent monitoring method based on multimodal perception and edge computing includes the following steps: Step 1: Collect data using hardware equipped with multimodal sensors, and obtain termite activity images and environmental data of the target area according to the planned path; Step 2: Build and deploy the improved target detection model YOLOv5n-CBAM-BiFPN at the edge to perform real-time target detection on the collected image data, identify termite activity areas, and transmit the detection results and key data to the cloud; Step 3: After receiving data in the cloud, trigger multi-model collaborative analysis, use the MiDaS depth estimation model to construct the three-dimensional structure of termite nests, and use the MaxEnt-Gaussian kernel coupling model to predict the spatiotemporal spread trend of termites. Step 4: Based on the analysis results of Step 3, the RAGFlow and DeepSeek dual-engine decision-making system is used for comprehensive reasoning to automatically generate a comprehensive prevention and control plan that includes specific prevention and control measures and operational guidance. Step 5: Push the ant infestation early warning information, 3D heat map and generated prevention and control plan to the user interaction terminal, and schedule the data collection terminal to perform precise application or verification according to the plan instructions.

[0019] In step 2, before inputting the data into the improved target detection model, the specific procedure is as follows: preprocessing is performed through a three-level serial image filtering mechanism; The first stage calculates the image gradient magnitude based on the Sobel edge detection operator and removes image frames lacking effective information due to motion blur or defocus. The second stage converts the preserved image to the HSV color space, and uses a lightweight UNet+ network for adaptive noise reduction and contrast enhancement to suppress thermal noise in thermal imaging images and uneven illumination in visible light images. The third stage involves inputting the high-quality image processed in the first two stages into the improved YOLOv5n-CBAM-BiFPN model to perform proficient termite target detection.

[0020] In step 2, the construction and operation mechanism of the improved target detection model YOLOv5n-CBAM-BiFPN is as follows: After the outputs of stages 3, 4, and 5 of the YOLOv5n backbone network, a CBAM attention mechanism is embedded. This mechanism enhances the model's response to small termite targets through dual-path adaptive recalibration in both channel and spatial dimensions, and avoids the premature loss of shallow detail features. Simultaneously, the original PANet is replaced with a BiFPN weighted bidirectional feature pyramid. Skip residual paths are introduced into the cross-scale connections of BiFPN, allowing the CBAM-enhanced features to retain the original scale information for BiFPN to adaptively select. This forms a non-linear multi-path interaction between shallow detail features and deep semantic features, overcoming interference from complex backgrounds and improving the detection recall rate of multi-scale termite targets.

[0021] In step 3, the specific implementation of the multi-model collaborative analysis is as follows: First, the cloud-based system performs ROI cropping on the termite activity areas detected at the edge. Then, SIFT and RANSAC algorithms are used to achieve pixel-level registration of visible light and thermal imaging images. The registered images are input into a lightweight modified MiDaS model to output a high-resolution depth map. By fusing depth maps from multiple perspectives, point cloud data is generated, automatically constructing a three-dimensional model containing structures such as the main termite chamber and termite tunnels. Subsequently, the collected environmental data on temperature, humidity, soil type, and human activity factors are fused to drive the MaxEnt-Gaussian kernel coupling model for termite spatiotemporal diffusion prediction. The core formula is expressed as: P(x, y, t) = G(Δx, Δy, t)·D(θ)dΔxdΔy; where P(x, y, t) represents the termite's time... t The probability of occurrence at spatial coordinates (x, y), G(Δx, Δy, t) is the Gaussian kernel function simulating spatial diffusion, and D(θ) is the environmental dependence matrix dynamically adjusted according to real-time environmental factors, to output a heat map of future temporal and spatial distribution.

[0022] In step 4, the specific operating strategy of the RAGFlow and DeepSeek dual-engine decision system is as follows: Upon receiving reported features including location, ant species, and severity of damage, the RAGFlow engine first retrieves the most relevant similar cases from a pre-built professional knowledge base containing a large number of papers, cases, and procedures based on the problem features, using a formula to calculate cosine similarity. Subsequently, the DeepSeek large language model further integrates real-time 3D nest models, soil parameter monitoring data, weather forecasts, and external data from ecological red lines based on the retrieved cases, performing multi-parameter comprehensive reasoning and decision-making to dynamically generate a detailed prevention and control plan that includes pesticide type, concentration ratio, precise application point map, and physical barrier blocking suggestions.

[0023] The improved target detection model YOLOv5n-CBAM-BiFPN consists of a backbone network, a feature fusion network, and a detection head. The backbone network is used to extract multi-level features of termite targets from complex backgrounds, the feature fusion network is used to perform cross-scale weighted concatenation of features to enhance the representation of small targets, and the detection head is used to output the final termite location and category information.

[0024] The data processing and input / output process of the backbone network is as follows: The preprocessed three-channel termite image matrix is ​​input to the first Focus module for slicing to obtain feature F1; Feature F1 is input to the first Conv module for processing and then to the first C3 module for feature extraction, and then to the first CBAM module for processing to obtain feature F2; Feature F2 is input to the second Conv module for processing and then to the second C3 module, and then to the second CBAM module for recalibration, outputting a first-scale termite feature map P3, which is used to focus on the local texture features of tiny termite targets; The first-scale termite feature map P3 is input to the third Conv module for processing and then to the third C3 module, and then to the third CBAM module for recalibration, outputting a second-scale termite feature map P4; The second-scale termite feature map P4 is input to the fourth Conv module for processing and then to the SPP spatial pyramid pooling module to increase the receptive field, then to the fourth C3 module, and finally to the fourth CBAM module for recalibration, outputting a third-scale termite feature map P5, which is used to extract deep semantic features of termite nests or high-density termite colonies.

[0025] The first to fourth CBAM modules mentioned above adopt the same internal network architecture. The specific processing procedure is as follows: the features output by the previous C3 module are input to the channel attention module. The spatial dimension is compressed by the global max pooling layer and the average pooling layer, respectively. After calculation and merging by the multilayer perceptron, the channel attention weight tensor is output. The channel attention weight tensor is multiplied element-wise with the original input features to obtain the channel refined features, thereby effectively suppressing the invalid channel noise interference caused by soil texture, vegetation occlusion and other factors in complex field environments. Subsequently, the channel refined features are input to the spatial attention module. After pooling and concatenation along the channel dimension, the spatial attention weight tensor is calculated by the spatial convolution layer and output. The spatial attention weight tensor is multiplied element-wise with the channel refined features to output the final recalibrated features, thereby accurately highlighting the geometric outline and physical location of the termite individual in the spatial dimension.

[0026] The aforementioned feature fusion network adopts a BiFPN structure. Its specific processing steps are as follows: The third-scale termite feature map P5 is input to the fifth Conv module for dimensionality reduction, then input to the first Upsample module for upsampling, and fused with the second-scale termite feature map P4 by the first Concat module. This is then processed sequentially by the fifth C3 module and the fifth CBAM module to obtain intermediate feature M4. The intermediate feature M4 is then input to the sixth Conv module for dimensionality reduction, then input to the second Upsample module for upsampling, and fused with the first-scale termite feature map P3 by the second Concat module. This is then processed sequentially by the sixth C3 module and the sixth CBAM module to obtain the fused first-scale output feature O3. This cross-scale skip connection preserves the shallow details of a single termite's tiny target to the maximum extent. The fused first-scale output feature O3 is then input to the seventh Conv module for downsampling, and fused with the intermediate feature M4 by the third Concat module. This is then processed sequentially by the seventh C3 module and the seventh CBAM module to obtain the fused second-scale output feature O4. The fused second-scale output feature O4 is input to the eighth Conv module for downsampling, and then input to the fourth Concat module for fusion with the feature processed by the fifth Conv module. The fused third-scale output feature O5 is obtained by sequentially processing by the eighth C3 module and the eighth CBAM module. Through the above top-down and bottom-up multi-path interaction, the problem of abrupt scale change of termite targets at different physical distances is effectively overcome.

[0027] The specific processing procedure of the above detection head is as follows: the fused first-scale output feature O3, second-scale output feature O4, and third-scale output feature O5 are input to the ninth, tenth, and eleventh Conv modules respectively for channel dimension adjustment; the adjusted feature matrices of the three dimensions are input to the three corresponding branches of the Detect module for single forward inference; the Detect module directly outputs a prediction tensor containing the predicted anchor box bounding box coordinate offset, target confidence, and termite category probability in three dimensions. After filtering by the non-maximum suppression algorithm, it outputs accurate two-dimensional physical coordinates and category information of termites, providing a high-confidence spatial positioning basis for subsequent cloud-based construction of a three-dimensional nest model and generation of precise pesticide application schemes.

[0028] Example: The intelligent termite macro-control and real-time location management platform in this embodiment adopts a three-level collaborative architecture of "end-edge-cloud".

[0029] Data Acquisition End: As the "end" layer, this includes hardware inspection equipment equipped with multimodal sensors. This can include a mobile platform for large-area rapid scanning and visible light image acquisition; and a ground-based mobile platform capable of penetrating complex terrains such as forests and the bottom of dams. This platform is equipped with infrared thermal imaging sensors and various environmental sensors to collect thermal imaging data and microscopic environmental parameters.

[0030] Edge computing layer: As the "edge" layer, it is deployed on front-end hardware devices, and its core is an embedded improved YOLOv5n-CBAM-BiFPN lightweight object detection model. For example... Figure 2 As shown, this model is not a simple concatenation of CBAM and BiFPN, but rather proposes a deeply coupled feature enhancement and multi-scale fusion architecture to address the dual technical challenges of "strong background interference and large target scale differences" in termite micro-target detection.

[0031] Specifically, this invention introduces a CBAM module into the YOLOv5n backbone network. Through a serial dual-path structure of channel attention and spatial attention, adaptive recalibration is performed simultaneously in both the feature channel dimension and the spatial dimension. This significantly improves the network's response gain to small termite targets while effectively suppressing interference from complex backgrounds (such as vegetation and soil texture). However, simple attention recalibration can lead to a shift in the original distribution of feature maps, weakening the efficiency of multi-scale information transmission. To address this, this invention replaces the original PANet feature pyramid with BiFPN, introducing a cross-scale weighted feature fusion mechanism. This enables a non-linear, learnable multi-path interaction between shallow detail features (P3) and deep semantic features (P5), thereby significantly improving the model's robustness in detecting termite targets with significant size differences (from a single worker ant to the entrance of a large nest), especially the recall rate of small targets.

[0032] It should be noted that, as is well known in the art, CBAM compresses some scale variation information when recalibrating feature map channel weights, potentially disrupting the original multi-scale distribution required by BiFPN; and the weighted fusion of BiFPN may amplify residual noise not fully suppressed by CBAM and propagate it to deeper feature layers. To overcome these technical biases, this invention adopts a non-obvious coupling strategy: CBAM is only embedded after the outputs of stages 3, 4, and 5 of the backbone network to avoid over-recalibration of shallow features leading to detail loss; simultaneously, a skip residual path is introduced in the cross-scale connections of BiFPN, ensuring that the features enhanced by CBAM still retain their original scale information for adaptive selection by BiFPN. CBAM and BiFPN thus form a "focus-fusion" sequential collaboration, rather than an independent superposition—CBAM provides BiFPN with cleaner, more responsive multi-scale feature maps, while BiFPN adaptively fuses the cross-scale information enhanced by CBAM. Their synergistic effect overcomes the technical bottleneck of single improvements failing to simultaneously address "anti-background interference" and "multi-scale adaptation." Comparative experiments show that, compared with the conventional concatenated structure of "CBAM first, then BiFPN", the present invention achieves an additional mAP improvement of about 1.2% on the same dataset, realizing an unexpected technical effect.

[0033] Furthermore, the edge computing layer deploys a three-level serial image filtering mechanism to reduce the bandwidth and cloud computing pressure from uploading invalid data: The first level uses the Sobel edge detection operator to quickly calculate image gradient magnitudes, eliminating image frames lacking effective information due to motion blur or defocus. The second level converts the retained images to the HSV color space, where a lightweight Unet+ network performs adaptive noise reduction and contrast enhancement to suppress thermal noise in thermal imaging images and uneven illumination in visible light images. The third level inputs the high-quality images processed in the first two levels into the improved YOLOv5n-CBAM-BiFPN model for accurate termite target detection. This three-level mechanism progressively advances from "coarse screening" to "refinement" and then to "precise recognition," forming a complete closed loop of edge-side intelligent filtering. Finally, the detection results (including target bounding box coordinates and category confidence) and the filtered key image data are uploaded to the cloud decision layer in real time via the 5G network, providing high signal-to-noise ratio input data for subsequent nest 3D reconstruction and diffusion trend prediction.

[0034] Cloud-based decision-making layer: As the "cloud" layer, it is the intelligent hub of the system, integrating multi-source data fusion and intelligent decision-making. It receives data from multiple edge devices and initiates multi-model collaborative analysis.

[0035] Nest 3D reconstruction: such as Figure 5As shown, the cloud first calls the MiDaS depth estimation model. The system performs ROI cropping on the termite activity areas detected by YOLOv5n and uses the SIFT+RANSAC algorithm to achieve pixel-level registration between visible light and thermal imaging images. The registered image is input into the lightweight modified MiDaS model, which outputs a high-resolution depth map. By fusing depth maps from multiple perspectives to generate point cloud data, a three-dimensional model of the nest, including the main ant chamber and ant tunnels, is automatically constructed, providing centimeter-level spatial coordinate information for precise pesticide application.

[0036] Diffusion trend prediction: such as Figure 3 As shown, the system integrates collected environmental data such as temperature, humidity, soil type, and human activity factors to drive the MaxEnt-Gaussian kernel coupling model for termite spatiotemporal dispersal prediction. This model innovatively introduces the time dimension t and the diffusion coefficient D into the traditional MaxEnt model, extending static distribution prediction to dynamic propagation simulation. Its core formula is P(x, y, t) = G(Δx, Δy, t)·D(θ)dΔxdΔy simulates spatial diffusion using a Gaussian kernel function G, while the environmental dependence matrix D(θ) dynamically adjusts the diffusion probability based on real-time environmental factors, thus more accurately predicting the heat map of termite population distribution in future time and space.

[0037] Intelligent prevention and control solution generation: such as Figure 4 As shown, the system integrates a dual-engine decision-making system of RAGFlow and DeepSeek. When a user reports ant infestation characteristics (such as location, ant species, and severity) through the interactive interface, the RAGFlow engine first, based on the problem characteristics Q, retrieves information from a pre-built professional knowledge base K containing a large number of papers, cases, and procedures, using a formula... Retrieve the most relevant similar cases Subsequently, based on the retrieved cases, the DeepSeek big model further integrates real-time monitoring data (such as the newly generated 3D model of the nest and soil parameters) and external data (such as weather forecasts and ecological red lines) to perform multi-parameter comprehensive reasoning and decision-making, and finally generates a detailed prevention and control plan that includes the type of pesticide, concentration ratio, precise application point map, and physical barrier blocking suggestions.

[0038] Data security and evidence storage: All key monitoring data, analysis results, and prevention and control plans will be stored through a "blockchain + IPFS" dual-chain mechanism to ensure that the data is authentic, reliable, and tamper-proof throughout the entire chain from collection to decision-making.

[0039] User interaction is achieved through a web management interface and a mobile mini-program. The web interface, based on Amap JSAPI 2.0, dynamically renders cloud-analyzed anthill 3D heatmaps and diffusion prediction maps on the map, supporting red dot flashing alerts and sandbox simulations. The mobile mini-program allows users to receive alert information and view detection reports anytime, anywhere, and directly trigger the decision engine to obtain professional prevention and control suggestions.

[0040] This invention, through the aforementioned "edge-cloud" collaboration and multi-model fusion technical solution, forms a complete closed loop of "data collection - edge intelligence - cloud fusion - intelligent decision-making - precise operation," fundamentally solving the problems of insufficient data fusion, inaccurate positioning, and imprecise decision-making in existing technologies, and providing a new, efficient, intelligent, and green solution for termite control.

[0041] To demonstrate the detection performance of the improved edge-end YOLOv5n-CBAM-BiFPN lightweight target detection model of this invention, a dedicated termite image dataset was constructed. This dataset contains 1359 images, with visible light and thermal infrared images mixed and labeled according to actual acquisition conditions. The images are sourced from publicly available online resources and termite samples raised by the inventors themselves, covering various lighting conditions, background complexity, and termite density. All images were precisely labeled with bounding boxes by entomologists using the LabelImg tool, and the labeled targets include adult termites, larvae, and nest entrances. The original image resolution was not uniformly cropped to preserve the scale diversity of the real scene. The dataset was randomly divided into a training set (973 images), a validation set (194 images), and a test set (192 images) in an 8:1:1 ratio.

[0042] Table 1 presents the detection results of the original YOLOv5n baseline model, the YOLOv5n model with only CBAM embedded (YOLOv5n-CBAM), the YOLOv5n model with only PANet replaced by BiFPN (YOLOv5n-BiFPN), and the method of this invention (YOLOv5n-CBAM-BiFPN). In terms of detection performance, the model of this invention has the highest precision, mAP@0.5, and recall, reaching 0.80156, 0.70193, and 0.70958, respectively. Compared to the baseline model, the model of this invention improves mAP@0.5 by 5.5% and recall by 8.9%; compared to YOLOv5n-CBAM, mAP@0.5 improves by 4.4% and recall by 6.9%; and compared to YOLOv5n-BiFPN, mAP@0.5 improves by 4.5% and recall by 5.2%. These results fully demonstrate the effectiveness of the deeply coupled architecture proposed in this invention.

[0043] From a feature-level perspective, termite micro-target detection faces two mutually constraining scientific problems: First, background noise (vegetation, soil texture) and target features highly overlap in both channel and spatial dimensions. While simple attention mechanisms can enhance target response, they can lead to a shift in the channel weight distribution of feature maps, compressing scale variation information and weakening the expressive power of multi-scale features. Second, while multi-scale feature fusion (such as BiFPN) can transmit information across layers, it easily amplifies and propagates shallow residual noise to deeper layers, causing localization errors. To address these contradictions, this invention adopts a non-obvious coupling strategy: First, CBAM is embedded only after the outputs of stages 3, 4, and 5 of the backbone network to avoid premature recalibration and loss of shallow details, while preserving the original distribution of deep semantics. Second, skip residual paths are introduced into the cross-scale connections of BiFPN, allowing the features enhanced by CBAM to retain original scale information for adaptive selection by BiFPN. Thus, CBAM and BiFPN form a "focus-fusion" sequential collaboration: CBAM suppresses noise and highlights the target in the channel and spatial dimensions, providing BiFPN with a cleaner, more responsive multi-scale feature map; BiFPN then performs weighted bidirectional fusion on this basis, while the skip residual ensures a direct path for the original information, avoiding the vicious cycle of feature distribution shift and noise amplification. Comparative experiments show that, compared with the conventional "CBAM first, then BiFPN" sequential structure, this invention achieves an additional mAP improvement of approximately 1.2% on the same dataset, realizing an unexpected technical effect.

[0044] The detection results of the model of this invention on a self-built termite dataset are as follows: Figure 7 As shown. Figure 7 The detection results are presented using four models: YOLOv5n, YOLOv5n-CBAM, YOLOv5n-BiFPN, and the method of this invention. From the detection performance, the model of this invention demonstrates more accurate bounding box localization, a significantly lower false negative rate for small targets, and very few false positives. This is attributed to CBAM's feature-focusing ability for small targets, BiFPN's multi-scale weighted fusion mechanism, and the deep coupling and jump residual design between the two. Therefore, the model of this invention performs excellently in the detection of small termite targets, providing reliable detection results and high-quality data support for subsequent nest location and control decisions.

[0045] Table 1 compares the detection performance of different models on a self-built termite dataset:

[0046] Ablation experiments further validated the independent contributions of each module. As shown in Table 1, after embedding CBAM alone, the model's mAP@0.5 improved from 0.66509 to 0.67215 (an improvement of 1.1%), and the recall improved from 0.65168 to 0.66349 (an improvement of 1.8%), indicating that CBAM effectively suppressed background noise through channel and spatial attention. However, the improvement was limited due to feature distribution shift. After replacing BiFPN alone, the model's mAP@0.5 improved to 0.67182 (an improvement of 1.0%), and the recall improved to 0.6745 (an improvement of 3.5%), demonstrating that BiFPN's cross-scale fusion enhanced the recall capability of multi-scale targets. However, the lack of an attention mechanism resulted in some noise being fused, leading to insufficient improvement in accuracy. After coupling CBAM and BiFPN in the specific manner of this invention, mAP@0.5 jumped to 0.70193 and recall increased to 0.70958, which is significantly better than a single module. This verifies the effectiveness of the synergy of "focus-fusion-residual" – CBAM solves noise interference, BiFPN solves scale adaptation, and jump residuals solve information loss. The three complement each other and jointly break through the technical bottleneck that it is difficult for a single improvement to simultaneously achieve "anti-background interference" and "multi-scale adaptation".

[0047] Furthermore, the three-level image filtering mechanism deployed at the edge of this invention achieves an invalid image rejection rate of 62% and compresses the amount of data uploaded to the cloud by 67%, significantly reducing network bandwidth and cloud storage pressure.

[0048] This filtering mechanism and the detection model form a joint optimization: the first two levels of filtering provide high-quality input images for the third level of detection, thereby improving the stability and efficiency of detection. In summary, the above improvements are not a simple stacking of independent technologies, but a substantial advancement through the synergy of CBAM and BiFPN, and the joint optimization of the three-level filtering and detection model, exhibiting outstanding substantive characteristics and significant progress.

Claims

1. A termite intelligent monitoring method based on multimodal perception and edge computing, characterized in that, Includes the following steps: Step 1: Collect data and obtain termite activity images and environmental data of the target area according to the planned path; Step 2: Build and deploy the improved target detection model YOLOv5n-CBAM-BiFPN at the edge to perform real-time target detection on the collected image data, identify termite activity areas, and transmit the detection results and key data to the cloud; Step 3: After receiving data in the cloud, trigger collaborative analysis to construct the three-dimensional structure of termite nests and predict the spatiotemporal spread trend of termites; Step 4: Based on the analysis results of Step 3, conduct comprehensive reasoning to obtain a comprehensive prevention and control plan that includes specific prevention and control measures and operational guidance; Step 5: Push the ant infestation early warning information, 3D heat map and generated prevention and control plan to the user interaction terminal, and schedule the data collection terminal to perform precise application or verification according to the plan instructions.

2. The method according to claim 1, characterized in that, In step 3, after receiving the data, the cloud triggers multi-model collaborative analysis, uses the MiDaS depth estimation model to construct the three-dimensional structure of the termite nest, and uses the MaxEnt-Gaussian kernel coupling model to predict the spatiotemporal spread trend of termites. The specific implementation of the multi-model collaborative analysis is as follows: First, the cloud-based system performs ROI cropping on the termite activity areas detected at the edge. Then, SIFT and RANSAC algorithms are used to achieve pixel-level registration of visible light and thermal imaging images. The registered images are input into a lightweight modified MiDaS model to output a high-resolution depth map. By fusing depth maps from multiple perspectives, point cloud data is generated, automatically constructing a three-dimensional model containing structures such as the main termite chamber and termite tunnels. Subsequently, the collected environmental data on temperature, humidity, soil type, and human activity factors are fused to drive the MaxEnt-Gaussian kernel coupling model for termite spatiotemporal diffusion prediction. The core formula is expressed as: P(x, y, t) = G(Δx, Δy, t)·D(θ)dΔxdΔy; where P(x, y, t) represents the termite's time... t The probability of occurrence at spatial coordinates (x, y), G(Δx, Δy, t) is the Gaussian kernel function simulating spatial diffusion, and D(θ) is the environmental dependence matrix dynamically adjusted according to real-time environmental factors, to output a heat map of future temporal and spatial distribution.

3. The method according to claim 2, characterized in that, In step 4, based on the analysis results of step 3, a comprehensive prevention and control plan containing specific prevention and control measures and operational guidance is automatically generated through integrated reasoning using the RAGFlow and DeepSeek dual-engine decision-making system. The specific operating strategy of the RAGFlow and DeepSeek dual-engine decision-making system is as follows: Upon receiving a report containing location, ant species, and severity of damage, the RAGFlow engine first retrieves the most relevant similar cases from a pre-built professional knowledge base containing a large number of papers, cases, and procedures by calculating cosine similarity using a formula based on the problem characteristics. Subsequently, based on the retrieved cases, the DeepSeek large language model further integrates real-time 3D nest models, soil parameter monitoring data, weather forecasts, and external data from ecological red lines to perform multi-parameter comprehensive reasoning and decision-making, dynamically generating detailed prevention and control plans that include pesticide types, concentration ratios, precise application point maps, and physical barrier blocking suggestions.

4. The method according to claim 1, characterized in that, In step 2, before inputting the data into the improved target detection model, the specific procedure is as follows: preprocessing is performed through a three-level serial image filtering mechanism; The first stage calculates the image gradient magnitude based on the Sobel edge detection operator and removes image frames lacking effective information due to motion blur or defocus. The second stage converts the preserved image to the HSV color space, and uses a lightweight UNet+ network for adaptive noise reduction and contrast enhancement to suppress thermal noise in thermal imaging images and uneven illumination in visible light images. The third stage involves inputting the high-quality image processed in the first two stages into the improved YOLOv5n-CBAM-BiFPN model to perform proficient termite target detection.

5. The method according to claim 1, characterized in that, In step 2, the construction and operation mechanism of the improved target detection model YOLOv5n-CBAM-BiFPN is as follows: After the outputs of stages 3, 4, and 5 of the YOLOv5n backbone network, a CBAM attention mechanism is embedded. This mechanism enhances the model's response to small termite targets through dual-path adaptive recalibration in both channel and spatial dimensions, and avoids the premature loss of shallow detail features. Simultaneously, the original PANet is replaced with a BiFPN weighted bidirectional feature pyramid. Skip residual paths are introduced into the cross-scale connections of BiFPN, allowing the CBAM-enhanced features to retain the original scale information for BiFPN to adaptively select. This forms a non-linear multi-path interaction between shallow detail features and deep semantic features, overcoming interference from complex backgrounds and improving the detection recall rate of multi-scale termite targets.

6. The method according to any one of claims 1 to 5, characterized in that, The improved target detection model YOLOv5n-CBAM-BiFPN consists of a backbone network, a feature fusion network, and a detection head. The backbone network is used to extract multi-level features of termite targets from complex backgrounds, the feature fusion network is used to perform cross-scale weighted concatenation of features to enhance the representation of small targets, and the detection head is used to output the final termite location and category information.

7. The method according to claim 6, characterized in that, The data processing and input / output process of the backbone network is as follows: the preprocessed three-channel termite image matrix is ​​input to the first Focus module for slicing to obtain feature F1; Feature F1 is input to the first Conv module for processing, then input to the first C3 module for feature extraction, and then processed by the first CBAM module to obtain feature F2; Feature F2 is input to the second Conv module for processing, then to the second C3 module, and then recalibrated by the second CBAM module to output a first-scale termite feature map P3, which is used to focus on the local texture features of tiny termite targets. The first-scale termite feature map P3 is input to the third Conv module for processing, then to the third C3 module, and then recalibrated by the third CBAM module to output a second-scale termite feature map P4. The second-scale termite feature map P4 is input to the fourth Conv module for processing, then to the SPP spatial pyramid pooling module to increase the receptive field, then to the fourth C3 module, and finally recalibrated by the fourth CBAM module to output a third-scale termite feature map P5, which is used to extract deep semantic features of termite nests or high-density termite colonies.

8. The method according to claim 7, characterized in that, The first to fourth CBAM modules adopt the same internal network architecture. The specific processing procedure is as follows: the features output by the previous C3 module are input to the channel attention module. The spatial dimension is compressed by the global max pooling layer and the average pooling layer, respectively. After calculation and merging by the multilayer perceptron, the channel attention weight tensor is output. The channel attention weight tensor is multiplied element-wise with the original input features to obtain the channel refined features, thereby effectively suppressing the invalid channel noise interference caused by soil texture, vegetation occlusion and other factors in complex field environments. Subsequently, the channel refined features are input to the spatial attention module. After pooling and concatenation along the channel dimension, the spatial attention weight tensor is calculated by the spatial convolution layer. The spatial attention weight tensor is multiplied element-wise with the channel refined features to output the final recalibrated features, thereby accurately highlighting the geometric outline and physical location of the termite individual in the spatial dimension.

9. The method according to claim 7, characterized in that, The feature fusion network adopts a BiFPN structure, and its specific processing is as follows: The third-scale termite feature map P5 is input to the fifth Conv module for dimensionality reduction, then input to the first Upsample module for upsampling, and then fused with the second-scale termite feature map P4 by the first Concat module. It is then processed by the fifth C3 module and the fifth CBAM module to obtain the intermediate feature M4. The intermediate feature M4 is input to the sixth Conv module for dimensionality reduction, then input to the second Upsample module for upsampling, and then fused with the first-scale termite feature map P3 by the second Concat module. It is then processed by the sixth C3 module and the sixth CBAM module to obtain the fused first-scale output feature O3. This cross-scale skip connection preserves the shallow details of the tiny target of a single termite to the maximum extent. The first-scale output feature O3 after fusion is input to the seventh Conv module for downsampling, and then input to the third Concat module for fusion with the intermediate feature M4. The second-scale output feature O4 after fusion is obtained by processing through the seventh C3 module and the seventh CBAM module in sequence. The fused second-scale output feature O4 is input to the eighth Conv module for downsampling, and then input to the fourth Concat module for fusion with the feature processed by the fifth Conv module. The fused third-scale output feature O5 is obtained by sequentially processing by the eighth C3 module and the eighth CBAM module. Through the above top-down and bottom-up multi-path interaction, the problem of abrupt scale change of termite targets at different physical distances is effectively overcome.

10. The method according to claim 9, characterized in that, The specific processing procedure of the detection head is as follows: the fused first-scale output feature O3, second-scale output feature O4, and third-scale output feature O5 are input to the ninth, tenth, and eleventh Conv modules respectively for channel dimension adjustment; the adjusted feature matrices of the three dimensions are input to the three corresponding branches of the Detect module for single forward inference; the Detect module directly outputs a prediction tensor containing the predicted anchor box bounding box coordinate offset, target confidence, and termite category probability in three dimensions. After filtering by the non-maximum suppression algorithm, it outputs accurate two-dimensional physical coordinates and category information of termites, providing a high-confidence spatial positioning basis for subsequent cloud-based construction of a three-dimensional nest model and generation of precise pesticide application schemes.

Images