Air-ground cooperative termite detection method and system based on multi-modal data fusion

Abstract

The invention discloses an air-ground cooperative termite detection method and system based on multi-modal data fusion. The method comprises the following steps: S1, collecting a first termite detection data set through an air detection platform; s2, performing local detection on the soil and wall hidden area through a ground detection platform to construct a second termite detection data set; s3, performing preliminary preprocessing on the first termite detection data set and the second termite detection data set in a ground data processing system; s4, generating fusion termite detection feature data; s5, outputting the optimized fusion termite detection feature data; s6, generating a preliminary termite detection result; s7, sending the third termite detection data set to a ground data processing system; s8, generating updated fusion termite detection feature data; and S9, inputting the updated fusion termite detection feature data into the termite detection model, and generating a final termite detection result in combination with the preliminary termite detection result. According to the invention, the termite activity detection requirements in a large-range and complex environment can be better met.

Description

Technical Field

[0001] The present invention relates to the technical field of termite detection, and in particular to an air-ground collaborative termite detection method and system based on multimodal data fusion. Background Art

[0002] With the rapid development of intelligent detection technology, high-tech means are gradually introduced into the field of pest control to improve the accuracy and efficiency of pest detection. Termites are highly destructive social insects, and their activities pose hidden dangers to buildings, agriculture, forestry and ecosystems that cannot be ignored. Traditional termite detection methods rely on manual inspections and single equipment detection, which are often limited by small detection range, low accuracy and poor real-time performance. They cannot effectively deal with the characteristics of termites that are highly concealed and spread rapidly, resulting in frequent missed detections and false detections.

[0003] At present, the commonly used termite detection technologies mainly include the following categories:

[0004] The first category is visual observation and simple tool detection that relies on human experience, such as judging the activity area of ​​termites by observing external damage signs on walls or wood. This method is highly dependent on the experience of inspectors, the detection results are easily affected by subjective factors, and it is difficult to cover hidden areas.

[0005] The second category is detection technology based on a single sensor, such as using infrared thermal imaging equipment or vibration sensors for detection. Under specific circumstances, a certain degree of automated detection can be achieved. However, due to the insufficient adaptability of data from a single sensor, it is easily affected by environmental noise and material shielding factors, resulting in reduced accuracy and stability of the detection results.

[0006] The third category is the drone reconnaissance technology developed in recent years, which uses drones equipped with optical imaging and thermal imaging equipment to scan large areas. Although this type of technology has improved the detection range, its data analysis capabilities are limited and it is usually unable to deeply integrate multimodal information in complex environments, resulting in insufficient target recognition capabilities.

[0007] In summary, the existing technologies have significant deficiencies in the coverage, detection accuracy, response speed and data fusion capabilities of termite detection, and cannot meet the actual needs of efficient and accurate detection. Therefore, there is an urgent need for a new method that can integrate multimodal data fusion, air-ground collaborative reconnaissance and intelligent optimization algorithm to solve the above problems. Summary of the invention

[0008] One object of the present invention is to propose an air-ground collaborative termite detection method and system based on multimodal data fusion, which can better meet the needs of termite activity detection in large-scale and complex environments.

[0009] According to an embodiment of the present invention, a method for air-ground collaborative termite detection based on multimodal data fusion includes the following steps:

[0010] S1. Collecting the first termite detection dataset through the aerial detection platform;

[0011] S2. Construct a second termite detection dataset by performing local detection on soil and wall hidden areas through the ground detection platform;

[0012] S3. Preliminary preprocessing of the first termite detection data set and the second termite detection data set in the ground data processing system;

[0013] S4. Performing time synchronization, spatial alignment, and weight allocation on the first termite detection feature data and the second termite detection feature data based on a multimodal data fusion algorithm to generate fused termite detection feature data;

[0014] S5. Optimize the weight distribution of the multimodal data fusion algorithm in combination with the gray wolf optimization algorithm, dynamically adjust the fusion weight of the first termite detection feature data and the second termite detection feature data using the gray wolf optimization algorithm, and output the optimized fused termite detection feature data;

[0015] S6. Inputting the optimized fused termite detection feature data into a termite detection model based on a deep learning algorithm, the termite detection model combines the optimized fused termite detection feature data to identify the activity range and activity intensity of termites in the target area, and generates a preliminary termite detection result;

[0016] S7. Based on the preliminary termite detection results, the aerial detection platform is controlled to perform a second scan of the suspected termite activity area in the target area to construct a third termite detection data set, and the third termite detection data set is sent to the ground data processing system;

[0017] S8. Preprocessing the third termite detection data set, and re-adjusting the weight distribution parameters of the multimodal data fusion algorithm based on the gray wolf optimization algorithm, fusing the third termite detection data set with the feature data of the preliminary termite detection results to generate updated fused termite detection feature data;

[0018] S9. Input the updated fused termite detection feature data into the termite detection model, and generate the final termite detection result in combination with the preliminary termite detection result.

[0019] Optionally, the S1 includes the following steps:

[0020] S11. Control the aerial detection platform equipped with cameras, infrared thermal imaging equipment and multi-spectral sensors to scan the target area R point by point according to the preset scanning path and collect optical image data D opt , infrared thermal imaging data Dir and multispectral image data D ms ;

[0021] S12. Eliminate environmental interference signals from the collected optical image data through a denoising algorithm and extract the image feature matrix F opt ={f opt,i,j}, where f opt,i,j Represents the grayscale or color feature value of the pixel in the i-th row and j-th column in the target area R;

[0022] S13. Temperature calibration and abnormal area marking are performed on the collected infrared thermal imaging data to construct the temperature distribution matrix T ir ={t ir,x,y}, where t ir,x,y Represents the temperature value at position (x, y) in the target area R;

[0023] S14. Generate a multispectral feature vector set S for the collected multispectral image data through band separation and feature extraction algorithm ms ={s ms,k}, where s ms,k represents the reflectivity characteristics of the kth band;

[0024] S15. The processed optical image feature matrix, infrared thermal imaging temperature distribution matrix and multi-spectral feature vector set are combined to construct the first termite detection data set:

[0025] D1={F opt ,T ir ,S ms}.

[0026] Optionally, S2 includes the following steps:

[0027] S21. Control the ground detection platform to deploy vibration sensors, microwave sensors and gas sensors, perform multi-point detection in the soil and wall hidden areas of the target area R according to the preset sampling point set, and collect vibration signal data D vib , microwave echo data D mw And ambient gas composition data D gas ;

[0028] S22. Perform time domain and frequency domain analysis on the collected vibration signal data to extract the vibration feature matrix F vib ;

[0029] S23. Perform waveform processing and echo delay analysis on the collected microwave echo data to construct a microwave feature matrix F mw ;

[0030] S24. Perform gas composition analysis on the collected ambient gas composition data to generate a gas concentration vector set Cgas ={c gas,n}, where c gas,n represents the concentration value of the nth type of termite-related gas in the target area R;

[0031] S25. The processed vibration feature matrix, microwave feature matrix and gas concentration vector set are combined to construct a second termite detection data set:

[0032] D2={F vib ,F mw ,C gas}.

[0033] Optionally, S4 includes the following steps:

[0034] S41. The first termite detection feature dataset after preliminary processing and the second termite detection feature dataset Perform time synchronization processing, use interpolation methods to time align feature data with different sampling frequencies, and generate a time-synchronized termite detection feature dataset

[0035] S43. Time-synchronized termite detection feature dataset Perform spatial alignment processing, and use the spatial interpolation algorithm to spatially match the feature data according to the relative positions of the aerial detection platform and the ground detection platform and the spatial distribution of the target area R, to generate a spatially aligned termite detection feature data set

[0036] S44. Assign initial fusion weights to each type of feature data:

[0037] W={w opt ,w ir ,w ms ,w vib ,w mw ,w gas};

[0038] Among them, w opt ,w ir ,w ms ,w vib ,w mw ,w gas They are the fusion weights of optical image feature data, infrared thermal imaging feature data, multi-spectral feature data, vibration feature data, microwave feature data and gas feature data;

[0039] S45. Spatially aligned termite detection feature dataset based on multimodal data fusion algorithm Perform fusion processing and calculate the fusion feature matrix F fused :

[0040] ″″″″″″

[0041] F fused =w opt ·F opt +w ir ·T ir +w ms ·S ms +w vib ·F vib +w mw ·F mw +w gas ·C gas ;

[0042] Among them, F fused Represents the fused termite detection feature data.

[0043] Optionally, S5 includes the following steps:

[0044] S51. The initial fusion weight set W and the spatially aligned termite detection feature dataset Input the Gray Wolf Optimization Algorithm, and the objective function J(W) is defined as the fusion feature matrix F fused The detection confidence Conf(F fused ) and detection coverage Cov(F fused )’s weighted comprehensive performance:

[0045] J(W)=α·Conf(F fused )+β·Cov(F fused );

[0046] Among them, α and β are adjustment parameters, Conf(F fused ) represents the reliability of the test results, Cov(F fused ) represents the coverage ratio within the target area R;

[0047] S52. Initialize the gray wolf pack, combine the air-ground coordination characteristics, and randomly generate several weight sets W i ={w opt,i ,w ir,i ,w ms,i ,w vib,i ,w mw,i ,w gas,i}, and optimize the initial position of the wolf pack with spatial constraints, so that the data synergy characteristics of the UAV and ground sensors are fully considered;

[0048] S53. Define a collaborative tracking model for the gray wolf pack, where each individual gray wolf represents a set of weights W. i , by dynamically adjusting the partition allocation weight P of the target area weight, forming a joint model of fusion weight and spatial distribution optimization:

[0049]

[0050] Where P weight represents the partition importance weight of the target region R, W α represents the current optimal gray wolf position, and A is the dynamic adjustment factor;

[0051] S54. Every time the gray wolf updates its position, the detection credibility Conf(F fused ) and detection coverage Cov(F fused ) into the optimization process, and combined with the scanning adjustment of the aerial platform and the refined detection supplement of the ground equipment, the objective function J(W) is dynamically optimized;

[0052] S55. Introduce a multimodal weight regularization mechanism to avoid a single modality data dominating, and constrain the weight set to satisfy the following relationship:

[0053]

[0054] where w i Indicates the fusion weight of each feature data;

[0055] S56. Repeat S53 and S54 until the objective function J(W) converges and outputs the optimal fusion weight set W * and regional distribution optimization weights

[0056] S57. Based on the optimal fusion weight set W * and regional distribution optimization weights Recalculate the fusion feature matrix

[0057]

[0058] ″″″

[0059] Among them, F i is the optimized time and space alignment feature matrix set, including F opt 、T ir , S ms 、″″″

[0060] F vib 、F mw , C gas .

[0061] Optionally, the S6 includes the following steps:

[0062] S61. Optimized fusion termite detection feature data Input to the termite detection model based on multi-layer convolutional neural network, the optimized fused termite detection feature data is regarded as an input tensor with spatial coordinates (x, y) and multi-channel feature dimension c, and the input layer feature representation is defined as:

[0063]

[0064] in, It represents the channel value of termite detection data after fusion of multimodal features at coordinate (x, y), including optical image, infrared thermal imaging, multispectral, vibration, microwave and gas information, which is used to characterize the multi-dimensional characteristics of potential termite activities in the target area R;

[0065] S62. Use a multi-layer convolutional neural network to perform feature extraction and pattern recognition on the input layer feature representation. The convolution operation of the lth layer is defined as follows:

[0066]

[0067] in, is the response value of the output feature of the lth layer at the coordinate (x, y) and the qth convolution kernel channel, C l-1 is the number of channels of the previous convolution output, U, V is the spatial radius of the convolution kernel, K l (u,v,c,q) is the parameter of the lth convolution kernel at offset (u,v) and input channel c corresponding to the qth kernel, which is used to extract local feature patterns suitable for termite activity recognition from multimodal features. l,q is the bias of the qth convolution kernel in the lth layer, σ(·) is a nonlinear activation function, which maps the potential termite activity morphology in the fusion feature into a separable high-dimensional feature representation;

[0068] S63. Output of the last convolution layer The prediction layer mapped to the termite activity range and activity intensity is introduced above, and the weighted sum of the features of each coordinate (x, y) is performed and the preliminary termite detection result set R is obtained through a specific mapping function. init :

[0069]

[0070] in, Indicates the probability or signal strength of termite activity at the coordinate (x, y), which is used to preliminarily identify whether there is termite activity in the area. Represents the potential intensity value of termite activity at the coordinate (x, y), which is used to evaluate the degree of damage caused by termites at that location. act,q and W int,qTo predict the layer mapping parameters, the high-dimensional convolutional features are transformed into indicators closely related to termite activity. φ(·) is a mapping function that transforms the accumulated feature values ​​into measurable activity probability or intensity scale;

[0071] S64. The probability or signal strength of termite activity corresponding to the coordinate (x, y) and the potential intensity value of termite activity are combined to form a preliminary termite detection result set:

[0072]

[0073] Among them, R init It represents the spatial distribution and intensity information obtained by preliminary identification of termite activities in the target area R.

[0074] Optionally, the S8 includes the following steps:

[0075] S81. Preprocessing the third termite detection data set, performing denoising, feature extraction and spatial segmentation on the optical image features, infrared thermal imaging features, multispectral features, vibration features, microwave features and gas features, respectively, to generate a preprocessed third termite detection feature data set;

[0076] S82. Temporally synchronize and spatially align the preprocessed third termite detection feature data set with the feature matrix of the preliminary termite detection result to generate a joint feature matrix for unified analysis of the performance of multimodal features in the target area;

[0077] S83. Input the joint feature matrix into the gray wolf optimization algorithm, dynamically adjust the weight distribution parameter set of the multimodal data fusion algorithm, optimize the detection credibility and coverage of the fusion feature matrix, and balance the stability of the weight distribution;

[0078] S84. Iteratively update the weight set through the gray wolf optimization algorithm, optimize the fusion eigenvalue of the joint feature matrix in each iteration, and finally output the optimal weight set;

[0079] S85. Based on the optimal weight set, the joint feature matrix is ​​fused to generate an updated fused termite detection feature matrix.

[0080] Optionally, the S9 includes the following steps:

[0081] S91. Update the fusion termite detection feature matrix Input the termite detection model and combine it with the preliminary termite detection results R init As a multidimensional input data set of the model, the input data is defined as I final ;

[0082] S92. Input data I finalThe multi-layer termite detection model is passed in for in-depth analysis. The multi-layer network in the model is used to extract and nonlinearly map the features layer by layer, construct the final termite activity distribution probability matrix and activity intensity matrix, and define the probability distribution P(x, y) and intensity value S(x, y) of the output layer.

[0083] S93. Calculate the termite activity risk assessment index R of the target area R based on the output probability distribution matrix P(x, y) and activity intensity matrix S(x, y) risk The formula is:

[0084]

[0085] S94. Output the final termite detection result R final , including the termite activity distribution matrix P(x,y), activity intensity matrix S(x,y) and risk assessment index R in the target area risk .

[0086] An air-ground collaborative termite detection system based on multimodal data fusion is used to execute an air-ground collaborative termite detection method based on multimodal data fusion, and includes the following modules:

[0087] The aerial detection module completes aerial scanning of the target area through the optical imaging sensor, infrared thermal imaging equipment and multi-spectral imaging equipment carried by the UAV platform, collects the first termite detection data set, and transmits the data to the ground data processing module;

[0088] The ground detection module detects the soil and hidden areas of the wall in the target area through the vibration sensors, microwave sensors and gas sensors deployed in the target area, collects the second termite detection data set, and transmits the data to the ground data processing module;

[0089] The data processing module is used to preliminarily preprocess the first termite detection data set and the second termite detection data set sent by the aerial detection module and the ground detection module, including denoising, feature extraction and target area division, and to fuse the preprocessed data sets through a multimodal data fusion algorithm to achieve time synchronization, spatial alignment and fusion weight allocation, generate a fusion feature matrix, and dynamically optimize the fusion weight parameters in combination with the gray wolf optimization algorithm;

[0090] Termite detection model module: The termite detection model based on multi-layer convolutional neural network accepts the optimized fusion feature matrix as input, extracts features and performs pattern recognition on the input data through the multi-layer network, and generates a termite activity probability distribution matrix and an activity intensity matrix, which are used to identify the activity range and activity intensity of termites in the target area;

[0091] The risk assessment module comprehensively analyzes termite activities in the target area based on the activity probability distribution matrix and activity intensity matrix generated by the termite detection model, and calculates the risk assessment indicators of termite activities, including activity area distribution, intensity range and overall risk level;

[0092] The result output module is used to integrate the preliminary detection results of the termite detection model with the final analysis results of the risk assessment module to generate a termite detection report for the target area, including a termite activity heat map, activity intensity distribution, and risk assessment information;

[0093] System control module, used for task allocation and equipment control of air detection module and ground detection module, and adjusting detection strategy through real-time feedback;

[0094] The data storage and historical analysis module is used to store the multimodal data collected during the termite detection process and the generated detection results.

[0095] The beneficial effects of the present invention are:

[0096] (1) The present invention collects multimodal data through the joint use of aerial detection platforms and ground detection platforms, and uses the Gray Wolf Optimization Algorithm to dynamically adjust the weight distribution of multimodal features, thereby optimizing the adaptability and accuracy of data fusion, and can effectively utilize the complementarity of different modal data, enhance the ability to identify hidden activity areas of termites, and perform excellently in the detection of complex environments.

[0097] (2) The present invention combines the wide-area scanning capability of the UAV platform with the local fine detection capability of the ground equipment through an air-ground collaborative detection system, and designs a multi-layer data processing flow to perform rapid preliminary scanning of the target area, precise analysis of key areas, and secondary data collection and supplementation. Thanks to the optimization of the air-ground collaborative task allocation by the Gray Wolf Optimization Algorithm, the detection efficiency is significantly improved and the time period from scanning to detection completion is shortened.

[0098] (3) The present invention adopts a method that combines the Grey Wolf Optimization Algorithm with the multimodal data fusion algorithm. By defining a comprehensive performance objective function, the distribution of data fusion weights is iteratively optimized. At the same time, a weight regularization mechanism is introduced to balance the contribution of each modal data to the detection results. By optimizing the weight distribution and dynamically adjusting the fusion strategy, the robustness and stability of the detection results are significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0100] Figure 1This is a flow chart of an air-ground collaborative termite detection method and system based on multimodal data fusion proposed by the present invention. DETAILED DESCRIPTION

[0101] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0102] refer to Figure 1 , an air-ground collaborative termite detection method based on multimodal data fusion, comprising the following steps:

[0103] S1. Collecting the first termite detection dataset through the aerial detection platform;

[0104] S2. Construct a second termite detection dataset by performing local detection on soil and wall hidden areas through the ground detection platform;

[0105] S3. Preliminary preprocessing of the first termite detection data set and the second termite detection data set in the ground data processing system;

[0106] S4. Performing time synchronization, spatial alignment, and weight allocation on the first termite detection feature data and the second termite detection feature data based on a multimodal data fusion algorithm to generate fused termite detection feature data;

[0107] S5. Optimize the weight distribution of the multimodal data fusion algorithm in combination with the gray wolf optimization algorithm, dynamically adjust the fusion weight of the first termite detection feature data and the second termite detection feature data using the gray wolf optimization algorithm, and output the optimized fused termite detection feature data;

[0108] S6. Inputting the optimized fused termite detection feature data into a termite detection model based on a deep learning algorithm, the termite detection model combines the optimized fused termite detection feature data to identify the activity range and activity intensity of termites in the target area, and generates a preliminary termite detection result;

[0109] S7. Based on the preliminary termite detection results, the aerial detection platform is controlled to perform a second scan of the suspected termite activity area in the target area to construct a third termite detection data set, and the third termite detection data set is sent to the ground data processing system;

[0110] S8. Preprocessing the third termite detection data set, and re-adjusting the weight distribution parameters of the multimodal data fusion algorithm based on the gray wolf optimization algorithm, fusing the third termite detection data set with the feature data of the preliminary termite detection results to generate updated fused termite detection feature data;

[0111] S9. Input the updated fused termite detection feature data into the termite detection model, and generate the final termite detection result in combination with the preliminary termite detection result.

[0112] In this implementation, S1 includes the following steps:

[0113] S11. Control the aerial detection platform equipped with cameras, infrared thermal imaging equipment and multi-spectral sensors to scan the target area R point by point according to the preset scanning path and collect optical image data D opt , infrared thermal imaging data D ir and multispectral image data D ms ;

[0114] S12. Eliminate environmental interference signals from the collected optical image data through a denoising algorithm and extract the image feature matrix F opt ={f opt,i,j}, where f opt,i,j Represents the grayscale or color feature value of the pixel in the i-th row and j-th column in the target area R;

[0115] S13. Temperature calibration and abnormal area marking are performed on the collected infrared thermal imaging data to construct the temperature distribution matrix T ir ={t ir,x,y}, where t ir,x,y Represents the temperature value at position (x, y) in the target area R;

[0116] S14. Generate a multispectral feature vector set S for the collected multispectral image data through band separation and feature extraction algorithm ms ={s ms,k}, where s ms,k Represents the reflectivity characteristics of the kth band;

[0117] S15. The processed optical image feature matrix, infrared thermal imaging temperature distribution matrix and multi-spectral feature vector set are combined to construct the first termite detection data set:

[0118] D1={F opt ,T ir ,S ms}.

[0119] In this implementation, S2 includes the following steps:

[0120] S21. Control the ground detection platform to deploy vibration sensors, microwave sensors and gas sensors, perform multi-point detection in the soil and wall hidden areas of the target area R according to the preset sampling point set, and collect vibration signal data D vib , microwave echo data D mw And ambient gas composition data D gas ;

[0121] S22. Perform time domain and frequency domain analysis on the collected vibration signal data to extract the vibration feature matrix F vib ;

[0122] S23. Perform waveform processing and echo delay analysis on the collected microwave echo data to construct a microwave feature matrix F mw ;

[0123] S24. Perform gas composition analysis on the collected ambient gas composition data to generate a gas concentration vector set C gas ={c gas,n}, where c gas,n represents the concentration value of the nth type of termite-related gas in the target area R;

[0124] S25. The processed vibration feature matrix, microwave feature matrix and gas concentration vector set are combined to construct a second termite detection data set:

[0125] D2={F vib ,F mw ,C gas}.

[0126] In this implementation, S4 includes the following steps:

[0127] S41. The first termite detection feature dataset after preliminary processing and the second termite detection feature dataset Perform time synchronization processing, use interpolation methods to time align feature data with different sampling frequencies, and generate a time-synchronized termite detection feature dataset

[0128] S43. Time-synchronized termite detection feature dataset Perform spatial alignment processing, and use the spatial interpolation algorithm to spatially match the feature data according to the relative positions of the aerial detection platform and the ground detection platform and the spatial distribution of the target area R, to generate a spatially aligned termite detection feature data set

[0129] S44. Assign initial fusion weights to each type of feature data:

[0130] W = {w opt ,w ir ,w ms ,w vib ,w mw ,w gas};

[0131] Among them, w opt ,w ir ,w ms ,wvib ,w mw ,w gas They are the fusion weights of optical image feature data, infrared thermal imaging feature data, multi-spectral feature data, vibration feature data, microwave feature data and gas feature data;

[0132] S45. Spatially aligned termite detection feature dataset based on multimodal data fusion algorithm Perform fusion processing and calculate the fusion feature matrix F fused :

[0133] ″″″″″″

[0134] F fused =w opt ·F opt +w ir ·T ir +w ms ·S ms +w vib ·F vib +w mw ·F mw +w gas ·C gas ;

[0135] Among them, F fused Represents the fused termite detection feature data.

[0136] In this implementation, S5 includes the following steps:

[0137] S51. The initial fusion weight set W and the spatially aligned termite detection feature dataset Input the Gray Wolf Optimization Algorithm, and the objective function J(W) is defined as the fusion feature matrix F fused The detection confidence Conf(F fused ) and detection coverage Cov(F fused )’s weighted comprehensive performance:

[0138] J(W)=α·Conf(F fused )+β·Cov(F fused );

[0139] Among them, α and β are adjustment parameters, Conf(F fused ) represents the reliability of the test results, Cov(F fused ) represents the coverage ratio within the target area R;

[0140] S52. Initialize the gray wolf pack, combine the air-ground coordination characteristics, and randomly generate several weight sets W i ={w opt,i ,w ir,i ,wms,i ,w vib,i ,w mw,i ,w gas,i}, and optimize the initial position of the wolf pack with spatial constraints, so that the data synergy characteristics of the UAV and ground sensors are fully considered;

[0141] S53. Define a collaborative tracking model for the gray wolf pack, where each individual gray wolf represents a set of weights W. i , by dynamically adjusting the partition allocation weight P of the target area weight , forming a joint model of fusion weight and spatial distribution optimization:

[0142]

[0143] Where P weight represents the partition importance weight of the target region R, W α represents the current optimal gray wolf position, and A is the dynamic adjustment factor;

[0144] S54. Every time the gray wolf updates its position, the detection credibility Conf(F fused ) and detection coverage Cov(F fused ) into the optimization process, and combined with the scanning adjustment of the aerial platform and the refined detection supplement of the ground equipment, the objective function J(W) is dynamically optimized;

[0145] S55. Introduce a multimodal weight regularization mechanism to avoid a single modality data dominating, and constrain the weight set to satisfy the following relationship:

[0146]

[0147] where w i Indicates the fusion weight of each feature data;

[0148] S56. Repeat S53 and S54 until the objective function J(W) converges and outputs the optimal fusion weight set W * and regional distribution optimization weights

[0149] S57. Based on the optimal fusion weight set W * and regional distribution optimization weights Recalculate the fusion feature matrix

[0150]

[0151] ″″″

[0152] Among them, F i is the optimized time and space alignment feature matrix set, including Fopt 、T ir , S ms 、″″″

[0153] F vib 、F mw , C gas .

[0154] In this implementation, S6 includes the following steps:

[0155] S61. Optimized fusion termite detection feature data Input to the termite detection model based on multi-layer convolutional neural network, the optimized fused termite detection feature data is regarded as an input tensor with spatial coordinates (x, y) and multi-channel feature dimension c, and the input layer feature representation is defined as:

[0156]

[0157] in, It represents the channel value of termite detection data after fusion of multimodal features at coordinate (x, y), including optical image, infrared thermal imaging, multispectral, vibration, microwave and gas information, which is used to characterize the multi-dimensional characteristics of potential termite activities in the target area R;

[0158] S62. Use a multi-layer convolutional neural network to perform feature extraction and pattern recognition on the input layer feature representation. The convolution operation of the lth layer is defined as follows:

[0159]

[0160] in, is the response value of the output feature of the lth layer at the coordinate (x, y) and the qth convolution kernel channel, C l-1 is the number of channels of the previous convolution output, U, V is the spatial radius of the convolution kernel, K l (u,v,c,q) is the parameter of the lth convolution kernel at the offset (u,v) and the input channel c corresponding to the qth kernel, which is used to extract local feature patterns suitable for termite activity recognition from multimodal features. l,q is the bias of the qth convolution kernel in the lth layer, σ(·) is a nonlinear activation function, which maps the potential termite activity morphology in the fusion feature into a separable high-dimensional feature representation;

[0161] S63. Output of the last convolution layer The prediction layer mapped to the termite activity range and activity intensity is introduced above, and the weighted sum of the features of each coordinate (x, y) is performed and the preliminary termite detection result set R is obtained through a specific mapping function. init :

[0162]

[0163] in, Indicates the probability or signal strength of termite activity at the coordinate (x, y), which is used to preliminarily identify whether there is termite activity in the area. Represents the potential intensity value of termite activity at the coordinate (x, y), which is used to evaluate the degree of damage caused by termites at that location. act,q and W int,q To predict the layer mapping parameters, the high-dimensional convolutional features are transformed into indicators closely related to termite activity. φ(·) is a mapping function that transforms the accumulated feature values ​​into measurable activity probability or intensity scale;

[0164] S64. The probability or signal strength of termite activity corresponding to the coordinate (x, y) and the potential intensity value of termite activity are combined to form a preliminary termite detection result set:

[0165]

[0166] Among them, R init It represents the spatial distribution and intensity information obtained by preliminary identification of termite activities in the target area R.

[0167] In this implementation, S8 includes the following steps:

[0168] S81. Preprocessing the third termite detection data set, performing denoising, feature extraction and spatial segmentation on the optical image features, infrared thermal imaging features, multispectral features, vibration features, microwave features and gas features, respectively, to generate a preprocessed third termite detection feature data set;

[0169] S82. Temporally synchronize and spatially align the preprocessed third termite detection feature data set with the feature matrix of the preliminary termite detection result to generate a joint feature matrix for unified analysis of the performance of multimodal features in the target area;

[0170] S83. Input the joint feature matrix into the gray wolf optimization algorithm, dynamically adjust the weight distribution parameter set of the multimodal data fusion algorithm, optimize the detection credibility and coverage of the fusion feature matrix, and balance the stability of the weight distribution;

[0171] S84. Iteratively update the weight set through the gray wolf optimization algorithm, optimize the fusion eigenvalue of the joint feature matrix in each iteration, and finally output the optimal weight set;

[0172] S85. Based on the optimal weight set, the joint feature matrix is ​​fused to generate an updated fused termite detection feature matrix.

[0173] In this implementation, S9 includes the following steps:

[0174] S91. Update the fusion termite detection feature matrix Input the termite detection model and combine it with the preliminary termite detection results R init As a multidimensional input data set of the model, the input data is defined as I final ;

[0175] S92. Input data I final The multi-layer termite detection model is passed in for in-depth analysis. The multi-layer network in the model is used to extract and nonlinearly map the features layer by layer, construct the final termite activity distribution probability matrix and activity intensity matrix, and define the probability distribution P(x, y) and intensity value S(x, y) of the output layer.

[0176] S93. Calculate the termite activity risk assessment index R of the target area R based on the output probability distribution matrix P(x, y) and activity intensity matrix S(x, y) risk The formula is:

[0177]

[0178] S94. Output the final termite detection result R final , including the termite activity distribution matrix P(x,y), activity intensity matrix S(x,y) and risk assessment index R in the target area risk .

[0179] An air-ground collaborative termite detection system based on multimodal data fusion is used to execute an air-ground collaborative termite detection method based on multimodal data fusion, and includes the following modules:

[0180] The aerial detection module completes aerial scanning of the target area through the optical imaging sensor, infrared thermal imaging equipment and multi-spectral imaging equipment carried by the UAV platform, collects the first termite detection data set, and transmits the data to the ground data processing module;

[0181] The ground detection module detects the soil and hidden areas of the wall in the target area through the vibration sensors, microwave sensors and gas sensors deployed in the target area, collects the second termite detection data set, and transmits the data to the ground data processing module;

[0182] The data processing module is used to preliminarily preprocess the first termite detection data set and the second termite detection data set sent by the aerial detection module and the ground detection module, including denoising, feature extraction and target area division, and to fuse the preprocessed data sets through a multimodal data fusion algorithm to achieve time synchronization, spatial alignment and fusion weight allocation, generate a fusion feature matrix, and dynamically optimize the fusion weight parameters in combination with the gray wolf optimization algorithm;

[0183] Termite detection model module: The termite detection model based on multi-layer convolutional neural network accepts the optimized fusion feature matrix as input, extracts features and performs pattern recognition on the input data through the multi-layer network, and generates a termite activity probability distribution matrix and an activity intensity matrix, which are used to identify the activity range and activity intensity of termites in the target area;

[0184] The risk assessment module comprehensively analyzes termite activities in the target area based on the activity probability distribution matrix and activity intensity matrix generated by the termite detection model, and calculates the risk assessment indicators of termite activities, including activity area distribution, intensity range and overall risk level;

[0185] The result output module is used to integrate the preliminary detection results of the termite detection model with the final analysis results of the risk assessment module to generate a termite detection report for the target area, including a termite activity heat map, activity intensity distribution, and risk assessment information;

[0186] System control module, used for task allocation and equipment control of air detection module and ground detection module, and adjusting detection strategy through real-time feedback;

[0187] The data storage and historical analysis module is used to store the multimodal data collected during the termite detection process and the generated detection results.

[0188] Embodiment 1:

[0189] In the embodiment, termite activity detection was conducted in a residential area in a southern city. The residential area covers an area of ​​20,000 square meters and has a complex building structure, including 2-3-story single-family houses and underground storage rooms. The detection background is that some residents reported that wood cavities and traces of mud and sand appeared in their homes, but traditional detection methods failed to effectively locate the termite activity area, causing more extensive building damage.

[0190] At the beginning of the inspection, the team first deployed two types of inspection equipment: aerial and ground: drones carrying high-resolution optical cameras, infrared thermal imagers and multi-spectral sensors; vibration sensors, microwave detectors and gas sensors were deployed on the ground to cover key areas of concern such as walls, basements and soil areas.

[0191] The inspection work started at 8 am. The drone first conducted a high-altitude scan of the area. During the scan, the drone found that the thermal imaging data of the northwest corner wall area of ​​a house (number R1) was abnormal, with a temperature fluctuation range of 33°C to 36°C, while the ambient temperature was only 29°C. In addition, multispectral images showed that the reflection characteristics of the area had a large change, and the peak of the spectral reflectivity in the near-infrared band reached 80%, which was much higher than the 40%-50% in other areas. These data were transmitted to the ground data processing center for analysis.

[0192] At the same time, ground equipment deployed vibration sensors around House R1. The sensors recorded a vibration signal amplitude of 15Hz at the northwest corner of the foundation, which was significantly higher than the 5-7Hz in other areas. The microwave detector also detected abnormal reflection signals in the area, with a peak echo intensity of 12dB, while that in ordinary areas was only 5dB. The gas sensor further detected that the methane concentration in the underground air was 10ppm, which was three times the background value.

[0193] After the abnormal data was processed, it was integrated and analyzed through a multimodal data fusion algorithm. The Gray Wolf optimization algorithm dynamically adjusted the fusion weights of thermal imaging data, vibration signals, and gas concentrations to generate a fusion feature matrix. The system identified that the probability of termite activity in the northwest corner of house number R1 was as high as 85%. At the same time, the activity intensity was assessed to be a high-risk level, and the system generated preliminary detection results.

[0194] Based on the preliminary test results, the team further dispatched drones and ground equipment to conduct a second scan of the northwest corner of house number R1. The drone flew at a low altitude (10 meters) to collect high-resolution images and enhance thermal imaging analysis of the area. The ground equipment increased the sampling frequency. The recorded data of the vibration sensor showed that the vibration waveform showed regular periodic changes with a period range of 1.2 seconds, which was consistent with the typical characteristics of termite activity. The echo signal delay of the microwave detector further showed that the termite activity in the area was concentrated between 40 cm and 60 cm underground.

[0195] After data fusion and optimization, the system generated the final detection results, determining that the area of ​​termite activity in the northwest corner wall and foundation of house number R1 was approximately 3.2 square meters, and the activity intensity reached 78%.

[0196] The following is the comparison data between the method of the present invention and the traditional method:

[0197]

[0198]

[0199] The system generated a final test report at 13:00 on September 15, which included:

[0200] Termite activity distribution map: Highlight the high-risk area of ​​house number R1.

[0201] Analysis of termite activity intensity: The peak activity intensity reached 78% and the average intensity was 55%.

[0202] Risk assessment: The northwest corner wall and foundation are assessed as high risk and immediate prevention and control is recommended.

[0203] The test results were sent to the customer and the construction party in real time. The construction team then carried out targeted pesticide treatment in the northwest corner area, successfully curbing the risk of termite spread. The entire detection process took less than 10 hours, which was significantly higher than the efficiency of traditional manual detection. At the same time, the accuracy and coverage of the test results met expectations.

[0204] This embodiment verifies the feasibility and effectiveness of the present invention through application in real scenarios, and has obvious advantages in termite detection in a large-scale hidden environment.

[0205] The present invention jointly collects multimodal data through an aerial detection platform and a ground detection platform, and uses the Gray Wolf Optimization Algorithm to dynamically adjust the weight distribution of multimodal features, thereby optimizing the adaptability and accuracy of data fusion, and can effectively utilize the complementarity of different modal data to enhance the ability to identify hidden activity areas of termites, and performs well in detection in complex environments.

[0206] The present invention combines the wide-area scanning capability of the UAV platform with the local fine detection capability of the ground equipment through an air-ground collaborative detection system, and designs a multi-layer data processing flow to perform rapid preliminary scanning of the target area, precise analysis of key areas, and secondary data collection and supplementation. Thanks to the optimization of the air-ground collaborative task allocation by the Gray Wolf optimization algorithm, the detection efficiency is significantly improved and the time period from scanning to completion of detection is shortened.

[0207] The present invention adopts a method that combines the Grey Wolf optimization algorithm with the multimodal data fusion algorithm. By defining a comprehensive performance objective function, the distribution of data fusion weights is iteratively optimized. At the same time, a weight regularization mechanism is introduced to balance the contribution of each modal data to the detection results. By optimizing the weight distribution and dynamically adjusting the fusion strategy, the robustness and stability of the detection results are significantly improved.

[0208] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for air-ground collaborative termite detection based on multimodal data fusion, characterized in that: The steps include: S1. Collecting the first termite detection dataset through the aerial detection platform; S2. Construct a second termite detection dataset by performing local detection on soil and wall hidden areas through the ground detection platform; S3. Preliminary preprocessing of the first termite detection data set and the second termite detection data set in the ground data processing system; S4. Performing time synchronization, spatial alignment, and weight allocation on the first termite detection feature data and the second termite detection feature data based on a multimodal data fusion algorithm to generate fused termite detection feature data; S5. Optimize the weight distribution of the multimodal data fusion algorithm in combination with the gray wolf optimization algorithm, dynamically adjust the fusion weight of the first termite detection feature data and the second termite detection feature data using the gray wolf optimization algorithm, and output the optimized fused termite detection feature data; S6. Inputting the optimized fused termite detection feature data into a termite detection model based on a deep learning algorithm, the termite detection model combines the optimized fused termite detection feature data to identify the activity range and activity intensity of termites in the target area, and generates a preliminary termite detection result; S7. Based on the preliminary termite detection results, the aerial detection platform is controlled to perform a second scan of the suspected termite activity area in the target area to construct a third termite detection data set, and the third termite detection data set is sent to the ground data processing system; S8. Preprocessing the third termite detection data set, and re-adjusting the weight distribution parameters of the multimodal data fusion algorithm based on the gray wolf optimization algorithm, fusing the third termite detection data set with the feature data of the preliminary termite detection results to generate updated fused termite detection feature data; S9. Input the updated fused termite detection feature data into the termite detection model, and generate the final termite detection result in combination with the preliminary termite detection result.

2. The air-ground collaborative termite detection method based on multimodal data fusion according to claim 1 is characterized in that: The S1 comprises the following steps: S11. Control the aerial detection platform equipped with cameras, infrared thermal imaging equipment and multi-spectral sensors to scan the target area R point by point according to the preset scanning path and collect optical image data D opt , infrared thermal imaging data D ir and multispectral image data D ms ; S12. Eliminate environmental interference signals from the collected optical image data through a denoising algorithm and extract the image feature matrix F opt ={f opt,i,j }, where f opt,i,j Represents the grayscale or color feature value of the pixel in the i-th row and j-th column in the target area R; S13. Temperature calibration and abnormal area marking are performed on the collected infrared thermal imaging data to construct the temperature distribution matrix T ir ={t ir,x,y }, where t ir,x,y Represents the temperature value at position (x, y) in the target area R; S14. Generate a multispectral feature vector set S for the collected multispectral image data through band separation and feature extraction algorithm ms ={s ms,k }, where s ms,k represents the reflectivity characteristics of the kth band; S15. The processed optical image feature matrix, infrared thermal imaging temperature distribution matrix and multi-spectral feature vector set are combined to construct the first termite detection data set: D1={F opt ,T ir ,S ms }。 3. The air-ground collaborative termite detection method based on multimodal data fusion according to claim 1 is characterized in that: The S2 comprises the following steps: S21. Control the ground detection platform to deploy vibration sensors, microwave sensors and gas sensors, perform multi-point detection in the soil and wall hidden areas of the target area R according to the preset sampling point set, and collect vibration signal data D vib , microwave echo data D mw And ambient gas composition data D gas ; S22. Perform time domain and frequency domain analysis on the collected vibration signal data to extract the vibration feature matrix F vib ; S23. Perform waveform processing and echo delay analysis on the collected microwave echo data to construct the microwave feature matrix F mw ; S24. Perform gas composition analysis on the collected ambient gas composition data to generate a gas concentration vector set C gas ={c gas,n }, where c gas,n represents the concentration value of the nth type of termite-related gas in the target area R; S25. The processed vibration feature matrix, microwave feature matrix and gas concentration vector set are combined to construct a second termite detection data set: D2={F vib ,F mw ,C gas }。 4. The air-ground collaborative termite detection method based on multimodal data fusion according to claim 1 is characterized in that: The S4 comprises the following steps: S41. The first termite detection feature dataset after preliminary processing and the second termite detection feature dataset Perform time synchronization processing, use interpolation methods to time align feature data with different sampling frequencies, and generate a time-synchronized termite detection feature dataset S43. Time-synchronized termite detection feature dataset Perform spatial alignment processing, and use the spatial interpolation algorithm to spatially match the feature data according to the relative positions of the aerial detection platform and the ground detection platform and the spatial distribution of the target area R, to generate a spatially aligned termite detection feature data set S44. Assign initial fusion weights to each type of feature data: In={in opt ,In ir ,In ms ,In vib ,In mw ,In gas }; Among them, w opt ,w ir ,w ms ,w vib ,w mw ,w gas They are the fusion weights of optical image feature data, infrared thermal imaging feature data, multi-spectral feature data, vibration feature data, microwave feature data and gas feature data; S45. Spatially aligned termite detection feature dataset based on multimodal data fusion algorithm Perform fusion processing and calculate the fusion feature matrix F fused : ″″″″″″ F fused =w opt ·F opt +w ir ·T ir +w ms ·S ms +w vib ·F vib +w mw ·F mw +w gas ·C gas ; Among them, F fused Represents the fused termite detection feature data.

5. The air-ground collaborative termite detection method based on multimodal data fusion according to claim 1 is characterized in that: The S5 comprises the following steps: S51. The initial fusion weight set W and the spatially aligned termite detection feature dataset Input the Gray Wolf Optimization Algorithm, and the objective function J(W) is defined as the fusion feature matrix F fused The detection confidence Conf(F fused ) and detection coverage Cov(F fused )’s weighted comprehensive performance: J(W)=α·Conf(F fused )+β·Cov(F fused ); Among them, α and β are adjustment parameters, Conf(F fused ) represents the reliability of the test results, Cov(F fused ) represents the coverage ratio within the target area R; S52. Initialize the gray wolf pack, combine the air-ground coordination characteristics, and randomly generate several weight sets W i ={w opt,i ,w ir,i ,w ms,i ,w vib,i ,w mw,i ,w gas,i }, and optimize the initial position of the wolf pack with spatial constraints, so that the data synergy characteristics of the UAV and ground sensors are fully considered; S53. Define a collaborative tracking model for the gray wolf pack, where each individual gray wolf represents a set of weights W. i , by dynamically adjusting the partition allocation weight P of the target area weight , forming a joint model of fusion weight and spatial distribution optimization: Where P weight represents the partition importance weight of the target region R, W α represents the current optimal gray wolf position, and A is the dynamic adjustment factor; S54. Every time the gray wolf updates its position, the detection credibility Conf(F fused ) and detection coverage Cov(F fused ) into the optimization process, and combined with the scanning adjustment of the aerial platform and the refined detection supplement of the ground equipment, the objective function J(W) is dynamically optimized; S55. Introduce a multimodal weight regularization mechanism to avoid a single modality data dominating, and constrain the weight set to satisfy the following relationship: where w i Indicates the fusion weight of each feature data; S56. Repeat S53 and S54 until the objective function J(W) converges and outputs the optimal fusion weight set W * and regional distribution optimization weights S57. Based on the optimal fusion weight set W * and regional distribution optimization weights Recalculate the fusion feature matrix ″″″ Among them, F i is the optimized time and space alignment feature matrix set, including F opt , T ir , S ms 、″″″ F vib 、F mw 、C gas 。 6. The air-ground collaborative termite detection method based on multimodal data fusion according to claim 1 is characterized in that: The S6 comprises the following steps: S61. Optimized fusion termite detection feature data Input to the termite detection model based on multi-layer convolutional neural network, the optimized fused termite detection feature data is regarded as an input tensor with spatial coordinates (x, y) and multi-channel feature dimension c, and the input layer feature representation is defined as: in, It represents the channel value of termite detection data after fusion of multimodal features at coordinate (x, y), including optical image, infrared thermal imaging, multispectral, vibration, microwave and gas information, which is used to characterize the multi-dimensional characteristics of potential termite activities in the target area R; S62. Use a multi-layer convolutional neural network to perform feature extraction and pattern recognition on the input layer feature representation. The convolution operation of the lth layer is defined as follows: in, is the response value of the output feature of the lth layer at the coordinate (x, y) and the qth convolution kernel channel, C l-1 is the number of channels of the previous convolution output, U, V is the spatial radius of the convolution kernel, K l (u,v,c,q) is the parameter of the lth convolution kernel at offset (u,v) and input channel c corresponding to the qth kernel, which is used to extract local feature patterns suitable for termite activity recognition from multimodal features. l,q is the bias of the qth convolution kernel in the lth layer, σ(·) is a nonlinear activation function, which maps the potential termite activity morphology in the fusion feature into a separable high-dimensional feature representation; S63. Output of the last convolution layer The prediction layer mapped to the termite activity range and activity intensity is introduced above, and the weighted sum of the features of each coordinate (x, y) is performed and the preliminary termite detection result set R is obtained through a specific mapping function. init : in, Indicates the probability or signal strength of termite activity at the coordinate (x, y), which is used to preliminarily identify whether there is termite activity in the area. Represents the potential intensity value of termite activity at the coordinate (x, y), which is used to evaluate the degree of damage caused by termites at that location. act,q and W int,q To predict the layer mapping parameters, the high-dimensional convolutional features are transformed into indicators closely related to termite activity. φ(·) is a mapping function that transforms the accumulated feature values ​​into measurable activity probability or intensity scale; S64. The probability or signal strength of termite activity corresponding to the coordinate (x, y) and the potential intensity value of termite activity are combined to form a preliminary termite detection result set: Among them, R init It represents the spatial distribution and intensity information obtained by preliminary identification of termite activities in the target area R.

7. The air-ground collaborative termite detection method based on multimodal data fusion according to claim 1 is characterized in that: The S8 comprises the following steps: S81. Preprocessing the third termite detection data set, performing denoising, feature extraction and spatial segmentation on the optical image features, infrared thermal imaging features, multispectral features, vibration features, microwave features and gas features, respectively, to generate a preprocessed third termite detection feature data set; S82. Temporally synchronize and spatially align the preprocessed third termite detection feature data set with the feature matrix of the preliminary termite detection result to generate a joint feature matrix for unified analysis of the performance of multimodal features in the target area; S83. Input the joint feature matrix into the gray wolf optimization algorithm, dynamically adjust the weight distribution parameter set of the multimodal data fusion algorithm, optimize the detection credibility and coverage of the fusion feature matrix, and balance the stability of the weight distribution; S84. Iteratively update the weight set through the gray wolf optimization algorithm, optimize the fusion eigenvalue of the joint feature matrix in each iteration, and finally output the optimal weight set; S85. Based on the optimal weight set, the joint feature matrix is ​​fused to generate an updated fused termite detection feature matrix.

8. The air-ground collaborative termite detection method based on multimodal data fusion according to claim 1 is characterized in that: The S9 comprises the following steps: S91. Update the fusion termite detection feature matrix Input the termite detection model and combine it with the preliminary termite detection results R init As a multidimensional input data set of the model, the input data is defined as I final ; S92. Input data I final The multi-layer termite detection model is passed in for in-depth analysis. The multi-layer network in the model is used to extract and nonlinearly map the features layer by layer, construct the final termite activity distribution probability matrix and activity intensity matrix, and define the probability distribution P(x, y) and intensity value S(x, y) of the output layer. S93. Calculate the termite activity risk assessment index R of the target area R based on the output probability distribution matrix P(x, y) and activity intensity matrix S(x, y) risk The formula is: S94. Output the final termite detection result R final , including the termite activity distribution matrix P(x,y), activity intensity matrix S(x,y) and risk assessment index R in the target area risk .

9. An air-ground collaborative termite detection system based on multimodal data fusion, used to execute the air-ground collaborative termite detection method based on multimodal data fusion according to any one of claims 1 to 8, characterized in that: Includes the following modules: The aerial detection module completes aerial scanning of the target area through the optical imaging sensor, infrared thermal imaging equipment and multi-spectral imaging equipment carried by the UAV platform, collects the first termite detection data set, and transmits the data to the ground data processing module; The ground detection module detects the soil and hidden areas of the wall in the target area through the vibration sensors, microwave sensors and gas sensors deployed in the target area, collects the second termite detection data set, and transmits the data to the ground data processing module; The data processing module is used to preliminarily preprocess the first termite detection data set and the second termite detection data set sent by the aerial detection module and the ground detection module, including denoising, feature extraction and target area division, and to fuse the preprocessed data sets through a multimodal data fusion algorithm to achieve time synchronization, spatial alignment and fusion weight allocation, generate a fusion feature matrix, and dynamically optimize the fusion weight parameters in combination with the gray wolf optimization algorithm; Termite detection model module: The termite detection model based on multi-layer convolutional neural network accepts the optimized fusion feature matrix as input, extracts features and performs pattern recognition on the input data through the multi-layer network, and generates a termite activity probability distribution matrix and an activity intensity matrix, which are used to identify the activity range and activity intensity of termites in the target area; The risk assessment module comprehensively analyzes termite activities in the target area based on the activity probability distribution matrix and activity intensity matrix generated by the termite detection model, and calculates the risk assessment indicators of termite activities, including activity area distribution, intensity range and overall risk level; The result output module is used to integrate the preliminary detection results of the termite detection model with the final analysis results of the risk assessment module to generate a termite detection report for the target area, including a termite activity heat map, activity intensity distribution, and risk assessment information; System control module, used for task allocation and equipment control of air detection module and ground detection module, and adjusting detection strategy through real-time feedback; The data storage and historical analysis module is used to store the multimodal data collected during the termite detection process and the generated detection results.

Images