Dam termite nest positioning method based on intelligent multi-mode fusion and detection equipment

Abstract

The invention relates to the technical field of dam termite prevention and control in water conservancy and hydropower engineering, in particular to a dam termite nest positioning method and detection equipment based on intelligent multi-mode fusion, which utilizes technologies such as geological radar, infrared thermal imaging, RTK (Real Time Kinematic) and GPS (Global Positioning System) positioning and is assisted by a machine learning algorithm to carry out data processing. An infrared thermal imaging camera carried by a crawler chassis is used for rapidly scanning the surface of a dam and finding out temperature abnormal points which may be termite activity areas, an underground high-resolution image is further obtained through a carried ground penetrating radar, meanwhile, coordinate information of the areas is collected, a high-precision dam area map is combined, and the termite activity area is obtained. The termite nest position is further accurately confirmed; by means of the method, the accuracy and efficiency of dam termite nest positioning can be remarkably improved, meanwhile, the environmental pollution risk caused by a traditional method is reduced, and the method is beneficial to protection of safe and stable operation of a water conservancy project and makes a contribution to environmental protection.

Description

Technical Field

[0001] The present invention relates to the technical field of dam termite prevention and control in water conservancy and hydropower projects, and in particular to a dam termite nest positioning method and detection equipment based on intelligent multimodal fusion. Background Art

[0002] Levees and dams are essential components of hydraulic engineering, controlling and managing water resources. However, damage caused by termites to dam structures is a widespread problem, as termites can build nests within them, weakening their stability and safety. To effectively manage and prevent this damage, specialized methods for locating termite nests on dams, detection equipment, and control methods are needed.

[0003] Traditional methods for locating termite nests rely primarily on manual inspection by experienced engineers or pest control experts, a method that is inefficient and inaccurate. With the advancement of technology, a variety of more advanced and effective locating technologies are now being used to detect termite nests, such as geological radar (GPR), infrared thermal imaging, and acoustic detection. GPR, among others, maps underground structures by transmitting high-frequency electromagnetic waves into the ground and receiving the reflected signals. This technology can identify unusual areas that may indicate the location of termite activity.

[0004] Modern detection equipment integrates these advanced technologies, resulting in more compact, portable, and easier operation. For example, some new termite detectors combine GPS and infrared sensors, capable of operating in diverse environments and providing high-resolution imagery, helping technicians pinpoint potential termite nests. Furthermore, drone-based remote sensing systems can scan large areas from the air, rapidly identifying potential risk locations and providing guidance for further ground-based investigations.

[0005] For discovered termite nests, control methods typically involve a combination of physical measures and chemical treatments. Physical measures involve directly destroying the nest structure or using mechanical barriers to prevent termites from entering key areas; chemical treatments rely on spraying pesticides or other toxic substances to eliminate termite colonies. In recent years, biological control techniques have also gained increasing attention. For example, introducing natural enemies or using specific microorganisms to inhibit termite reproduction are environmentally friendly and can reduce environmental impact.

[0006] In summary, it is necessary to adopt scientific and reasonable termite nest location methods, advanced detection equipment and comprehensive control strategies in dam maintenance work. However, there are still some problems with the current methods and equipment, which are mainly reflected in the following aspects:

[0007] 1. Detection depth and accuracy limitations: Single detection technologies, such as geological radar (GPR) or infrared thermal imaging, may not accurately identify deep termite nests due to insufficient detection depth or insufficient resolution. In addition, factors such as soil properties and moisture content can also affect signal propagation, leading to inaccurate detection results.

[0008] 2. Reliance on human experience and skills: Despite the availability of advanced technology and equipment, actual operations still rely on the experience of technicians to interpret data and make decisions. Inadequate training or lack of experience can lead to misjudgments or omissions of important potential risks.

[0009] 3. Costly and time-consuming: Although some advanced detection technologies and equipment have improved positioning efficiency, they are expensive and take a long time in large-scale surveys, increasing the overall maintenance cost.

[0010] Based on this, this case is brought forward. Summary of the Invention

[0011] One of the objectives of the present invention is to provide a detection device for termite nests on dams based on intelligent multimodal fusion, which fundamentally improves the technical level of termite nest detection, achieves efficient surveys, and reduces damage to existing structures.

[0012] In order to achieve the above object, the technical solution of the present invention is as follows:

[0013] A detection device for termite nests on dams based on intelligent multimodal fusion, comprising:

[0014] The crawler detection platform is used to travel on the embankment and is equipped with a geological radar, an infrared thermal imaging camera, and a positioning sensor. The geological radar is used to construct remote sensing images of underground termite nests. The infrared thermal imaging camera is used to detect the heat generated by termite activity in the burrows to form a thermal image. The positioning sensor is used for positioning and navigation.

[0015] The multi-mode data fusion module is used to fuse the data collected by the geological radar and infrared thermal imaging camera through a machine learning algorithm to identify the surface and underground survival characteristics of dam termites and obtain the geological radar nodes and surface infrared nodes related to the dam termite nests;

[0016] The precise positioning module is used to project the geological radar nodes and surface infrared nodes into a unified coordinate system to ensure the consistency of the geological radar data and infrared thermal imaging data in physical space, import the spatially aligned data into the dam area map, and determine the precise location of the termite nest on the dam.

[0017] Furthermore, a path planning module is included for planning the detection path of the crawler detection platform according to the precise location of the termite nest on the dam.

[0018] Furthermore, it includes a cloud platform for receiving data collected by geological radar, infrared thermal imaging camera and positioning sensor, as well as data processed by multi-mode data fusion module and precise positioning module.

[0019] Furthermore, the cloud platform includes an AI monitoring module and / or an AI management module;

[0020] The AI monitoring module is used to analyze data uploaded to the cloud platform, identify the location of termite nests, monitor changes in termite activity in real time, and issue early warning information in a timely manner;

[0021] The AI management module is used to remotely plan and schedule tasks of the tracked detection platform.

[0022] Furthermore, the crawler detection platform includes a power module, a control module, a mounting rod, a triangular crawler mechanism installed at both ends of the mounting rod, a balance wheel mechanism connected to the mounting rod, and several geological radars, infrared thermal imaging cameras and positioning sensors arranged on the rod body of the mounting rod;

[0023] The triangular crawler mechanism includes a crawler, a plurality of sprockets for driving the crawler, and a motor for driving the sprockets;

[0024] The balancing wheel mechanism includes a balancing wheel, a balancing wheel shaft mounted on the balancing wheel and parallel to the mounting rod, and a shock absorber for connecting the balancing wheel shaft and the mounting rod;

[0025] The control module includes a motor control unit and a detection control unit. The motor control unit is used to control the operation of the motor; the detection control unit is used to control the operation of the geological radar, infrared thermal imaging camera and positioning sensor.

[0026] Furthermore, the crawler detection platform is equipped with an independent control computer, which is connected to the upper platform for communication upwards and to the control module for communication downwards;

[0027] The independent control computer includes the multi-mode data fusion module, the precise positioning module and the path planning module.

[0028] Furthermore, the positioning sensor adopts an RTK and GPS integrated positioning sensor.

[0029] A second object of the present invention is to provide a method for locating termite nests on dams based on the above-mentioned detection device, comprising the following steps:

[0030] S1. The tracked detection platform travels along the initially planned route. The platform uses a geological radar to acquire remote sensing images of underground termite nests. An infrared thermal imaging camera detects abnormal ground temperatures and detects the heat generated by termite activity in the termite burrows, generating a thermal image. A positioning sensor acquires the coordinates of the measurement points.

[0031] S2. Using machine learning algorithms, we fuse data collected by geological radar and infrared thermal imaging cameras to identify surface and underground survival characteristics of dam termites and obtain geological radar nodes and surface infrared nodes associated with dam termite nests.

[0032] S3. Project the geological radar nodes and surface infrared nodes into the coordinate system of the dam to ensure the consistency of the geological radar data and infrared thermal imaging data in physical space. Import the spatially aligned data into a high-precision dam area map to determine the precise location of the termite nest on the dam.

[0033] Furthermore, the method includes step S4, planning a detection path of the crawler detection platform according to the precise location of the termite nest on the dam, and performing the detection task.

[0034] Furthermore, the step S3 further includes the following process: uploading the data collected by the geological radar, infrared thermal imaging camera and positioning sensor, the fused and processed data, and the precise location data of the termite nests on the dam to the cloud platform for remote monitoring and management;

[0035] Remote monitoring uses AI to analyze data uploaded to the cloud platform, identify the location of termite nests, monitor changes in termite activity in real time, and issue timely warning information;

[0036] Remote management is the use of AI to remotely plan and schedule the tasks of crawler detection platforms.

[0037] The advantages of the present invention are:

[0038] 1. This solution is a comprehensive dam termite nest detection method that integrates multiple advanced technologies. It aims to precisely locate termite nests within dams by combining physical testing with data analysis. The system utilizes technologies such as geological radar (GPR), infrared thermal imaging, RTK, and GPS positioning, supplemented by machine learning algorithms for data processing. An infrared thermal imaging camera mounted on a tracked chassis rapidly scans the dam surface, identifying temperature anomalies that may indicate termite activity. Furthermore, a ground-penetrating radar (GPR) is used to obtain high-resolution underground images. The coordinate information of the area, combined with a high-precision map of the dam area, further pinpoints the location of termite nests.

[0039] 2. Through intelligent multimodal fusion positioning, combined with the powerful detection capabilities of the tracked mobile detection platform and the integration of deep learning algorithms, the accuracy and efficiency of locating termite nests on embankments can be significantly improved, while also reducing the environmental pollution risks brought by traditional methods. This method not only helps to protect the safe and stable operation of water conservancy projects, but also contributes to environmental protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 2. Schematic diagram of the flow of the method for detecting and locating termite nests on dams in an embodiment;

[0041] Figure 2 is a structural diagram of a crawler detection device in an embodiment;

[0042] Figure 3 Schematic diagram of the working process of the crawler detection device in the embodiment;

[0043] Label Description

[0044] 1. Motor; 2. Track; 3. Sprocket; 4. Control module; 5. RTK, GPS integrated positioning sensor; 6. Geological radar; 7. High-definition camera with integrated infrared sensor; 8. Power module; 9. Shock absorber; 10. Balance wheel axle; 11. Balance wheel; 12. Connecting sleeve. DETAILED DESCRIPTION

[0045] The present invention is further described in detail below in conjunction with the embodiments. It should be understood that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inside", "outside", etc. used herein to indicate directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, they should not be understood as limiting the present invention.

[0046] like Figure 2 As shown, this embodiment proposes a detection device for termite nests on embankments based on intelligent multimodal fusion, including an independent control computer, a power module 8, a control module 4, a mounting rod, a triangular crawler mechanism installed at both ends of the mounting rod, a balance wheel mechanism connected to the mounting rod, and several geological radars 6, infrared thermal imaging cameras 7 and positioning sensors 5 arranged on the rod body of the mounting rod.

[0047] The triangular crawler mechanism includes a crawler 2, several sprockets 3 for driving the crawler 2, and a motor 1 for driving the sprockets 3. The crawler detection device is driven by dual motors. The motor 1 directly drives the sprockets 3, which in turn drive the crawler 2 to enable it to navigate the complex environment of large slopes on dams. The choice of a triangular crawler further enhances the obstacle-crossing capability of the detection device.

[0048] The two-meter-long mounting rod is connected to a triangular crawler mechanism at each end. The rod is equipped with four ground-penetrating radar transmitter terminals (i.e., geological radar 6), a wide-angle, high-definition camera with integrated infrared capabilities (i.e., infrared thermal imaging camera 7), and several RTK5 and GPS integrated positioning sensors 5. These sensors enable the detection of underground termite nests, the collection of surface termite activity data, temperature data at the location of the termite holes, and the collection of dam coordinate information. Specifically, the geological radar (GPR) is used to detect underground structures, capable of penetrating the soil and returning reflected waveforms to construct remote sensing images of underground termite nests. The infrared thermal imaging camera can detect ground temperature anomalies, primarily detecting heat generated by termite activity at the termite holes, forming a thermal image to assist in determining the location of the termite nest. The RTK5 and GPS integrated sensors 5 provide high-precision positioning and automatic navigation capabilities for dam monitoring, combining path planning and data transmission to achieve efficient and accurate inspections.

[0049] The balancing wheel mechanism includes a balancing wheel 11, a balancing wheel axle 10 mounted on the balancing wheel 11 and parallel to the mounting rod, and a shock absorber 9 connecting the balancing wheel axle 10 to the mounting rod. The shock absorber 9, balancing wheel axle 10, and balancing wheel 11 form the complete balancing assembly of the crawler chassis, ensuring stable navigation under challenging road conditions such as steep slopes, large obstacles, and deep pits. The balancing wheel 11, a Mecanum wheel, provides balance and guidance for the crawler chassis, while also supporting the mounting rod and ensuring stable operation of the detection equipment.

[0050] The power module 8 uses two groups of lithium batteries to power the electrical devices on the detection device.

[0051] The control module 4 includes a motor control unit and a detection control unit. The motor control unit is used to control the operation of the motor; the detection control unit is used to control the operation of the geological radar, infrared thermal imaging camera and positioning sensor.

[0052] An independent control computer is also installed on the detection equipment, including a multi-mode data fusion module, a precise positioning module and a path planning module, which can realize edge deployment, process the data detected by geological radar, infrared thermal imaging camera, positioning sensor, etc. in real time, and transmit the data and results back to the cloud platform (i.e. the upper platform) in real time. At the same time, it can issue control commands to the control module to enable the detection equipment to perform detection tasks.

[0053] The multimodal data fusion module adopts a multimodal fusion architecture based on an improved dual-stream graph convolutional network and collaborative training mechanism (TSGC-CT). It drives supervised learning through a historical case database to achieve deep fusion of geological radar and infrared thermal imaging data. Specifically, it includes radar feature stream extraction, infrared feature stream extraction and model collaborative training. Through radar feature stream extraction and infrared feature stream extraction, the surface and underground survival characteristics of dam termites can be identified, and the geological radar nodes and surface infrared nodes related to the dam termite nests can be obtained.

[0054] Radar feature flow: Dynamic graph convolution (DGCNN) is used to capture the topological relationships of underground structures. By considering the dynamic changes and spatial relationships between nodes, DGCNN can more effectively extract features from radar data. These features reflect changes in underground structures such as soil density and moisture, thereby indirectly indicating the possibility of termite activity.

[0055] Infrared feature stream: Combined with the Graph Attention Network (GATv2) to strengthen the correlation of abnormal areas. GATv2 uses the attention mechanism to assign different weights to different nodes, allowing the model to focus on abnormal hotspots in infrared images. These areas are often closely related to termite nest activity.

[0056] Collaborative training mechanism (TSGC-CT): Based on the improvement of the two-stream graph convolutional network, a collaborative training mechanism is introduced. Through the interaction between the radar feature stream and the infrared feature stream, the generalization ability and recognition accuracy of the model are further improved. The collaborative training mechanism allows the model to be continuously iterated and updated during the training process, using unlabeled data to improve performance while reducing the risk of overfitting.

[0057] The precise positioning module projects the geological radar nodes and surface infrared nodes into a unified coordinate system to ensure the consistency of the geological radar data and infrared thermal imaging data in physical space, and imports the spatially aligned data into a high-precision dam area map to determine the precise location of the termite nest on the dam.

[0058] The path planning module plans the detection path of the crawler detection platform according to the precise location of the termite nest on the dam and performs the detection task.

[0059] In addition, the detection equipment can also be supported by cloud platforms. The data collected by the geological radar, infrared thermal imaging camera and positioning sensor, the fused and processed data, and the precise location data of the termite nests on the dam are all synchronized to the cloud platform through high-speed networks such as 5G. The powerful computing resources of the cloud are used for rapid processing, which can realize remote AI monitoring and management. AI monitoring mainly refers to the intelligent analysis of uploaded data, the identification of the location of termite nests, the real-time monitoring of changes in termite activity, the timely issuance of early warning information, and the provision of decision-making support for prevention and control work. AI management mainly refers to the remote planning and scheduling of detection equipment tasks, including inspection routes, inspection times, data collection parameters, etc. According to actual needs, the task plan is dynamically adjusted to ensure the efficiency and pertinence of the inspection work.

[0060] Through the design of the cloud platform, integrated online and offline termite inspection and prevention can be achieved.

[0061] like Figure 1 As shown, the method for locating termite nests on embankments based on the detection device includes the following steps:

[0062] S1. The tracked detection platform travels along the initially planned route. The platform's equipped geological radar acquires remote sensing images of underground termite nests. An infrared thermal imaging camera detects abnormal ground temperatures and detects the heat generated by termite activity in the burrows, generating a thermal image. Positioning sensors acquire the coordinates of each measuring point on the embankment for positioning and navigation.

[0063] S2. Using machine learning algorithms, we fuse data collected by geological radar and infrared thermal imaging cameras to identify surface and underground survival characteristics of dam termites and obtain geological radar nodes and surface infrared nodes associated with dam termite nests.

[0064] S3. Project the geological radar nodes and surface infrared nodes into the embankment's coordinate system to ensure the consistency of the geological radar data and infrared thermal imaging data in physical space. Import the spatially aligned data into a high-precision map of the embankment area to determine the precise location of the termite nest on the embankment.

[0065] During this process, data collected by geological radar, infrared thermal imaging cameras and positioning sensors, fused and processed data, and the precise location data of termite nests on the dam can also be uploaded to the cloud platform for remote monitoring and management.

[0066] S4. Based on the precise location of the termite nest on the dam, plan the detection path of the crawler detection platform and execute the detection tasks (such as marking the location of the termite nest, placing bait and killing packages, etc., or executing other tasks issued by the cloud platform).

[0067] like Figure 3 As shown, the implementation process of the detection equipment is as follows:

[0068] After the system is initialized, it begins to acquire multi-source data from geological radar, infrared thermal imaging cameras, and positioning sensors, processes these data using the multimodal data fusion method in steps S2 and S3, and plans the detection path. After the path planning is completed, a self-check of the assembly components is performed to ensure the reliability of the system hardware and software. If the self-check is normal, the detection task is executed to identify and locate the location of the termite nest on the dam. If the self-check is abnormal, the fault is marked through the log system. Minor abnormalities trigger dynamic compensation, and serious faults (such as hardware damage) initiate the shutdown and maintenance process.

[0069] The above embodiments are only used to illustrate the concept of the present invention, and are not intended to limit the protection of the present invention. Any non-substantial changes to the present invention using this concept should fall within the scope of protection of the present invention.

Claims

1. A detection device for termite nests on embankments based on intelligent multimodal fusion, characterized in that: include: The crawler detection platform is used to travel on the embankment and is equipped with a geological radar, an infrared thermal imaging camera, and a positioning sensor. The geological radar is used to construct remote sensing images of underground termite nests. The infrared thermal imaging camera is used to detect the heat generated by termite activity in the burrows to form a thermal image. The positioning sensor is used for positioning and navigation. The multi-mode data fusion module is used to fuse the data collected by the geological radar and infrared thermal imaging camera through a machine learning algorithm to identify the surface and underground survival characteristics of dam termites and obtain the geological radar nodes and surface infrared nodes related to the dam termite nests; The precise positioning module is used to project the geological radar nodes and surface infrared nodes into a unified coordinate system to ensure the consistency of the geological radar data and infrared thermal imaging data in physical space, import the spatially aligned data into the dam area map, and determine the precise location of the termite nest on the dam.

2. The device for detecting termite nests on dams based on intelligent multimodal fusion according to claim 1, characterized in that: It includes a path planning module for planning the detection path of the crawler detection platform according to the precise location of the termite nest on the dam.

3. The device for detecting termite nests on dams based on intelligent multimodal fusion according to claim 1, characterized in that: It includes a cloud platform for receiving data collected by geological radar, infrared thermal imaging camera and positioning sensor, as well as data processed by multi-mode data fusion module and precise positioning module.

4. The device for detecting termite nests on dams based on intelligent multimodal fusion according to claim 1, characterized in that: The cloud platform includes an AI monitoring module and / or an AI management module; The AI monitoring module is used to analyze data uploaded to the cloud platform, identify the location of termite nests, monitor changes in termite activity in real time, and issue early warning information in a timely manner; The AI management module is used to remotely plan and schedule tasks of the tracked detection platform.

5. The device for detecting termite nests on dams based on intelligent multimodal fusion according to claim 2, characterized in that: The crawler detection platform includes a power module, a control module, a mounting rod, a triangular crawler mechanism installed at both ends of the mounting rod, a balance wheel mechanism connected to the mounting rod, and several geological radars, infrared thermal imaging cameras and positioning sensors arranged on the rod body of the mounting rod; The triangular crawler mechanism includes a crawler, a plurality of sprockets for driving the crawler, and a motor for driving the sprockets; The balancing wheel mechanism includes a balancing wheel, a balancing wheel shaft mounted on the balancing wheel and parallel to the mounting rod, and a shock absorber for connecting the balancing wheel shaft and the mounting rod; The control module includes a motor control unit and a detection control unit. The motor control unit is used to control the operation of the motor; the detection control unit is used to control the operation of the geological radar, infrared thermal imaging camera and positioning sensor.

6. The device for detecting termite nests on dams based on intelligent multimodal fusion according to claim 5, characterized in that: The crawler detection platform is equipped with an independent control computer, which is connected to the upper platform for communication upwards and to the control module for communication downwards; The independent control computer includes the multi-mode data fusion module, the precise positioning module and the path planning module.

7. The device for detecting termite nests on dams based on intelligent multimodal fusion according to claim 1, characterized in that: The positioning sensor adopts an RTK and GPS integrated positioning sensor.

8. A method for locating termite nests on dams based on the detection device according to any one of claims 1 to 7, characterized in that: The following steps are involved: S1. The tracked detection platform travels along the initially planned route. The platform uses a geological radar to acquire remote sensing images of underground termite nests. An infrared thermal imaging camera detects abnormal ground temperatures and detects the heat generated by termite activity in the termite burrows, generating a thermal image. A positioning sensor acquires the coordinates of the measurement points. S2. Using machine learning algorithms, we fuse data collected by geological radar and infrared thermal imaging cameras to identify surface and underground survival characteristics of dam termites and obtain geological radar nodes and surface infrared nodes associated with dam termite nests. S3. Project the geological radar nodes and surface infrared nodes into the coordinate system of the dam to ensure the consistency of the geological radar data and infrared thermal imaging data in physical space. Import the spatially aligned data into a high-precision dam area map to determine the precise location of the termite nest on the dam.

9. A method for locating termite nests on a dam as claimed in claim 8, characterized in that: include: S4. Plan the detection path of the crawler detection platform based on the precise location of the termite nest on the dam and perform the detection task.

10. The method for locating termite nests on a dam as claimed in claim 8, characterized in that: The step S3 also The following processes are included: Upload data collected by geological radar, infrared thermal imaging cameras and positioning sensors, the fused and processed data, and the precise location data of termite nests on the dam to the cloud platform for remote monitoring and management; Remote monitoring uses AI to analyze data uploaded to the cloud platform, identify the location of termite nests, monitor changes in termite activity in real time, and issue timely warning information; Remote management is the use of AI to remotely plan and schedule the tasks of crawler detection platforms.

Images