Forestry intelligent robot inspection and removal system and method based on multi-mode perception

Abstract

The invention relates to the technical field of forestry inspection, and discloses a forestry intelligent robot inspection and removal system and method based on multi-modal sensing, and the forestry intelligent robot inspection and removal system based on multi-modal sensing comprises a sensing module, a processing module, an execution module, a power supply module and a communication module. The sensing module is used for acquiring multi-modal data of a forest region; the processing module is in communication connection with the sensing module and is used for analyzing and processing the multi-modal data and generating an inspection or removal instruction; the execution module is in communication connection with the processing module and is used for executing corresponding operation according to the inspection or removal instruction; the power supply module provides operation energy for the system. The forestry intelligent robot is provided with the sensing module to obtain multi-modal data of a forest region, so that the problem that complex conditions of the forest region are difficult to deal with in real time due to the fact that traditional forestry inspection and removal mostly adopt manual operation is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of forestry inspection, and in particular to a forestry intelligent robot inspection and eradication system and method based on multimodal perception. Background Art

[0002] Forestry inspections are a crucial measure for safeguarding the health and safety of forest resources. Their core mission encompasses a wide range of tasks, including monitoring the health of forest ecosystems, such as vegetation growth, animal population dynamics, and soil quality, to promptly detect and address issues like pest invasions and vegetation degradation; inspecting forest resource utilization to prevent illegal logging and overharvesting; and preventing and controlling forest fires to mitigate the threat they pose to forest ecosystems.

[0003] Traditional forestry inspections and eradication are mostly done manually. However, due to the low efficiency, high cost and lack of accuracy of manual operations, it is difficult to respond to complex conditions in forest areas in real time. Summary of the Invention

[0004] In response to the shortcomings of the existing technology, the present invention provides a forestry intelligent robot inspection and eradication system and method based on multimodal perception, which solves the problem that traditional forestry inspection and eradication are mostly done manually, which is difficult to respond to complex conditions in forest areas in real time due to low efficiency, high cost and insufficient accuracy of manual operation.

[0005] The present invention provides the following technical solution: a forestry intelligent robot inspection and control system based on multimodal perception, comprising a perception module, a processing module, an execution module, a power supply module and a communication module; The perception module is used to obtain multimodal data of the forest area; The processing module is in communication with the perception module and is used to analyze and process the multimodal data and generate inspection or elimination instructions; The execution module is in communication with the processing module and is configured to execute corresponding operations according to the inspection or eradication instructions; The power supply module provides operating energy for the system; The communication module is used to transmit the inspection or eradication instructions, and can perform multi-robot collaborative operations.

[0006] By adopting the above technical solution, a perception module is set up using a forestry intelligent robot to obtain multimodal data of the forest area, the processing module analyzes and processes the multimodal data and generates inspection or eradication instructions, the execution module performs corresponding operations according to the instructions, the power supply module provides operating energy, and the communication module transmits instructions and can perform multi-robot collaborative operations, thereby realizing the automated process of forest inspection and eradication, thereby improving the traditional forestry inspection and eradication that mostly adopts manual operations. Due to the low efficiency, high cost and lack of accuracy of manual operations, it is difficult to respond to complex conditions in the forest area in real time.

[0007] Preferably, the sensing module includes a meteorological sensor array, a biological monitoring unit and a positioning device; The meteorological sensor array is used to collect meteorological parameters; The biological monitoring unit includes a multispectral imager, a thermal infrared camera and a CO2 concentration sensor, which are used to obtain multispectral images of vegetation, detect heat source distribution and monitor the respiratory state of vegetation respectively; The positioning device is used for centimeter-level positioning and building a three-dimensional map of the forest area.

[0008] Preferably, the processing module includes an edge computing layer, which integrates an embedded AI processor, a data fusion module and a blockchain module; The embedded AI processor deploys a pest and disease identification model, the data fusion module aligns multi-source spatiotemporal data using a spatiotemporal alignment algorithm F=align(T1, T2, S), where T1 and T2 are time series and S is a spatial coordinate, and the blockchain module stores and encrypts operation records.

[0009] Preferably, the execution module includes a six-degree-of-freedom robotic arm, a puncture injection device, an emergency fire extinguishing bomb projector, and a constant temperature and humidity sample container; The robotic arm is connected to the sensor or actuator via a quick-change interface, and is used to switch between the inspection sensor and the removal tool as needed; The injection device controls the puncture depth based on impedance feedback, and the projector is used for fire emergency treatment.

[0010] Preferably, the impedance feedback control algorithm of the injection device is D=k·F+b, wherein D is the puncture depth, F is the real-time trunk resistance, k is the adjustment coefficient, and b is the reference depth.

[0011] Preferably, the power supply module includes a solar power supply unit, which is composed of a solar panel, an energy storage battery and an energy management system; The solar panel is electrically connected to the energy storage battery through a charge controller, and is used to convert light energy into electrical energy and store it in the energy storage battery through the charge controller; The energy storage battery is electrically connected to the sensing module, the processing module, and the execution module, and is used to power the system; The energy management system is respectively connected to the solar panel, energy storage battery and processing module for optimizing the solar panel power output through the maximum power point tracking algorithm and managing the energy distribution of the energy storage battery through the charge and discharge control strategy.

[0012] The forestry intelligent robot inspection and control method based on multimodal perception includes the following steps: S1: The sensing module collects meteorological parameters, vegetation multispectral images, heat source data, and location information to build a multidimensional database of the forest area; S2: The processing module analyzes the data in the database and uses the convolutional neural network model y = f(Wx + b) to identify pests and diseases, where x is the multispectral image data, W is the network weight, b is the bias, f is the activation function, and y is the recognition result. Inspection or control instructions are generated based on the analysis results. S3: Transmit instructions through the communication module to control the execution module to perform sample collection, targeted drug injection or fire extinguishing bomb projection operations. If it is an elimination instruction, trigger the robotic arm to switch to the corresponding execution tool through the quick-change interface.

[0013] Preferably, in step S1, the positioning device combines RTK-GPS and SLAM lidar to generate an inspection path through an adaptive path planning algorithm P=f(S, T, E), where S is the forest stand structure parameter, T is the terrain data, and E is the ecological parameter.

[0014] Preferably, in step S3, the communication module is a LoRa+5G dual-mode ad hoc network communication module, which is used to transmit inspection or eradication instructions and can perform multi-robot collaborative operations; wherein, after the first robot marks the target coordinates, subsequent robots receive the coordinate information through the communication unit and automatically go to perform the eradication operation.

[0015] Preferably, in step S2, the processing module calculates the fire risk index in real time based on the temperature, humidity, evaporation and other data collected by the meteorological sensor array. When the dryness is ≥ level 4, the thermal infrared and visible light dual-channel scanning in step S3 is triggered; if a fire point is detected, the execution module launches a fire extinguishing bomb within 3 minutes and triggers an alarm, and at the same time records the fire risk handling process through the blockchain module.

[0016] The present invention provides a forestry intelligent robot inspection and control system and method based on multimodal perception. Beneficial effects: 1. The present invention sets a perception module in a forestry intelligent robot to obtain multimodal data of the forest area, the processing module analyzes and processes the multimodal data and generates inspection or eradication instructions, the execution module performs corresponding operations according to the instructions, the power supply module provides operating energy, and the communication module transmits instructions and can perform multi-robot collaborative operations, thereby realizing the automation process of forest inspection and eradication, thereby improving the traditional forestry inspection and eradication that mostly adopts manual operation. Due to the low efficiency, high cost and lack of accuracy of manual operation, it is difficult to respond to complex conditions in the forest area in real time.

[0017] 2. The present invention obtains multimodal data such as meteorological parameters and vegetation multispectral images through the meteorological sensor array, biological monitoring unit and positioning device in the perception module, thereby providing more comprehensive and accurate information for forest area data collection, thereby improving the traditional data collection that mostly relies on single manual naked eye observation. Due to the limited observation range and insufficient accuracy, it results in low data accuracy and low early detection rate of pests and diseases.

[0018] 3. The present invention uses a six-degree-of-freedom robotic arm in the execution module combined with a quick-change interface to switch inspection sensors and eradication tools on demand, thereby achieving precise operations, thereby improving the problem that traditional prevention and control methods mostly use blind large-scale operations, and the extensive operation methods cause serious ecological damage. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a schematic diagram of the system architecture of the forestry intelligent robot inspection and eradication system based on multimodal perception proposed in the present invention; Figure 2 This is a schematic diagram of the structural composition of the perception module of the forestry intelligent robot inspection and eradication system based on multimodal perception proposed in the present invention; Figure 3 This is a schematic diagram of the structure of the biological monitoring unit of the forestry intelligent robot inspection and eradication system based on multimodal perception proposed in the present invention; Figure 4 This is a schematic diagram of the structural composition of the processing module of the forestry intelligent robot inspection and eradication system based on multimodal perception proposed in the present invention; Figure 5 This is a schematic diagram of the structural composition of the execution module of the forestry intelligent robot inspection and eradication system based on multimodal perception proposed in the present invention; Figure 6 This is a schematic diagram of the structural composition of the power supply module of the forestry intelligent robot inspection and eradication system based on multimodal perception proposed in the present invention; Figure 7 This is a schematic diagram of the method steps of the forestry intelligent robot inspection and eradication method based on multimodal perception proposed in the present invention. DETAILED DESCRIPTION

[0020] The following will clearly and completely describe the technical solution of the present invention in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0021] Please see the attached Figure 1 -Attached Figure 6 In a first embodiment of the present invention, the present invention provides a forestry intelligent robot inspection and control system based on multimodal perception, comprising a perception module, a processing module, an execution module, a power supply module, and a communication module; the perception module is used to obtain multimodal data of a forest area; The processing module is in communication with the perception module and is used to analyze and process the multimodal data and generate inspection or elimination instructions; The execution module is in communication with the processing module and is used to perform corresponding operations according to the inspection or eradication instructions; The power supply module provides operating energy for the system; The communication module is used to transmit inspection or eradication instructions and enables multi-robot collaborative operations.

[0022] Specifically, by acquiring multimodal data of the forest area through the perception module, the real-time collection of meteorological parameters, vegetation spectra, heat source distribution and spatial coordinates can be realized, providing a data basis for pest and disease identification and fire risk warning; through the spatiotemporal synchronization of multi-source data, the processing module is supported to build a forest ecological model; precise positioning and environmental modeling ensure the accuracy of robot path planning and operation execution, and realize closed-loop management of monitoring, decision-making and execution; the processing module is connected in communication with the perception module, and can receive multimodal data such as meteorology and spectrum in real time; through data fusion to eliminate information bias, differentiated instructions are generated based on spatiotemporal characteristics, and the execution module is driven to operate accurately, achieving an automated closed loop from data collection to decision output; the execution module is connected in communication with the processing module, and can receive inspection or control instructions in real time, and drive the robotic arm, fire bomb projector and other components to perform precise operations based on the instructions, such as responding to the fire point to launch fire bombs within 3 minutes, and the positioning error of diseased plant injection is ≤5cm; the peripheral equipment is dispatched to work together through the multi-machine collaborative mechanism, realizing an efficient closed loop from instruction reception to task execution; the power supply module provides operating energy for the system, which can ensure the perception module , continuous and stable power supply for the processing module and the execution module; a solar energy + lithium battery hybrid power supply solution is adopted, combined with the energy management system to dynamically adjust the power consumption, so that the equipment can operate 7×24 hours without an external power supply in the forest area; the communication module transmits instructions and realizes multi-robot collaborative operation, which can ensure low-latency transmission of inspection and eradication instructions; it supports cluster operation of more than 10 robots, through the first-machine marking and multi-machine response mode, thereby improving the task execution efficiency, and the forest operation coverage is expanded to 8 times that of a single robot, while reducing energy consumption and resource consumption; the forestry intelligent robot is equipped with a perception module to obtain multi-modal data of the forest area, the processing module analyzes and processes the multi-modal data and generates inspection or eradication instructions, the execution module performs corresponding operations according to the instructions, the power supply module provides operating energy, the communication module transmits instructions and can perform multi-robot collaborative operation, thereby realizing the automation process of forest inspection and eradication, thereby improving the traditional forestry inspection and eradication mostly adopts manual operation, which is difficult to respond to complex conditions in the forest area in real time due to low efficiency, high cost and lack of accuracy.

[0023] The perception module includes a meteorological sensor array, a biological monitoring unit, and a positioning device; The meteorological sensor array is used to collect meteorological parameters; The biological monitoring unit includes a multispectral imager, a thermal infrared camera, and a CO2 concentration sensor, which are used to obtain multispectral images of vegetation, detect heat source distribution, and monitor vegetation respiration status, respectively; The positioning device is used for centimeter-level positioning and construction of three-dimensional maps of the forest area.

[0024] Specifically, the meteorological sensor array uses wind speed / direction sensors, temperature and humidity sensors, evaporimeter and soil moisture meter to obtain parameters such as forest temperature and humidity, wind speed, soil moisture content, etc., providing basic data for fire risk index calculation and vegetation growth status analysis. The data acquisition frequency is ≥1 time / minute, and the temperature and humidity accuracy is ±0.5℃ / ±2%RH; the multispectral imager in the biological monitoring unit: covers the 400-1700nm band, obtains vegetation canopy reflectance spectrum data, and identifies chlorophyll content and early leaf color change characteristics of pests and diseases through band combination, thereby improving the recognition accuracy; thermal infrared camera: detects the temperature field distribution in the forest area with an accuracy of ±0.5℃, and uses temperature anomaly point identification for early detection of fire sources and avoidance of animal activity areas. The thermal imaging resolution is ≥6 40×512 pixels; CO2 concentration sensor: monitors changes in near-ground CO2 concentration in forest areas, with a detection range of 0-5000ppm and an accuracy of ±10ppm, assisting in the assessment of vegetation photosynthesis intensity and the status of the ecological carbon cycle; the positioning device can achieve centimeter-level positioning: RTK-GPS combined with a ground-based augmentation system achieves a plane positioning accuracy of ±2cm and an elevation accuracy of ±5cm, meeting the needs of precise marking of diseased trees and recording of operation trajectories. Precise marking of diseased trees, such as positioning of injection points 1.3m from the trunk; 3D map construction: The SLAM lidar scans the environment at a frequency of 10Hz to generate a 3D model of the forest area with a point cloud density of ≥10 points / ㎡, supporting autonomous obstacle avoidance and adaptive path planning for the robot. The obstacle recognition distance for autonomous obstacle avoidance by the robot is ≥20m.

[0025] The processing module includes an edge computing layer that integrates an embedded AI processor, a data fusion module, and a blockchain module; The embedded AI processor deploys a pest and disease identification model. The data fusion module aligns multi-source spatiotemporal data through the spatiotemporal alignment algorithm F=align(T1,T2,S), where T1 and T2 are time series and S is the spatial coordinate. The blockchain module stores and encrypts operation records.

[0026] Specifically, the embedded AI processor deploys a convolutional neural network model y=f(Wx+b) to process multispectral images, extracts leaf texture / color features through a residual network structure, and realizes the classification and identification of more than 20 common forestry pests and diseases, such as pine wood nematodes and gypsy moths; the YOLOv5 model trained based on thermal infrared data detects temperature anomalies and triggers graded warnings in combination with meteorological parameters, such as dryness level; the spatiotemporal alignment algorithm in the data fusion module eliminates sensor timestamp deviations (and spatial coordinate differences) through the formula F=align(T1, T2, S), generates fused data in a unified coordinate system, and supports cross-sensor correlation analysis, such as the spatiotemporal coupling relationship between temperature and humidity and the distribution of pests and diseases; principal component analysis is used to analyze the relationship between temperature and humidity and the distribution of pests and diseases. The PCA algorithm compresses high-dimensional meteorological, biological, and location data into key feature vectors, thereby reducing the amount of data while retaining a large amount of information entropy and improving edge computing efficiency. The blockchain module packages inspection tracks, identification results, eradication operations and other data into blocks, which are encrypted and stored using the SHA-256 hash algorithm to ensure that the data cannot be tampered with. It supports data synchronization between multiple robot nodes to form a blockchain ledger for forestry operations, providing audit and traceability basis for forest resource management departments, such as historical records of pest and disease outbreaks. The edge computing layer processes local data to reduce cloud transmission delays, thereby reducing the response time for command generation. After integrating multi-source data, it generates differentiated operation instructions based on the rule engine, such as injection depth parameters for different tree species / insect conditions.

[0027] The execution module includes a six-degree-of-freedom robotic arm, a puncture injection device, an emergency fire extinguishing bomb projector, and a constant temperature and humidity sample container; The robotic arm is connected to the sensor or actuator through a quick-change interface to switch between inspection sensors and treatment tools as needed; The injection device controls the puncture depth based on impedance feedback, and the projector is used for fire emergency response.

[0028] Specifically, the six-degree-of-freedom robotic arm realizes the automatic replacement of the multispectral imager and the injection device / fire extinguishing bomb launcher through the quick-change interface, that is, the automatic replacement of the inspection mode and the elimination mode, and supports dynamic switching of functions in complex forest scenes; the six-degree-of-freedom motion control can complete fine operations such as injecting medicine at 1.3m from the trunk and fluorescent marking of dead wood, and the load ≥15kg meets the requirements of carrying equipment such as sample collection containers; the puncture injection device is based on the algorithm D=k·F+b, D is the puncture depth, and F is the real-time resistance. The puncture depth is automatically adjusted according to the density of the trunk xylem, the injection pressure is controlled within the range of 0.5-2MPa, and the puncture depth is adjustable within 0-70mm, ensuring uniform diffusion of the agent while reducing the bark damage rate; for pests and diseases such as pine wood nematodes, the precise injection of 3ml avermectin nanocapsules per plant is achieved, thereby improving the utilization rate of the agent and improving the traditional spraying The spread of pesticides; the emergency fire extinguishing bomb launcher has a range of ≥30m and can launch 1-2kg fire extinguishing bombs. When the dryness is ≥ level 4, it triggers thermal infrared / visible light dual-channel scanning and completes the launch within 3 minutes after the fire point is found; while launching, the fire point coordinates are sent to the management platform through the communication module, triggering an audible and visual alarm within a radius of 1km; the constant temperature and humidity sample container maintains an environment of 4℃±1℃ and 60%±5% humidity to store drilled wood chip samples, with a storage time of ≥24 hours to ensure the activity of living samples of pests and diseases during laboratory testing; the sealed cabin is made of food-grade 316L stainless steel to avoid cross-contamination of samples and supports classified storage of up to 20 samples; closed-loop operation from inspection data collection to elimination execution reduces manual intervention; the quick-change interface is combined with impedance feedback control to achieve millimeter-level precision connection of "identification-positioning-disposal".

[0029] The impedance feedback control algorithm of the injection device is D=k·F+b, where D is the puncture depth, F is the real-time trunk resistance, k is the adjustment coefficient, and b is the reference depth.

[0030] Specifically, D: The puncture depth is dynamically adjusted according to the real-time trunk resistance F, the adjustment coefficient k is preset to 0.1-0.3, and is calibrated according to the wood hardness of the tree species. The default reference depth b is 10mm, which can ensure that the agent is injected into the wood; when the robotic arm drives the injection needle to contact the trunk, the pressure sensor collects the resistance F in real time, and the algorithm calculates the target depth through D=k·F+b. The servo motor controls the puncture process to avoid damage to the trunk due to differences in trunk material, such as rotten wood / healthy wood, caused by excessively deep puncture or failure of the agent to reach the target area due to excessive shallow puncture; for pine wood nematodes, which need to be injected into the wood The algorithm reduces the error of puncture depth and enables the avermectin nanocapsules to accurately reach the duct area 5-20mm under the bark, thereby improving the effective utilization rate of the agent; when the resistance F exceeds the threshold, such as encountering a trunk knot, the algorithm automatically reduces the k value, slows down the puncture speed and adjusts the angle, thereby reducing the area of ​​bark damage; more than 20 common tree species such as pine, poplar, camphor, etc. are calibrated in advance for the resistance-depth relationship, and the corresponding k and b parameter libraries are stored. After the robot recognizes the tree species, it automatically calls the matching parameters to achieve adaptive injection of different tree species without manual adjustment.

[0031] The power supply module includes a solar power supply unit, which is composed of solar panels, energy storage batteries and energy management system; The solar panel is electrically connected to the energy storage battery through the charge controller, and is used to convert light energy into electrical energy and store it in the energy storage battery through the charge controller; The energy storage battery is electrically connected to the sensing module, processing module, and execution module to provide power to the system; The energy management system is connected to the solar panels, energy storage batteries and processing modules respectively, and is used to optimize the solar panel power output through the maximum power point tracking algorithm and manage the energy distribution of the energy storage batteries through the charge and discharge control strategy.

[0032] Specifically, the solar panels use single crystal silicon solar panels with a conversion efficiency of ≥22%, at 1000W / m 2The output power is ≥100W under standard light, and the photovoltaic effect converts light energy directly into DC electricity to charge the energy storage battery; the surface is covered with scratch-resistant tempered glass, with weather resistance up to IP67, and can work stably in an environment of -20℃~70℃, adapting to rainy and high ultraviolet scenes in forest areas; the energy storage battery adopts lithium polymer battery with a capacity of ≥200Ah and a nominal voltage of 24V, which stores the electricity converted by the solar panel and provides a stable DC power supply for the sensing module, processing module, and execution module, while controlling the voltage fluctuation ≤±5%; under the conditions of power consumption ≤30W in inspection mode and peak power consumption ≤120W in elimination mode, the system can support continuous operation for ≥72 hours in a fully charged state, meeting the needs of long-term field operations in forest areas; the charging controller is connected in series between the solar panel and the energy storage battery, and converts the unstable DC output of the solar panel into constant voltage DC through PWM (pulse width modulation) technology to avoid overcharging and damaging the battery; the output current / voltage of the solar panel is monitored in real time, and the charging parameters are dynamically adjusted to make the charging Efficiency remains above 90%. The energy management system calculates the maximum power point of the solar panel in real time using the incremental conductance method (INC) and adjusts the operating voltage to 24V. This reduces fluctuations in solar panel output power and improves energy utilization under complex lighting conditions such as cloudy or shaded conditions. The charging phase is divided into three stages: constant current charging at 5A from 0-80% charge, constant voltage charging at 28V from 80-95% charge, and float charging at 27V from 95-100% charge, extending battery life. The discharge phase triggers low-power mode when the battery charge is ≤20%, shutting down non-essential sensors and prioritizing positioning and communication functions to ensure the robot's safe return to the charging point. In forest areas with an average daily sunlight of ≥4 hours, the solar panel generates an average daily power generation of ≥400Wh, meeting the system's daily power consumption requirements and achieving continuous operation with "zero external power supply". The energy management system dynamically allocates power according to the operating mode. For example, in the treatment mode, the robot arm motor is prioritized for powering, while the inspection mode focuses on powering sensors to ensure maximum energy utilization.

[0033] Please see the attached Figure 7 The forestry intelligent robot inspection and control method based on multimodal perception includes the following steps: S1: The sensing module collects meteorological parameters, vegetation multispectral images, heat source data, and location information to build a multidimensional database of the forest area; S2: The processing module analyzes the data in the database and uses the convolutional neural network model y = f(Wx + b) to identify pests and diseases, where x is the multispectral image data, W is the network weight, b is the bias, f is the activation function, and y is the recognition result. Inspection or control instructions are generated based on the analysis results. S3: Transmit instructions through the communication module to control the execution module to perform sample collection, targeted drug injection or fire extinguishing bomb projection operations. If it is an elimination instruction, the robotic arm is triggered to switch to the corresponding execution tool through the quick-change interface.

[0034] Specifically, S1: Multi-dimensional data acquisition and database construction uses meteorological sensor arrays, multispectral imagers, thermal infrared cameras and RTK-GPS+SLAM lidar to realize real-time acquisition of forest environmental parameters, biological characteristics and spatial coordinates. The meteorological sensor array collects temperature, humidity, wind speed, etc. The multispectral imager obtains 400-1700nm vegetation spectrum, the thermal infrared camera monitors the distribution of heat sources, and the RTK-GPS+SLAM lidar is used for centimeter-level positioning; the collected meteorological, spectral, thermal infrared and location data are integrated according to timestamps and geographic coordinates to form a structured data set with time and space labels, supporting historical data tracing and trend analysis, and providing a data basis for subsequent pest and disease identification and operation decision-making; S2: Data intelligent analysis and instruction generation, using the convolutional neural network model y=f(Wx+b) to extract features from multispectral images, such as leaf texture and color anomalies, and through the trained weight matrix W and bias b, to achieve classification and identification of more than 20 kinds of pests and diseases; based on the identification results and meteorological parameters, such as dryness level, generate differentiated operation instructions: Inspection instructions: Mark suspected diseased plants Coordinates, planning the re-inspection path; extermination command: trigger the robotic arm to switch the injection device / fire extinguishing bomb projector, and attach operating parameters such as injection depth and agent dosage; S3: command execution and tool switching, transmit the command through the LoRa+5G communication module, and control the execution module to complete: sample collection: the robotic arm drills wood chips and stores them in a constant temperature and humidity container; targeted injection: the injection device achieves precise puncture at 1.3m of the tree trunk based on the impedance feedback algorithm D=k·F+b; fire hazard treatment: the fire extinguishing bomb projector quickly responds to the fire point, and when receiving the extermination command, the robotic arm Automatic replacement of sensors and execution tools is accomplished through a quick-change interface, supporting seamless integration of inspection and control modes; full-process automated closed-loop: achieving unmanned "monitoring-analysis-disposal" operations and reducing manual intervention; precise prevention and control and ecological protection: improving the accuracy of pest and disease identification, reducing pesticide usage, shortening fire risk response time, and balancing forestry management efficiency and ecological sustainability; data-driven decision-making: a multidimensional database supports long-term tracking of ecological changes in forest areas, providing forestry departments with data support such as disaster warnings and resource assessments, facilitating intelligent management.

[0035] In step S1, the positioning device combines RTK-GPS and SLAM lidar to generate an inspection path through an adaptive path planning algorithm P = f(S, T, E), where S is the forest stand structure parameter, T is the terrain data, and E is the ecological parameter.

[0036] Specifically, through the centimeter-level positioning of RTK-GPS: providing absolute geographic coordinates with a plane accuracy of ±2cm and an elevation accuracy of ±5cm, ensuring the global positioning benchmark of the inspection path, which is suitable for coordinate marking and trajectory recording in large-scale operation areas in forest areas; SLAM lidar environmental modeling: scanning the surrounding environment at a frequency of 10Hz, generating a three-dimensional environmental map with a point cloud density of ≥10 points / ㎡, identifying obstacles such as trees and gullies in real time, and building the environmental model required for local obstacle avoidance; S forest stand structure parameters in the adaptive path planning algorithm: including tree species density, canopy height, canopy density, etc., obtained by fusing multispectral imager and lidar data; T terrain data: three-dimensional terrain grids with a resolution of ≤0.5m generated by SLAM lidar, including slope, slope direction and other information; E ecological parameters: such as endangered species activity areas, water source protection areas, etc., are determined by superposition of CO2 concentration sensors and GIS data; based on the improved AI algorithm, with P=f(S,T,E) as the framework, the path cost function is dynamically optimized to balance the following goals: increase inspection coverage Terrain Adaptation: For mountainous terrain with slopes greater than 30°, the algorithm automatically generates a zigzag inspection route while controlling the robotic arm to maintain balance and prevent the robot from tipping over. In swampy areas, highland paths are prioritized to ensure safe passage for the equipment. Stand Structure Adaptation: Inspection intervals are adjusted based on canopy density, and canopy data is supplemented with the penetrating bands of the multispectral imager to ensure early identification of pests and diseases without blind spots. Ecological Protection Mechanism: When animal nest heat sources, such as areas with temperatures greater than 30°C, are detected by the thermal infrared camera, the algorithm automatically generates a detour with an avoidance distance of 50m or more to minimize interference with wildlife and reduce the ecological disturbance rate. Data Closed-Loop Support: The generated path is synchronously fed back to the processing module and overlaid with historical pest and disease data to form a "prioritized inspection of high-risk areas" strategy. For example, the inspection frequency in areas with high pine wood nematode prevalence is increased to twice a week. Execution Efficiency Optimization: Path planning results drive the optimization of the robotic arm's motion trajectory. For example, during pesticide injection operations, the path connection time between adjacent diseased plants is shortened, improving the efficiency of random path operations.

[0037] In step S3, the communication module is a LoRa+5G dual-mode ad hoc network communication module, which is used to transmit inspection or eradication instructions and can perform multi-robot collaborative operations; among them, after the first robot marks the target coordinates, the subsequent robots receive the coordinate information through the communication unit and automatically go to perform the eradication operation.

[0038] Specifically, LoRa technology features: long-distance transmission, meeting the low-power command transmission needs of large forest areas, with power consumption ≤10mW in sleep mode; strong anti-interference capabilities, using spread spectrum technology (adjustable from SF7 to SF12), reducing packet loss in tree-obstructed environments. 5G technology features: high-bandwidth data backhaul, with a peak rate of ≥100Mbps, supporting real-time transmission of multispectral imagery and thermal infrared video, reducing latency; positioning assistance, combined with 5G base stations, enabling sub-meter collaborative positioning of robot clusters, improving the synchronization accuracy of multi-machine operations; first robot tagging mechanism: Upon discovering pests, diseases, or fire hazards, RTK-GPS obtains centimeter-level coordinates with an accuracy of ±2cm, and broadcasts a tag containing target type, location, and priority information via the LoRa module; the tag information is accompanied by a blockchain timestamp to ensure that task records cannot be tampered with, supporting subsequent operation traceability. Subsequent robot response process: After receiving the LoRa broadcast, the robot analyzes the target coordinates and plans the optimal path based on the Dijkstra algorithm, taking into account factors such as terrain and obstacles. Upon arrival at the target point, the robot uploads on-site multispectral imagery and thermal infrared video data via 5G to the first robot or management platform. After confirming the treatment plan, the injection / firefighting operation is carried out. Cost optimization: Manual inspection costs are reduced while energy consumption per robot is reduced through multi-robot collaboration. A shared data link avoids duplication of work and reduces the consumption of firefighting agents and fire extinguishing ammunition. Real-time command transmission: Treatment commands, such as injection depth parameters, are transmitted in real time via 5G channels, ensuring synchronization of robot arm movements with commands. Inspection commands, such as coordinate rechecks, are transmitted via LoRa, ensuring command reliability in weak signal areas and improving communication success rates. Cluster management capabilities: The management platform monitors the status of all robots, including battery level, location, and operation progress, through the communication module, and dynamically adjusts task allocation. For example, a robot with a battery level below 30% will automatically exit the operation and return to the charging point. The robot also supports breakpoint resuming, automatically reissuing commands that were not executed due to communication interruptions when the signal is restored, thereby improving task completion rates.

[0039] In step S2, the processing module calculates the fire risk index in real time based on the temperature, humidity, evaporation and other data collected by the meteorological sensor array. When the dryness is ≥ level 4, the thermal infrared and visible light dual-channel scanning in step S3 is triggered; if a fire point is detected, the execution module launches a fire extinguishing bomb within 3 minutes and triggers an alarm, while recording the fire risk disposal process through the blockchain module.

[0040] Specifically, this system enables fire risk index calculation and risk grading. Data collection and analysis: The meteorological sensor array collects parameters such as temperature, humidity, and evaporation in real time. The processing module dynamically calculates the fire risk index based on the NFDRS (National Forest Fire Danger Rating System) algorithm and historical meteorological data. Risk grading: Fire risk levels are divided into five levels, with level 1 being the lowest and level 5 being the highest. When the dryness is ≥ level 4, dual-channel scanning of thermal infrared and visible light is triggered. This system enables dual-channel scanning and fire location. Thermal infrared detection: A thermal infrared camera is used to detect the temperature field in the forest area, identifying abnormal heat sources ≥30°C. The detection distance is ≥200 meters, and the smallest fire point that can be identified is 1 meter in diameter. Visible light confirmation: A visible light camera with a resolution of ≥1920×1080 performs secondary verification of abnormal thermal infrared areas, combined with image processing algorithms such as YOLOv5 to confirm the fire status and avoid misjudgment. Precise positioning: The coordinates of the fire point are determined by RTK-GPS and synchronously recorded in the system database. It can achieve emergency disposal and rapid response, fire bomb projection: the fire bomb launcher of the execution module responds to the fire point within 3 minutes, with a range of ≥30 meters, and a single fire bomb covers an area of ​​≥50㎡, ensuring that the initial fire is effectively controlled; linkage alarm: triggers the sound and light alarm device, and at the same time sends the fire point location information to the management platform and surrounding personnel through the communication module; multi-machine collaboration: through the LoRa+5G communication module, dispatches surrounding robots to rush to the scene for support, and realizes multi-machine joint firefighting operations. It can also play a role in blockchain evidence storage and data traceability, disposal process records: the blockchain module encrypts and stores data such as fire risk warning time, fire point coordinates, number of fire bombs used, and disposal personnel operation logs in real time; tamper-proof features: using the SHA-256 hash algorithm and distributed ledger technology to ensure data integrity and traceability, providing a credible basis for subsequent accident investigations and responsibility determination; data sharing: supports access by multiple nodes such as forestry management departments and fire units, improving cross-departmental collaboration efficiency. It can also achieve system synergy effects, early warning and prevention and control: realize the automation of the entire process of fire risk from "monitoring-warning-handling-recording" to reduce fire losses; dynamic response mechanism: automatically adjust the inspection frequency according to the fire risk level, such as increasing it to 1 time / hour at level 4 to strengthen monitoring of high-risk areas; data closed-loop management: fire risk handling data is fed back to the processing module, optimizing the fire risk index calculation model, and continuously improving the accuracy of warnings.

[0041] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. The forestry intelligent robot inspection and control system based on multimodal perception is characterized by: It includes a perception module, a processing module, an execution module, a power supply module and a communication module; The perception module is used to obtain multimodal data of the forest area; The processing module is in communication with the perception module and is used to analyze and process the multimodal data and generate inspection or elimination instructions; The execution module is in communication with the processing module and is configured to execute corresponding operations according to the inspection or eradication instructions; The power supply module provides operating energy for the system; The communication module is used to transmit the inspection or eradication instructions, and can perform multi-robot collaborative operations.

2. The multimodal sensing-based forestry intelligent robot inspection and control system according to claim 1 is characterized in that: The sensing module includes a meteorological sensor array, a biological monitoring unit and a positioning device; The meteorological sensor array is used to collect meteorological parameters; The biological monitoring unit includes a multispectral imager, a thermal infrared camera and a CO2 concentration sensor, which are used to obtain multispectral images of vegetation, detect heat source distribution and monitor the respiratory state of vegetation respectively; The positioning device is used for centimeter-level positioning and building a three-dimensional map of the forest area.

3. The multimodal sensing-based forestry intelligent robot inspection and control system according to claim 1 is characterized in that: The processing module includes an edge computing layer, which integrates an embedded AI processor, a data fusion module and a blockchain module; The embedded AI processor deploys a pest and disease identification model, the data fusion module aligns multi-source spatiotemporal data using a spatiotemporal alignment algorithm F=align(T1, T2, S), where T1 and T2 are time series and S is a spatial coordinate, and the blockchain module stores and encrypts operation records.

4. The multimodal sensing-based forestry intelligent robot inspection and control system according to claim 1 is characterized in that: The execution module includes a six-degree-of-freedom robotic arm, a puncture injection device, an emergency fire extinguishing bomb projector, and a constant temperature and humidity sample container; the robotic arm is connected to the sensor or actuator through a quick-change interface to switch the inspection sensor and the elimination tool as needed; The injection device controls the puncture depth based on impedance feedback, and the projector is used for fire emergency treatment.

5. The multimodal sensing-based forestry intelligent robot inspection and control system according to claim 4 is characterized in that: The impedance feedback control algorithm of the injection device is D=k·F+b, where D is the puncture depth, F is the real-time trunk resistance, k is the adjustment coefficient, and b is the reference depth.

6. The multimodal sensing-based forestry intelligent robot inspection and control system according to claim 1 is characterized in that: The power supply module includes a solar power supply unit, which is composed of a solar panel, an energy storage battery and an energy management system; The solar panel is electrically connected to the energy storage battery through a charge controller, and is used to convert light energy into electrical energy and store it in the energy storage battery through the charge controller; The energy storage battery is electrically connected to the sensing module, the processing module, and the execution module, and is used to power the system; The energy management system is respectively connected to the solar panel, energy storage battery and processing module for optimizing the solar panel power output through the maximum power point tracking algorithm and managing the energy distribution of the energy storage battery through the charge and discharge control strategy.

7. A forestry intelligent robot inspection and eradication method based on multimodal perception, applied to a forestry intelligent robot inspection and eradication system based on multimodal perception according to any one of claims 1 to 6, characterized in that: The following steps are involved: S1: The sensing module collects meteorological parameters, vegetation multispectral images, heat source data, and location information to build a multidimensional database of the forest area; S2: The processing module analyzes the data in the database and uses the convolutional neural network model y = f(Wx + b) to identify pests and diseases, where x is the multispectral image data, W is the network weight, b is the bias, f is the activation function, and y is the recognition result. Inspection or control instructions are generated based on the analysis results. S3: Transmit instructions through the communication module to control the execution module to perform sample collection, targeted drug injection or fire extinguishing bomb projection operations. If it is an elimination instruction, trigger the robotic arm to switch to the corresponding execution tool through the quick-change interface.

8. The forestry intelligent robot inspection and eradication method based on multimodal perception according to claim 7 is characterized in that: In step S1, the positioning device combines RTK-GPS and SLAM lidar to generate an inspection path through an adaptive path planning algorithm P=f(S, T, E), where S is the forest stand structure parameter, T is the terrain data, and E is the ecological parameter.

9. The forestry intelligent robot inspection and eradication method based on multimodal perception according to claim 7 is characterized in that: In step S3, the communication module is a LoRa+5G dual-mode ad hoc network communication module, which is used to transmit inspection or eradication instructions and can perform multi-robot collaborative operations; wherein, after the first robot marks the target coordinates, the subsequent robots receive the coordinate information through the communication unit and automatically go to perform the eradication operation.

10. The forestry intelligent robot inspection and eradication method based on multimodal perception according to claim 7 is characterized in that: In step S2, the processing module calculates the fire risk index in real time based on the temperature, humidity, evaporation and other data collected by the meteorological sensor array. When the dryness is ≥ level 4, the thermal infrared and visible light dual-channel scanning in step S3 is triggered; if a fire point is detected, the execution module launches a fire extinguishing bomb within 3 minutes and triggers an alarm, and at the same time records the fire risk handling process through the blockchain module.

Images