A fire rescue robot hierarchical control system fusing multi-modal perception and physical evolution prediction

Abstract

The application relates to the technical field of a fire rescue robot hierarchical control system fusing multi-modal perception and physical evolution prediction, and discloses a fire rescue robot hierarchical control system fusing multi-modal perception and physical evolution prediction. The system comprises a multi-modal global perception module, a physical evolution prediction module and a hierarchical collaborative control module. The perception module fuses multi-source data to construct a semantic voxel map; the prediction module performs evolution deduction on fire field thermodynamics, smoke spread and structural stability; and the control module realizes autonomous decision-making and motion control under dynamic constraints through decoupling and collaboration of the strategic, tactical and operation layers. The application solves the problems of perception failure and decision lag under extreme environments, predicts risks through physical laws, and significantly improves the operation safety and rescue efficiency of the robot in a dynamic disaster site.

Description

Technical Field

[0001] This invention belongs to the field of robot control technology, specifically relating to a hierarchical control system for fire rescue robots that integrates multimodal perception and physical evolution prediction. Background Technology

[0002] Specialized robot technology is playing an increasingly significant role in public safety and emergency rescue, especially in extreme scenarios such as high-rise building fires or large-scale industrial accidents. Intelligent robots can effectively replace rescue personnel in high-risk environments to perform reconnaissance and search and rescue tasks. With the deep integration of artificial intelligence, complex control theory, and multi-source sensing technology, modern rescue robots are evolving towards greater autonomy and intelligent perception. As an important component of the emergency management system, the reliability and intelligence level of firefighting robot systems are directly related to improving rescue efficiency and ensuring the safety of life and property of personnel on site.

[0003] Among these, autonomous operating systems integrating multimodal perception, environmental prediction, and hierarchical control are a core research and development focus in the field of fire and rescue robots. These systems integrate various hardware devices such as lidar, thermal imagers, and acoustic sensors to build a deep understanding model of the disaster site environment and utilize a hierarchical control architecture to decompose macroscopic rescue tasks into executable low-level motion sequences. This technological direction aims to enhance the robot's autonomous navigation and decision-making capabilities under conditions of limited communication, complex environments, and extremely high uncertainty by deeply coupling sensory data with the laws of physical evolution.

[0004] However, existing firefighting robot control systems still face severe challenges in dealing with extremely complex scenarios such as high-rise building fires. In fire scenes filled with dense smoke and high temperatures, traditional visual sensors often suffer severe visual degradation due to obstructed optical paths, leading to loss of positioning accuracy and perception failure. Simultaneously, existing control logic is mostly based on static or quasi-static environmental assumptions, lacking the ability to predict the physical evolution trends of the disaster environment, such as building collapse or the instantaneous spread of fire, making it difficult to avoid potential risks in real time in highly dynamic scenarios. Furthermore, the complex constraints and massive amounts of unstructured information accompanying long-distance rescue missions make the decision-making process extremely difficult. Traditional control algorithms often cannot balance the optimality of the overall task with the real-time response of the underlying layers, causing the robot to easily experience decision-making delays or control oscillations during the execution of complex commands. These problems, manifesting in extreme environments, severely restrict the practical effectiveness of firefighting robots and have become urgent technical challenges that need to be addressed. Summary of the Invention

[0005] The purpose of this invention is to provide a hierarchical control system for fire rescue robots that integrates multimodal perception and physical evolution prediction, in order to solve the problems of existing fire rescue robots such as perception failure, lack of dynamic environmental prediction capabilities, and delayed decision response in extreme environments such as dense smoke and high temperature.

[0006] The technical solution of this invention includes: a multimodal global perception module, a physical evolution prediction module, and a hierarchical collaborative control module; the multimodal global perception module is used to collect raw observation data of the disaster site environment in real time and to construct a high-dimensional environmental characterization by utilizing the information complementarity of heterogeneous sensors; the physical evolution prediction module is used to perform physical modeling and spatiotemporal evolution extrapolation of the fire thermodynamic distribution, smoke spread trend, and building structural stability based on the environmental characterization output by the multimodal global perception module; the hierarchical collaborative control module is used to receive the prediction results of the physical evolution prediction module and, through the decoupling and collaboration of the strategic task layer, tactical path layer, and operation execution layer, realize the robot's autonomous decision-making and high-precision motion control under complex dynamic constraints.

[0007] Furthermore, the multimodal global perception module includes a strong interference suppression unit, a multi-source feature alignment unit, and an environmental voxel construction unit. The strong interference suppression unit utilizes the physical properties of infrared thermal imaging sensors penetrating dense smoke, combined with the preliminary detection of obstacle positions by an acoustic array, to perform weighted filtering on the lidar point cloud data, eliminating pseudo-random noise caused by suspended particulate matter. The multi-source feature alignment unit maps the temperature field features of the infrared image, the geometric structure features of the lidar, and the spatial position features of the acoustic sensor to a unified Cartesian coordinate system, ensuring absolute consistency of different modal data in the spatiotemporal dimension through timestamp synchronization technology. The environmental voxel construction unit transforms the aligned multimodal data into a semantic voxel map containing temperature attributes, material attributes, and geometric occupancy attributes.

[0008] In one embodiment of the present invention, the physical evolution prediction module includes a fire thermodynamic modeling unit, a smoke dynamics evolution unit, and a structural risk assessment unit. The fire thermodynamic modeling unit, based on the real-time temperature gradient in the environmental voxel map and combined with the physical laws of heat conduction, heat convection, and heat radiation, constructs a fire energy dissipation model to calculate the spatial distribution changes of the fire temperature field within the next 60 seconds. The smoke dynamics evolution unit uses fluid dynamics analysis methods, combined with information on wind direction, air pressure, and building openings within the fire area, to simulate the diffusion path and concentration evolution of smoke, generating a dynamic prediction map of the visibility degradation area. The structural risk assessment unit, by monitoring the cumulative temperature effect and deformation data of the building's load-bearing components and combining the decay curve of material mechanical properties with temperature changes, predicts the probability of building structure collapse and outputs an obstacle avoidance zone index with a time scale.

[0009] Furthermore, the hierarchical collaborative control module adopts a top-down three-layer architecture: the strategic mission layer is responsible for the decomposition of macro rescue objectives and global resource scheduling; the tactical path layer is responsible for real-time planning of obstacle avoidance paths under the dynamic spatiotemporal constraints generated by the physical evolution prediction module; and the operation execution layer is responsible for converting tactical paths into robot drive joint commands.

[0010] As one embodiment of the present invention, the strategic mission layer constructs a multi-objective optimization model, using search and rescue efficiency, robot energy efficiency, and its own safety as evaluation indicators to generate a rescue sequence; when the structural collapse probability output by the physical evolution prediction module exceeds a preset safety threshold of 15%, the strategic mission layer automatically initiates a risk avoidance procedure and readjusts the rescue priority.

[0011] Furthermore, the tactical path layer employs a dynamic search algorithm based on a spatiotemporal state grid. This algorithm introduces a fourth dimension, namely the time dimension, on top of the three-dimensional spatial coordinates, transforming the fire spread prediction and smoke diffusion prediction output by the physical evolution prediction module into spatiotemporal obstacles. By searching for a trajectory with the minimum cost in the spatiotemporal state grid, the tactical path layer ensures that the robot not only avoids static obstacles at the current moment, but also does not collide with evolving high-temperature or collapse zones on its future operating path.

[0012] In one embodiment of the present invention, the operation execution layer includes a nonlinear predictive control unit and a motor vector control unit; the nonlinear predictive control unit calculates the optimal control sequence based on the trajectory instructions issued by the tactical path layer, taking into account the robot's mass distribution, ground friction coefficient, and dynamic constraints; the motor vector control unit achieves precise torque control of the drive motor through high-frequency sampling of motor current and encoder feedback, ensuring the robot's motion robustness in harsh terrain.

[0013] Furthermore, the system also includes a communication relay compensation module, which is used to monitor the signal strength of the communication link in a closed environment where wireless signals are limited; when the signal strength attenuates to below 20%, the hierarchical collaborative control module switches to local autonomous mode, and the strategic mission layer issues an autonomous return command or a local exploration command to find a signal enhancement area.

[0014] As one embodiment of the present invention, when generating a semantic voxel map, the environmental voxel construction unit divides the space into cubic voxels with a side length of 0.1 meters; the attribute value of each voxel is stored in a multi-channel tensor, wherein the first channel stores the geometric occupancy probability, ranging from 0 to 1; the second channel stores the real-time temperature value in degrees Celsius; the third channel stores the material type label, used to distinguish between combustibles, non-combustible obstacles and fire rescue passages; and the fourth channel stores the thermal radiation intensity.

[0015] Furthermore, when performing spatiotemporal alignment, the multi-source feature alignment unit uses the sampling frequency of the lidar as a reference to interpolate the infrared thermal imaging data and the acoustic sensor data to ensure that the data alignment deviation of all modes is less than 2 milliseconds.

[0016] As one embodiment of the present invention, the heat conduction model used in the fire thermodynamic modeling unit takes into account the relationship between the thermal conductivity of building materials and temperature. In the prediction process, the unit divides the calculation area into a macroscopic grid of 1 meter by 1 meter by 1 meter, solves the energy conservation equation using the finite volume method, and outputs a vector diagram of the temperature field evolution in the next 120 seconds.

[0017] Furthermore, when planning a path, the tactical path layer also calculates the survival factor of the path. The survival factor is determined by the predicted temperature, predicted flue gas concentration and predicted collapse probability along the path. When the survival factor of the pre-selected path is lower than 0.85, the system will automatically increase the cost weight of the path, prompting the planner to select a safer, albeit longer, alternative path.

[0018] As one embodiment of the present invention, the sampling period of the nonlinear predictive control unit is set to 10 milliseconds and the prediction time domain is set to 2 seconds. At each sampling time, the unit obtains the optimal acceleration sequence within the next 2 seconds by solving a quadratic programming problem with inequality constraints, and executes only the first control variable in the sequence. Then, at the next sampling time, it performs rolling optimization using the latest sensing data.

[0019] Furthermore, the strong interference suppression unit establishes a Gaussian mixture model to characterize the backscattering features of particulate matter in dense smoke. When the intensity distribution of the lidar reflection signal conforms to the characteristics of this Gaussian mixture model, the system identifies it as environmental interference and removes it, thereby increasing the robot's obstacle recognition accuracy in dense smoke environments from 45% in traditional methods to over 92%.

[0020] As one embodiment of the present invention, the structural risk assessment unit has a built-in database of the yield strength of common building materials at different temperatures; the unit obtains the surface temperature of the load-bearing column in the voxel map in real time and estimates the core temperature inside the column using a one-dimensional heat conduction model; once the core temperature reaches the critical softening point, the system immediately sends the highest level alarm signal to the hierarchical collaborative control module and forcibly interrupts the current operation task, switching to the emergency evacuation procedure.

[0021] Furthermore, the strategic task layer in the hierarchical collaborative control module has multi-machine collaborative functions; when multiple rescue robots are working simultaneously, the strategic task layer divides the task area into sub-regions with a mutual overlap rate of 10% through a distributed consensus protocol and assigns them to different robot units to maximize coverage efficiency.

[0022] As one embodiment of the present invention, the multimodal global perception module also includes an acoustic-assisted positioning unit; in extreme cases where the lidar and infrared sensors experience electron drift due to extreme high temperatures, the acoustic-assisted positioning unit uses the microphone array carried by the robot to capture specific frequency sound waves emitted by fire hydrants or preset acoustic beacons, and uses the time difference of arrival algorithm to calculate the robot's relative position to the beacon, providing bottom-level positioning redundancy support for the hierarchical control system.

[0023] Furthermore, the system's power management module monitors the voltage, current, and internal resistance distribution of the power battery in real time. When the physical evolution prediction module predicts that the temperature in the area ahead will exceed 80 degrees Celsius, the power management module automatically increases the power level of the heat dissipation system and adjusts the battery's discharge rate to prevent the battery from experiencing thermal runaway due to excessively high ambient temperatures.

[0024] As one embodiment of the present invention, the operation execution layer also integrates a tactile force feedback algorithm; when the robot's robotic arm performs disassembly or handling tasks, it senses the contact force in real time through a pressure sensor installed on the end effector and feeds back the torque fluctuation to the nonlinear predictive control unit to achieve hybrid control of force and position, preventing secondary injury to trapped personnel during search and rescue.

[0025] Furthermore, the flue gas dynamics evolution unit utilizes microfluidic simulation technology based on the lattice Boltzmann method to calculate the turbulence intensity of flue gas in the fire scene in real time; when a deflagration risk is predicted in a specific area, the unit will generate a virtual physical repulsion field to guide the tactical path layer to avoid the core risk area.

[0026] As one embodiment of the present invention, the layers of the hierarchical collaborative control module exchange data through high-speed industrial Ethernet to ensure that the end-to-end latency between strategic instructions, tactical trajectories and execution actions is less than 5 milliseconds.

[0027] Compared with the prior art, the advantages and positive effects of the present invention are as follows: 1. This invention fundamentally solves the problem of perception distortion in firefighting robots under dense smoke and high temperature interference by constructing a multimodal global perception module that integrates infrared, laser and acoustic technologies. By utilizing the penetrability of infrared thermal imaging and the redundancy of acoustic-assisted positioning, the system can still maintain accurate modeling of environmental geometry and temperature field in scenarios where visible light is severely limited, reducing the positioning drift rate by more than 70% and greatly improving the robot's survival and operation capabilities in extreme environments.

[0028] 2. This invention introduces a physical evolution prediction module, elevating static obstacle avoidance to a dynamic prediction level. Through real-time modeling and prediction of fire thermodynamics, smoke dynamics, and building structures, the system can predict the risk evolution trend within the next minute, enabling the robot to possess advanced intelligent behaviors such as predicting collapse and avoiding deflagration zones. This advanced decision-making mechanism based on physical laws changes the limitation of traditional robots that can only passively respond to known obstacles, significantly improving the safety and success rate of rescue missions.

[0029] 3. The hierarchical collaborative control architecture adopted in this invention effectively balances the global optimality and real-time responsiveness of task execution. By decoupling macroscopic task decomposition, spatiotemporal path planning, and high-frequency dynamic control, the system can complete the self-correction of the control law within 10 milliseconds when facing instantaneous dynamic changes in the fire scene. At the same time, the combination of nonlinear predictive control and multi-objective optimization model ensures the stability of the robot's motion in complex unstructured terrain, eliminates control oscillations caused by sensor noise or sudden changes in commands, and provides a solid theoretical guarantee for high-intensity fire rescue missions.

[0030] 4. This invention solves the risk of loss of control caused by communication interruption in complex building structures by integrating communication relay compensation and autonomous return mechanisms; combined with multi-machine collaborative protocols, it realizes the paradigm shift of fire rescue from single-machine individual operation to multi-machine cluster collaboration, significantly improving the search coverage and rescue efficiency of disaster sites. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the overall technical architecture of the hierarchical control system for fire rescue robots that integrates multimodal perception and physical evolution prediction, as proposed in this invention. Figure 2 This is a schematic diagram of the core principle framework of the physical evolution prediction module in this invention; Figure 3 This is a logical flow diagram of the multimodal global perception module in this invention; Figure 4 This is a schematic diagram of the multi-level interaction relationship and data flow of the hierarchical collaborative control module in this invention. Detailed Implementation Example 1

[0032] Please refer to the appendix. Figure 1This embodiment discloses a hierarchical control system for a fire rescue robot that integrates multimodal perception and physical evolution prediction. The system's overall architecture is divided into multiple interconnected logical levels and functional modules. The core operating logic of this system lies in achieving high-dimensional, in-depth understanding of the disaster site environment through a multimodal global perception module. Subsequently, a physical evolution prediction module performs forward-looking extrapolation of the acquired environmental information over time. Finally, a hierarchical collaborative control module makes optimal motion and task decisions based on the prediction results. This design architecture not only solves the problem of the robot's ability to see clearly in extreme environments but also addresses the engineering challenges of ensuring the robot's stability and accuracy in the face of dynamic risks.

[0033] Please refer to the attached document. Figure 3 The multimodal global perception module, serving as the input source for the entire control system, is responsible for transforming the complex physical signals of the disaster site into a computable representation in digital space. Internally, the multimodal global perception module is subdivided into a strong interference suppression unit, a multi-source feature alignment unit, and an environmental voxel construction unit. When processing data, the strong interference suppression unit first utilizes the physical characteristics of infrared thermal imaging sensors penetrating dense smoke to obtain infrared thermal images unobstructed by suspended particulate matter. Simultaneously, an acoustic array sensor actively emits sound pulses and receives reflected waves, using the propagation characteristics of sound waves in non-uniform media to perform a rough acoustic scan of the obstacles' outlines. The core of the strong interference suppression unit lies in using the aforementioned infrared and acoustic data as a mask to perform deep cleaning of the raw point cloud data acquired by the lidar. When the lidar beam encounters micron-sized carbon black particles or water vapor clouds in the dense smoke, it generates a large number of pseudo-random noise points, leading to misjudgments by the robot navigation algorithm. The strong interference suppression unit establishes a Gaussian mixture model to characterize the backscattering characteristics of particulate matter in the dense smoke. When the intensity distribution and spatial clustering of the lidar reflection signal conform to the probabilistic characteristics of the Gaussian mixture model, the system classifies it as environmental interference rather than a physical obstacle and removes it from the point cloud coordinate stream. This processing method significantly improves the robot's obstacle recognition accuracy in dense smoke environments from 45% of traditional methods to over 92%, providing realistic terrain data support for subsequent path planning.

[0034] After data cleaning, the multi-source feature alignment unit begins spatiotemporal synchronization. This unit maps the temperature field features carried in the infrared image, the geometric structure features extracted by the lidar, and the spatial location features resolved by the acoustic sensor to a unified Cartesian coordinate system. To eliminate data misalignment caused by inconsistent sampling frequencies of different sensors, the multi-source feature alignment unit uses the lidar sampling frequency as the global system clock reference and performs high-order interpolation processing on the infrared thermal imaging data and the acoustic sensor data. The synchronization mechanism within this unit ensures that the data alignment deviation of all modes is less than 2 milliseconds, thereby avoiding motion blur and positioning errors caused by sensor time delay differences during high-speed robot movement.

[0035] The aligned multimodal data stream is input into the environmental voxel construction unit. This unit divides the perceived space into cubic voxels with sides of 0.1 meters and assigns a multi-dimensional attribute tensor to each voxel. The attribute values ​​of each voxel are stored in a 4-channel tensor. The first channel stores the geometric occupancy probability, ranging from 0 to 1, describing the presence of physical obstacles at that point. The second channel stores the real-time temperature value in degrees Celsius, accurately reflecting the thermal load at that location. The third channel stores the material type label, distinguishing combustible materials, non-combustible obstacles, and pre-defined fire rescue routes through the fusion and classification of multimodal data. The fourth channel stores the thermal radiation intensity, providing energy flux data for subsequent fire thermodynamic modeling. In this way, the environmental voxel construction unit generates a dynamic map containing semantic information, providing a complete data foundation for the physical evolution prediction module.

[0036] Please refer to the attached document. Figure 2 The physical evolution prediction module, based on the environmental voxel map output by the multimodal global perception module, conducts in-depth physical modeling and spatiotemporal evolution simulation. This module consists of a fire field thermodynamic modeling unit, a smoke dynamics evolution unit, and a structural risk assessment unit. The fire field thermodynamic modeling unit is the core of the physical evolution, and its working principle is based on the law of conservation of energy to numerically simulate the heat conduction, convection, and radiation processes within the fire field. This unit divides the computational domain into a 1m x 1m x 1m macroscopic grid and uses the finite volume method to solve for the fire field energy distribution.

[0037] In the specific calculation process of this unit, in order to accurately describe the spatial distribution changes of the temperature field, the following thermophysical evolution control formula is introduced:

[0038] In the above formula, the physical quantities represented by the letters and their engineering significance are as follows: The density value in the first term on the left is determined by the material type label, while the specific heat capacity is dynamically corrected according to the nonlinear function of the real-time temperature value; The first term on the right side of the equal sign represents the contribution of heat conduction, in which the thermal conductivity is set as a dynamic parameter that changes with temperature, reflecting the degradation of the physical properties of building materials after being heated; The last term represents the intensity of the internal heat source per unit volume, which is mainly converted from the data of the thermal radiation intensity channel.

[0039] The fire thermodynamics modeling unit, by solving this partial differential equation, can calculate the spatial evolution vector diagram of the fire temperature field over the next 60 to 120 seconds. This means the robot can not only perceive the current fire situation but also predict the direction of heat wave diffusion. Simultaneously, the smoke dynamics evolution unit utilizes microfluidic simulation technology based on the lattice Boltzmann method, combined with prior information such as wind direction, air pressure, and building openings obtained from sensors, to simulate the diffusion path and concentration distribution of smoke. The dynamic visibility degradation area prediction map generated by this unit can inform the control system which areas will completely lose visibility in the next tens of seconds, thus allowing for advance planning of avoidance routes.

[0040] The structural risk assessment unit focuses on monitoring building safety. This unit extracts the cumulative temperature effect and deformation data of load-bearing components from the environmental voxel map in real time and compares it with a built-in database of yield strength of common building materials. By calculating the core temperature distribution inside the load-bearing components, the unit can assess the remaining load-bearing capacity of the components. Once the probability of structural collapse is predicted to exceed a preset safety threshold of 15%, the structural risk assessment unit immediately outputs a time-scaled obstacle avoidance zone index, forcing the hierarchical collaborative control module to modify the task path, ensuring the robot's survival safety during the search and rescue process.

[0041] Please refer to the attached document. Figure 4 The hierarchical collaborative control module, acting as the system's brain, receives predictions from the physical evolution prediction module. Through decoupled collaboration between the strategic task layer, tactical path layer, and operational execution layer, it achieves precise control under complex dynamic constraints. The strategic task layer, located at the top of the architecture, is responsible for decomposing macro-level rescue objectives. This layer generates the optimal rescue task sequence by constructing a multi-objective optimization model, using search and rescue efficiency, robot remaining energy, task urgency, and its own safety as comprehensive evaluation indicators. When it receives a high collapse risk alarm or deflagration risk prediction from the physical evolution prediction module, the strategic task layer automatically initiates a risk avoidance procedure and re-prioritizes tasks. For example, the system might force the robot to switch from a high-risk internal reconnaissance task to an emergency evacuation task.

[0042] The tactical path layer, under the command of the strategic mission layer, performs specific path planning. This layer employs a dynamic search algorithm based on a spatiotemporal state grid. Traditional path planning only considers three-dimensional spatial coordinates, while this system introduces a fourth dimension, the time dimension. The tactical path layer transforms the fire spread trend output by the fire thermodynamic modeling unit and the smoke diffusion trajectory output by the smoke dynamics evolution unit into obstacles in the spatiotemporal domain. This layer ensures that the robot can avoid not only the current static walls but also potential future flames or dense smoke by searching for a trajectory with the minimum cost in the four-dimensional spatiotemporal state grid. During the calculation process, the tactical path layer also introduces the crucial parameter of survival factor. The survival factor is calculated by weighting the predicted temperature, smoke concentration, and predicted collapse probability along the path. If the survival factor of a pre-selected path is lower than 0.85, the system automatically increases the cost weight of that path, guiding the robot to choose a longer but safer alternative route.

[0043] The operation execution layer, located at the bottom of the control architecture, is responsible for translating the abstract paths generated by higher layers into concrete hardware drive instructions. This layer contains a nonlinear predictive control unit and a motor vector control unit. The sampling period of the nonlinear predictive control unit is set to 10 milliseconds, and the prediction time domain is set to 2 seconds. At each sampling moment, this unit solves a quadratic programming problem with kinematic and dynamic constraints to obtain the optimal acceleration sequence for the next 2 seconds. The system executes only the first control variable in this sequence, and then performs rolling optimization at the next sampling moment using the latest sensing and prediction data.

[0044] To achieve an accurate response to the output commands of the nonlinear predictive control unit, the operation execution layer introduces the following optimal control objective function:

[0045] In this objective function, the first term represents the state error term, which measures the deviation between the robot's predicted trajectory and the reference path. The weight matrix reflects the different levels of importance given to position, velocity, and attitude errors. The second term represents the control increment term, which limits abrupt changes in the actuator's movements to ensure smooth torque output from the motor. By minimizing this objective function in each control cycle, the system achieves highly robust motion in harsh and uneven terrain.

[0046] The motor vector control unit in the operation execution layer achieves sub-millimeter-level control of the drive motor through high-frequency sampling of motor current and encoder feedback. Furthermore, the operation execution layer integrates a tactile force feedback algorithm. When the robot's robotic arm performs obstacle clearing or rescue operations involving trapped personnel, pressure sensors mounted on the end effector feed back the real-time sensed contact force to the nonlinear predictive control unit, enabling hybrid force and position control. This mechanism effectively prevents secondary crush injuries to trapped personnel caused by improper force control during stressful search and rescue operations.

[0047] To address signal shielding issues in complex building environments, this system is also equipped with a communication relay compensation module. This module monitors the wireless link signal strength between the robot and the command center in real time. Once the signal strength weakens to below 20%, the hierarchical collaborative control module immediately switches to local autonomous mode. At this point, the strategic mission layer issues an autonomous return command based on the prior map and existing predictive data, or directs the robot to search for areas with enhanced signal strength for local exploration, and resumes the mission once communication is restored.

[0048] The power management module safeguards the system's energy security at the underlying level. It monitors the battery's voltage, current, temperature rise rate, and internal resistance distribution in real time. When the physics evolution prediction module anticipates that the operating temperature in the area ahead will exceed 80 degrees Celsius, the power management module will forcibly activate two emergency measures: first, it will increase the power of the electronic equipment's cooling system to its maximum level; second, it will limit the battery's discharge rate by adjusting the inverter's modulation strategy. These measures effectively prevent the battery from experiencing thermal runaway under extreme external temperatures, ensuring the robot's stable operation for extended periods in high-temperature fire environments.

[0049] The multimodal global perception module also includes an acoustic-assisted localization unit. In extremely rare high-temperature environments, the laser in a lidar system or the imaging chip in an infrared sensor may experience significant electronic drift due to thermal effects, causing conventional visual localization to fail. In this situation, the acoustic-assisted localization unit provides redundant support. It utilizes a high-sensitivity microphone array carried by the robot to capture specific frequency sound waves emitted from fire hydrants, alarms, or pre-set acoustic beacons. Using a time-of-arrival algorithm, this unit can calculate the robot's relative position to these known sound sources, thus providing low-level localization calibration for the hierarchical control system and ensuring that the robot can still find an escape route even when visually impaired.

[0050] At the advanced application level of the system, the strategic mission layer also possesses multi-machine collaborative capabilities. When multiple fire rescue robots of the same or different models are deployed on-site, the strategic mission layer manages cluster operations through a distributed consensus protocol. The system divides complex disaster area mission zones into multiple sub-regions, maintaining a 10% overlap between regions to facilitate handover and relay search and rescue. This cluster operation mode significantly improves search coverage in large-area disaster areas, ensuring optimal allocation of rescue resources.

[0051] Data interaction between functional modules is achieved via high-speed industrial Ethernet. Strategic commands, tactical trajectories, real-time perceived voxel maps, and underlying motor control commands are transmitted in the internal network as high-priority packets, ensuring that end-to-end communication latency is consistently below 5 milliseconds. This extremely low latency enables the robot to react instantly to sudden physical changes such as deflagration and collapse in a fire. Example 2

[0052] Building upon Example 1, this embodiment further refines the configuration of the flue gas dynamics evolution unit within the physical evolution prediction module, specifically targeting high-risk scenarios such as chemical plants containing flammable and explosive gases. In the chemical plant scenario, the flue gas dynamics evolution unit, in addition to calculating the visible flue gas concentration, also focuses on enhancing the simulation of the diffusion concentration of volatile organic compounds.

[0053] This unit incorporates a multi-component fluid dynamics model when simulating the flow field using the lattice Boltzmann method. The system acquires real-time hazardous gas concentration samples via photoionization sensors integrated into the multimodal global sensing module, and uses these samples as boundary conditions for the flue gas dynamic evolution unit. During the simulation, this unit can predict the diffusion envelope of hazardous gases over the next 180 seconds based on the distribution of ventilation ducts, equipment layout, and real-time meteorological data within the plant area.

[0054] When the gas concentration in a specific area is predicted to approach its explosive limit, the physics evolution prediction module generates a virtual physical repulsion field. This repulsion field manifests as an extremely high potential barrier in the cost function of the tactical path layer. During pathfinding, the tactical path layer automatically marks this area as impassable, guiding the robot to avoid the potential explosion core. Unlike conventional obstacle avoidance, this method is based on scientific predictions of gas diffusion patterns, enabling the avoidance of areas that are currently safe but will soon become dangerous.

[0055] Meanwhile, at the operation execution layer, the nonlinear predictive control unit adjusts the smoothness of the acceleration sequence for such flammable and explosive environments. To prevent sparks from being generated by high-speed friction between the robot tracks and the ground, the system automatically limits the robot's instantaneous acceleration and maximum travel speed. The motor vector control unit also enters a low-power explosion-proof mode, reducing electromagnetic interference and potential static electricity accumulation by optimizing the switching frequency.

[0056] In this specific implementation, the system's strategic mission layer also performs data cross-checking with the chemical plant's fixed monitoring system. When a fixed sensor detects an abnormal increase in pressure in a pressure vessel, this information is transmitted to the hierarchical control system via the communication relay compensation module. The strategic mission layer immediately invokes the physical evolution prediction module to numerically extrapolate the impact range of the shock wave following the vessel's rupture, and dynamically adjusts the warning distance of the rescue robot based on this calculation.

[0057] Furthermore, in this embodiment, the environmental voxel construction unit refines the voxel size to 0.05 meters to capture the minute gaps between precision chemical pipelines, ensuring the robot has higher maneuverability in complex pipe gallery structures. The strong interference suppression unit addresses potential acid and alkali spray interference within chemical plants by adding a specific chemical aerosol scattering model. By adjusting the mean and variance parameters of the Gaussian mixture model, it ensures clear laser point cloud data can still be obtained under chemical spray conditions.

[0058] This enhanced control system, which integrates hazardous gas diffusion prediction, greatly expands the robot's applicability in special rescue fields such as petrochemical plants. Its core advantage lies in the deeper coupling of perception, prediction, and control at a higher dimension, enabling the robot to perceive not only thermal fields but also chemical fields and explosion risk fields, truly achieving intelligent and safe operation in chemical plant fire environments. Example 3

[0059] This embodiment focuses on fire rescue applications in large underground spaces or tunnels, and specifically enhances the collaboration mechanism between the communication relay compensation module and the hierarchical collaborative control module. In underground environments, GPS signals are completely ineffective, and radio signals suffer severe attenuation due to tunnel bends and metal structure shielding.

[0060] To address this issue, the multi-source feature alignment unit in this embodiment enhances the dead reckoning function based on odometry and inertial measurement units. This unit samples data from the robot's internal gyroscope and accelerometer at high frequency, and combines this with the travel distance provided by the motor encoder in the operation execution layer, using an extended Kalman filter algorithm to make a preliminary estimate of the robot's pose. Simultaneously, the acoustic-assisted localization unit plays a dominant role in this scenario; it listens to the echo signals reflected from the tunnel walls and uses echo ranging principles to correct the accumulated errors in dead reckoning in real time.

[0061] The communication relay compensation module employs a multi-stage skip strategy in underground environments. When the robot degrades into the tunnel, causing the signal strength to attenuate to 30%, the system automatically assesses the current link stability. If further signal deterioration is predicted, the strategic mission layer instructs the robot to deploy a miniature communication relay node. This node, equipped with an independent power supply and signal-enhancing antenna, serves as an information bridge between the robot and the ground command center.

[0062] In the physical evolution prediction module, the smoke dynamics evolution unit focuses on modeling the piston effect within the tunnel. In tunnel fires, due to spatial limitations, smoke flow is significantly affected by strong airflow generated by vehicle movement or exhaust fans. The smoke dynamics evolution unit receives fan operating parameters from the tunnel monitoring system and calculates the pressure gradient distribution within the tunnel in real time. The prediction model generated by this unit can accurately indicate the hierarchical distribution of smoke within the next 5 minutes, determining which intervals along the tunnel's height belong to smoke-free or low-concentration zones.

[0063] Based on the vertical distribution of smoke concentration, the tactical path layer not only plans the robot's horizontal path but also dynamically adjusts its working height by controlling the robot's variable-height chassis. When predictions indicate that high-temperature smoke accumulates in the upper part of the tunnel while the air below is relatively clean, the operation execution layer receives a command to reduce the chassis height, allowing the robot to move in a low posture and avoid damage to sensitive electronic components from the high-temperature smoke layer.

[0064] Furthermore, the structural risk assessment unit adds an early warning system for tunnel lining spalling in the tunnel environment. By scanning the geometric data of the tunnel ceiling in real time and combining it with temperature field prediction, the unit can detect lining voids and cracks caused by thermal expansion. Once the predicted probability of lining collapse exceeds a preset value, the system automatically generates a dynamic obstacle avoidance zone in the voxel map, directing the robot to flexibly avoid obstacles.

[0065] Since underground spaces are often accompanied by severe dampness and water accumulation, the nonlinear predictive control unit in the operation execution layer sets the ground friction coefficient as a time-varying parameter in this embodiment. By monitoring the slip ratio of the drive wheels, this unit can calculate the degree of slippage of the ground in real time and adjust the weight matrix of the control increment term accordingly. When the robot travels on a waterlogged section of road, the system automatically reduces the turning radius and turning speed to prevent sideslip and mission failure.

[0066] This implementation method, optimized for underground spaces, solves the problems of firefighting robots losing contact, getting lost, or being trapped by smoke in confined spaces such as tunnels by integrating acoustic positioning, communication relay, and environmental adaptive control. Its comprehensive perception and prediction capabilities ensure that even in extremely harsh underground fire environments, the robot can maintain efficient operation and a very high survival rate. Example 4

[0067] This embodiment describes a specific implementation method suitable for indoor search and rescue in high-rise buildings. In high-rise building scenarios, structures such as stairs, narrow corridors, and doorways place extremely high demands on the robot's mobility.

[0068] In this embodiment, the environment voxel construction unit of the multimodal global perception module employs a multi-scale voxel strategy. In open hall areas, the voxel size is maintained at 0.2 meters to improve computational speed; while in fine structures such as stairwells or door frame edges, the voxels are automatically refined to 0.02 meters. This non-uniform mesh generation technique ensures that the robot has millimeter-level perception accuracy when traversing narrow paths, while maintaining efficient processing performance in open areas.

[0069] The fire thermodynamics modeling unit of the physical evolution prediction module focuses on the chimney effect in high-rise scenarios. Due to the presence of vertical passages such as elevator shafts and stairwells within buildings, the thermal convection velocity after a fire is much higher than in horizontal areas. The modeling unit constructs a three-dimensional model of rising thermal airflow by acquiring real-time information about the opening status of these vertical passages. This model can accurately predict the speed at which flames spread upwards and the diffusion logic of smoke between different floors.

[0070] Based on these vertical spread predictions, the strategic mission layer generates a three-dimensional search and rescue priority matrix. If the physical evolution prediction module indicates that an evacuation stairwell will be completely blocked by dense smoke in 3 minutes, the strategic mission layer will immediately raise the search and rescue priority of that area to the highest level and deploy nearby robots to intervene via the fastest route.

[0071] In this embodiment, the tactical path layer optimizes the spatiotemporal state lattice search algorithm by introducing a three-dimensional Euclidean distance map. This algorithm can plan not only plan planar paths but also the robot's stepping trajectory when going up and down stairs. When the robot needs to pass through a burning doorway, the tactical path layer combines the instantaneous thermal radiation distribution output by the fire thermodynamic modeling unit to calculate an instantaneous penetration path with minimal thermal damage.

[0072] The haptic feedback algorithm at the operation execution layer plays a crucial role in high-rise indoor environments. Due to the presence of numerous scattered obstacles such as furniture and collapsed ceilings, the robot inevitably comes into physical contact with its environment during movement. The haptic feedback algorithm can identify the physical characteristics of these obstacles, distinguishing between rigid walls and lightweight, pushable partitions. Based on this feedback information, the nonlinear predictive control unit dynamically adjusts the drive torque. If it encounters pushable debris, the system automatically increases the motor torque output to clear a path; if it encounters a load-bearing wall, it immediately switches to path replanning logic.

[0073] For elevator systems in high-rise buildings, the system's communication relay compensation module can deeply interface with the building's intelligent management system. In an emergency, the robot can request control of the elevator through the internal communication protocol. By sending commands at a specific frequency, the robot can call the elevator to a designated floor and integrate the sensor data inside the elevator into its own comprehensive perception system.

[0074] Finally, the power management module in this embodiment particularly strengthens the management of power consumption peaks generated by the frequent movements of the robotic arm. When performing high-intensity indoor obstacle-breaking tasks, the power management module predicts the energy demand of subsequent actions and rationally allocates the instantaneous current of the power battery to ensure that the robotic arm is provided with sufficient instantaneous burst force without damaging the battery life.

[0075] In summary, this embodiment demonstrates the system's high adaptability in complex high-rise building environments. Through a series of optimizations to perception precision, physical prediction dimensions, and control flexibility, the system achieves autonomous navigation and intelligent search and rescue in unstructured indoor environments, significantly improving the efficiency of fire rescue in high-rise buildings. Example 5

[0076] This embodiment focuses on robotic collaborative operations in large-scale forest fire prevention and urban-rural fringe fire scenarios. In such large-scale environments, the perception range of a single robot is limited, therefore, multi-robot collaborative functionality becomes the core pillar of the system.

[0077] In this embodiment, the strategic mission layer employs a task allocation protocol based on a distributed auction mechanism. When multiple robotic units are deployed along the edge of a forest fire covering several square kilometers, each robot's multimodal global perception module generates a local environmental voxel map in real time. The strategic mission layer stitches these local maps together into a global macro-situation map via industrial Ethernet. For newly discovered fire points or trapped targets, the strategic mission layer conducts real-time task bidding and allocation based on each robot's remaining battery power, predicted hazard coefficient from the fire source, and its respective firefighting payload.

[0078] The physical evolution prediction module introduces a dynamic forest fire spread model in forest fire scenarios. The fire thermodynamics modeling unit uses terrain slope, tree species flammability coefficient, and instantaneous wind speed as core variables. Its calculation formula is extended to an energy equation that considers the coupling of the external wind field, enabling it to predict the leapfrog spread path of forest fires within the next 30 minutes.

[0079] The smoke dynamics evolution unit focuses on analyzing the long-distance drift of smoke on a large spatial scale. Due to the enormous volume of smoke generated by forest fires and its severe influence by airflow, this unit uses large eddy simulation technology to calculate large-scale turbulence above the fire site, predict the impact range of the smoke-covered area on the robot's optical perception system, and guide the robot to move in advance to a location with favorable wind direction.

[0080] In this embodiment, the environmental voxel construction unit refines the terrain material property channels, adding subdivided dimensions such as humidity and debris thickness. This data primarily comes from the robot's acoustic array analyzing the acoustic characteristics of the surface soil. By distinguishing between dry, flammable areas and damp, safe areas, the tactical path layer can plan a path that can quickly reach the target point while providing a physical barrier in the event of a sudden change in fire intensity.

[0081] The communication relay compensation module employs a satellite link as backup in field environments. When the local industrial Ethernet or dedicated communication frequency band experiences severe packet loss due to forest fire interference, the system automatically switches to a low-Earth orbit satellite communication terminal to ensure the continuous issuance of strategic commands. Although the satellite link has higher latency, it is sufficient to maintain macro-level command at the strategic mission level.

[0082] Due to the extremely complex terrain in the field, the nonlinear predictive control unit in the operation execution layer incorporates terrain-adaptive self-learning logic. By analyzing the slippage rate of the robot's tracks under different vegetation and slope conditions, this unit can correct the robot's dynamic model parameters online. The motor vector control unit optimizes the efficiency curve for long-distance travel in the field, intelligently adjusting the motor's magnetic field orientation parameters to improve the robot's energy efficiency ratio by more than 15% in full-speed cruising mode.

[0083] Furthermore, multi-robot collaboration is also reflected in the relay of firefighting methods. While one robot uses the tactile force feedback algorithm of its end effector to dig firebreaks, another robot carrying a high-pressure fine water mist device will precisely pre-spray areas that are about to reignite based on deflagration predictions provided by the physical evolution prediction module. This high degree of tactical coordination enables the system to form an intelligent firebreak in large-scale disaster areas, greatly improving firefighting efficiency and reducing operational risks for ground personnel.

[0084] The above embodiments are merely illustrative of the technical concept and features of the present invention, intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly, and should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A fire rescue robot hierarchical control system that fuses multi-modal perception and physical evolution prediction, characterized in that, include: The multimodal global perception module is used to collect raw observation data of the disaster site environment in real time, and to construct a high-dimensional environmental characterization including temperature attributes, material attributes and geometric occupancy attributes through the information complementarity of heterogeneous sensors; the physical evolution prediction module is used to perform physical modeling and spatiotemporal evolution extrapolation of the fire thermodynamic distribution, smoke spread trend and building structure stability based on the high-dimensional environmental characterization output by the multimodal global perception module, using the fire thermodynamic modeling unit, the smoke dynamics evolution unit and the structural risk assessment unit. The hierarchical collaborative control module is used to receive the prediction results output by the physical evolution prediction module. Through the decoupling and collaboration of the strategic task layer, tactical path layer and operation execution layer, it enables the robot to make autonomous decisions and achieve high-precision motion control under complex dynamic constraints. The communication relay compensation module monitors the signal strength of the communication link in a closed environment with limited wireless signals. When the signal strength attenuates to below 20% of the preset value, the instruction layered collaborative control module switches to local autonomous mode. The strategic mission layer constructs a multi-objective optimization model based on search and rescue efficiency, robot energy efficiency, and safety indicators, and adjusts the rescue priority in real time according to the collapse probability output by the structural risk assessment unit. The tactical path layer employs a dynamic search algorithm based on a spatiotemporal state lattice, transforming the fire spread prediction and smoke diffusion prediction output by the physical evolution prediction module into spatiotemporal obstacles to search for the trajectory with the minimum cost in a four-dimensional spatiotemporal dimension. The operation execution layer includes a nonlinear prediction control unit, which solves a quadratic programming problem with inequality constraints based on the trajectory instructions issued by the tactical path layer, considering robot dynamics constraints, to output the optimal acceleration sequence and control the drive motor execution.

2. The hierarchical control system for a fire rescue robot integrating multimodal perception and physical evolution prediction according to claim 1, characterized in that, The multimodal global perception module includes a strong interference suppression unit and a multi-source feature alignment unit. The strong interference suppression unit is used to establish a Gaussian mixture model to characterize the backscattering features of particulate matter in dense smoke. When the intensity distribution of the lidar reflected signal conforms to the characteristics of the Gaussian mixture model, the corresponding signal is identified as environmental interference and removed. The multi-source feature alignment unit is used to map the temperature field features of the infrared image, the geometric structure features of the lidar, and the spatial position features of the acoustic sensor to a unified Cartesian coordinate system. Using the lidar sampling frequency as a reference, it performs interpolation processing on the infrared thermal imaging data and the acoustic sensor data to make the data alignment deviation of different modes less than 2 milliseconds.

3. The hierarchical control system for a fire rescue robot integrating multimodal perception and physical evolution prediction according to claim 1, characterized in that, The multimodal global perception module also includes an environmental voxel construction unit, which is used to divide the acquired space into cubic voxels with a side length of 0.1 meters and construct a 4-channel tensor attribute for each voxel. The four-channel tensor properties include: the first channel stores the geometric occupancy probability ranging from 0 to 1; the second channel stores the real-time temperature value in degrees Celsius; the third channel stores the material type label used to distinguish between combustibles, non-combustible obstacles and fire rescue channels; and the fourth channel stores the thermal radiation intensity used for thermodynamic modeling.

4. The hierarchical control system for a fire rescue robot integrating multimodal perception and physical evolution prediction according to claim 1, characterized in that, The evolution and deduction process of the fire field thermodynamic modeling unit is as follows: The computational domain is divided into a macroscopic grid of 1 meter by 1 meter by 1 meter; based on the law of conservation of energy, the thermophysical evolution control equation is solved using the finite volume method; the left-hand side of the thermophysical evolution control equation consists of the material density, specific heat capacity, and the rate of change of temperature over time, while the right-hand side consists of the heat conduction contribution term involving the dynamic thermal conductivity and the internal heat source intensity term derived from the thermal radiation intensity; by solving the thermophysical evolution control equation, a vector diagram of the spatial distribution evolution of the fire field temperature field within the next 120 seconds is output.

5. A hierarchical control system for a fire rescue robot integrating multimodal perception and physical evolution prediction according to claim 1, characterized in that, The specific execution logic of the flue gas dynamics evolution unit and the structural risk assessment unit is as follows: The flue gas dynamics evolution unit uses microfluidic simulation technology based on the grid Boltzmann method, combined with information on wind direction, air pressure and building openings inside the fire scene, to generate a dynamic prediction map of the line of sight degradation area; The structural risk assessment unit monitors the cumulative temperature effect and deformation data of the building's load-bearing components, and combines this with a built-in database of building material yield strength to predict the probability of building structure collapse; When the collapse probability exceeds 15%, it outputs an obstacle avoidance zone index with a time scale to the tactical path layer.

6. A hierarchical control system for a fire rescue robot integrating multimodal perception and physical evolution prediction according to claim 1, characterized in that, When planning a path, the tactical path layer also performs the following steps: calculating the path survival factor, which is determined by a weighted average of the predicted temperature, predicted smoke concentration, and predicted collapse probability along the path; comparing the path survival factor with a preset threshold of 0.85; and increasing the cost weight of the path in the dynamic search algorithm when the survival factor of the pre-selected path is lower than 0.85, so as to prompt the planner to select a safer alternative path.

7. A hierarchical control system for a fire rescue robot integrating multimodal perception and physical evolution prediction according to claim 1, characterized in that, The control logic of the nonlinear predictive control unit in the operation execution layer is as follows: the sampling period is set to 10 milliseconds and the prediction time domain is 2 seconds; at each sampling time, the optimal control sequence is solved by minimizing the optimal control objective function; the optimal control objective function includes a state error term for measuring the deviation between the predicted trajectory and the reference path, and a control increment term for limiting abrupt changes in the actuator's actions; after obtaining the optimal acceleration sequence, only the first control quantity in the sequence is executed, and rolling optimization is performed at the next sampling time using the latest sensing data.

8. A hierarchical control system for a fire rescue robot integrating multimodal perception and physical evolution prediction according to claim 1, characterized in that, It also includes a power management module for real-time monitoring of the voltage, current and internal resistance distribution of the power battery; the power management module is also used to: when the physical evolution prediction module predicts that the temperature in the area in front of the robot exceeds 80 degrees Celsius, increase the power level of the heat dissipation system and adjust the battery discharge rate to avoid the risk of battery thermal runaway.

9. A hierarchical control system for a fire rescue robot integrating multimodal perception and physical evolution prediction according to claim 1, characterized in that, The multimodal global perception module also includes an acoustic-assisted localization unit. This unit is used to capture specific frequency sound waves emitted by a preset acoustic beacon using a microphone array carried by the robot in the event of visual perception failure. It also uses a time-of-arrival (TOA) algorithm to calculate the robot's relative position to the preset acoustic beacon, providing redundant support for the underlying localization layer. The strategic task layer also features multi-machine collaboration capabilities, used to divide the task area into sub-regions with a 10% overlap rate through a distributed consensus protocol, and to assign tasks within each sub-region to different robot units.

10. A hierarchical control system for a fire rescue robot integrating multimodal perception and physical evolution prediction according to claim 1, characterized in that, The operation execution layer also integrates a tactile force feedback algorithm, and the modules interact with each other via industrial Ethernet. The tactile force feedback algorithm is used to sense contact force through pressure sensors installed on the robot's end effector and feed back torque fluctuations to the nonlinear predictive control unit to achieve hybrid control of force and position. The strategic task layer, tactical path layer and operation execution layer of the hierarchical collaborative control module exchange data via high-speed industrial Ethernet, and the end-to-end delay between strategic commands, tactical trajectories and execution actions is less than 5 milliseconds.

Images