Smoke environment sensing and obstacle avoidance navigation method and system based on fire-fighting robot

Abstract

The application relates to the field of intelligent navigation of fire-fighting robots, and discloses a smoke environment sensing and obstacle avoidance navigation method and system based on a fire-fighting robot; the method collects smoke concentration, visible light images, infrared thermal images, laser radar point clouds, laser radar echo intensity, millimeter wave radar echo and robot pose data during the travel of the fire-fighting robot; smoke degradation credibility of multiple azimuth sectors is calculated according to the above data, and a layered environment map including an entity obstacle layer, a smoke shielding layer and a thermal risk layer is constructed; a comprehensive passing cost map is generated according to the layered environment map, and an obstacle avoidance navigation path is planned. When there is a low credibility smoke shielding area in front of the path, the fire-fighting robot is controlled to perform an active detection action, and the map layer attributes are corrected according to the re-collected multiple frames of sensing results, and a return and evacuation safety corridor is maintained.

Description

Technical Field

[0001] This invention relates to the field of intelligent navigation for firefighting robots, specifically to a method and system for smoke environment perception and obstacle avoidance navigation based on firefighting robots. Background Technology

[0002] Firefighting robots are typically used for reconnaissance, firefighting assistance, environmental detection, and access to hazardous areas at fire scenes. Compared to manual entry into a fire scene, firefighting robots can perform some high-risk tasks in environments with high temperatures, smoke, toxic gases, and complex obstacles. To achieve autonomous movement, existing firefighting robots are generally equipped with sensing devices such as visible light cameras, infrared thermal imagers, lidar, ultrasonic sensors, smoke sensors, and inertial measurement units. Combined with environmental mapping, path planning, and obstacle avoidance control methods, the robot can move towards the target area and avoid obstacles.

[0003] However, in actual fire scenes, smoke can affect different sensing devices to varying degrees. Visible light images are easily blurred at the edges due to smoke obscuring them; lidar is easily affected by smoke particles, water mist, or hot air currents, producing unstable echoes; and while infrared thermal images can reflect temperature distribution, they may be difficult to directly correspond to physical obstacles due to high-temperature smoke diffusion, heat source obstruction, or local reflection. Existing navigation methods for firefighting robots mostly fuse the collected sensing data directly into an environmental map, or detour after detecting obstacles, typically without uniformly distinguishing and processing the decreased perception reliability caused by smoke, false obstacle echoes, high-temperature risk areas, and reversible paths.

[0004] Therefore, in environments with dense smoke and localized high temperatures, firefighting robots are prone to misinterpreting smoke scattering or water mist echoes as physical obstacles, and may also misjudge hazardous areas obscured by smoke as passable areas. This leads to frequent changes in path planning, unstable passage judgments, and even difficulty in reliably returning after penetrating deep into the fire due to decreased reliability of the retreat path perception. Therefore, it is necessary to provide a firefighting robot smoke environment perception and obstacle avoidance navigation method and system that can distinguish between physical obstacles, smoke obscuration, and thermal risk areas in smoke environments, and can actively detect and maintain retreat paths based on low-reliability areas. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a smoke environment perception and obstacle avoidance navigation method and system based on firefighting robots, in order to solve the technical problems existing in the prior art.

[0006] The above-mentioned technical objective of the present invention is achieved through the following technical solution: A smoke environment perception and obstacle avoidance navigation method based on firefighting robots includes the following steps: S1: When the firefighting robot moves within the target work area, it collects multi-source sensing data, including smoke concentration data, visible light image data, infrared thermal image data, lidar point cloud data, lidar echo intensity data, millimeter-wave radar echo data, and robot pose data. S2: Based on smoke concentration data, edge preservation in visible light image data, thermal gradient in infrared thermal image data, attenuation of lidar echo intensity data, drift of lidar point cloud data, and stability of millimeter-wave radar echo data, calculate the smoke degradation reliability of multiple directional sectors around the fire-fighting robot. S3: Based on the reliability of smoke degradation and multi-source perception data, construct a layered environment map around the fire-fighting robot. The layered environment map includes a physical obstacle layer, a smoke obscuring layer, and a thermal risk layer. S4: Generate a comprehensive passage cost map based on the physical obstacle layer, smoke cover layer, thermal risk layer and robot pose data, and plan the obstacle avoidance navigation path of the fire-fighting robot based on the comprehensive passage cost map; S5: When there is a target grid area with smoke degradation confidence level lower than the preset detection confidence level threshold ahead of the obstacle avoidance navigation path, and the target grid area is located in the smoke obscuring layer and has not been written into the physical obstacle layer, control the fire-fighting robot to perform active detection actions; the active detection actions include reducing the travel speed, adjusting the observation angle, and conducting short-distance probing when the grid area corresponding to the braking distance in front of the fire-fighting robot has not been written into the physical obstacle layer. S6: After performing active detection, re-acquire the lidar point cloud data, lidar echo intensity data, millimeter-wave radar echo data and infrared thermal image data corresponding to the target grid area, and correct the map layer attributes of the target grid area based on the re-acquired multi-frame perception results. S7: Regenerate the comprehensive access cost map based on the revised layered environment map and update the obstacle avoidance navigation path of the fire-fighting robot; S8: Record the smoke degradation confidence level and thermal risk status of the areas already passed during the firefighting robot's movement, and maintain a safe retreat corridor based on the smoke degradation confidence level and thermal risk status of the areas already passed. The safe retreat corridor serves as a constraint condition for subsequent obstacle avoidance navigation path planning.

[0007] Preferably, in step S2, the 360-degree area around the fire robot is divided into eight directional sectors, with the current direction of travel of the fire robot as the reference direction, and each directional sector corresponds to a 45-degree angle range. Statistical analysis was performed on smoke concentration data, visible light image data, infrared thermal image data, lidar point cloud data, lidar echo intensity data, and millimeter-wave radar echo data for each azimuth sector. Specifically, the smoke concentration data in the corresponding azimuth sector is converted into a smoke concentration attenuation term; the ratio of the number of edge pixels in the visible light image data to the number of edge pixels in the calibrated clear state is used as an edge retention term; the temperature difference between adjacent sampling areas in the infrared thermal image data is used as a thermal gradient term; the decrease ratio of the lidar echo intensity data relative to the calibrated echo intensity is used as a lidar echo attenuation term; the point cloud position offset at the same spatial location in multiple consecutive frames of lidar point cloud data is used as a point cloud drift term; and the proportion of echo occurrences at the same spatial location in multiple consecutive frames of millimeter-wave radar echo data is used as a millimeter-wave stability term. After normalizing the smoke concentration attenuation term, edge preservation term, thermal gradient term, laser echo attenuation term, point cloud drift term, and millimeter wave stability term, the smoke degradation confidence of the corresponding directional sector is obtained by weighted summation according to the fixed weights stored in the fire robot controller.

[0008] Preferably, in step S3, when constructing the layered environment map, the area around the fire-fighting robot is divided into grid areas; The physical obstacle layer is used to record obstacle areas with stable geometric boundaries, the smoke obscuring layer is used to record areas where the perception credibility is reduced due to smoke but has not yet been confirmed as physical obstacles, and the thermal risk layer is used to record areas where the highest temperature value is greater than a preset passage temperature threshold or the temperature difference between adjacent grid areas is greater than a preset thermal gradient threshold.

[0009] Preferably, when constructing the physical obstacle layer, for any grid area, when the lidar point cloud data in the grid area forms a continuous geometric boundary in three consecutive frames, the maximum positional offset of the continuous geometric boundary in the three consecutive frames does not exceed one grid width, and the millimeter-wave radar echo data has a stable echo in the grid area for three consecutive frames, the grid area is written into the physical obstacle layer.

[0010] Preferably, when constructing the smoke masking layer, for any grid area, when the edge retention item in the visible light image data corresponding to the grid area is lower than the preset edge retention threshold, there is a near-range sheet echo in the lidar point cloud data, and the millimeter-wave radar echo data does not form three consecutive stable echoes in the grid area, the grid area is written into the smoke masking layer. The preset edge preservation threshold and the near-range sheet echo determination threshold are both calibration parameters stored in the controller before the fire-fighting robot enters the target work area.

[0011] Preferably, when constructing the thermal risk layer, for any grid area, when the highest temperature value in the infrared thermal image data corresponding to the grid area is greater than a preset passage temperature threshold, or when the temperature difference between the grid area and the adjacent grid area is greater than a preset thermal gradient threshold, the grid area is written into the thermal risk layer. The preset passage temperature threshold is determined based on the heat resistance rating of the fire robot's shell and the upper limit of the operating temperature of its internal electrical components, while the preset thermal gradient threshold is determined based on the amount of temperature change that the fire robot can withstand per unit travel distance.

[0012] Preferably, in step S4, when generating the comprehensive passage cost map, the grid area in the physical obstacle layer is set as an impassable marker, the grid area in the smoke obscuring layer is set as a deceleration detection marker, and the grid area in the thermal risk layer is set as a thermal risk avoidance marker. For grid regions that are not written into the entity obstacle layer, smoke masking layer, and thermal risk layer, the corresponding passage cost is determined based on the smoke degradation confidence level of the directional sector to which the grid region belongs. The lower the smoke degradation confidence level, the higher the corresponding passage cost.

[0013] Preferably, in step S5, the active detection action includes: controlling the fire-fighting robot to stop moving in a straight line before entering the target grid area, and recording the current pose as the detection start pose; Control the firefighting robot to reduce its preset detection speed; The firefighting robot is controlled to turn to the first detection direction and the second detection direction respectively. The first detection direction is the direction formed by turning to the left by a preset detection angle relative to the initial detection posture, and the second detection direction is the direction formed by turning to the right by a preset detection angle relative to the initial detection posture. Three frames of lidar point cloud data, three frames of lidar echo intensity data, three frames of millimeter-wave radar echo data, and three frames of infrared thermal image data were collected in the first and second detection directions, respectively. When the grid area corresponding to the braking distance in front of the fire-fighting robot is not written into the physical obstacle layer, the fire-fighting robot is controlled to move forward along the original direction of travel for a preset detection distance and then stop, and the corresponding multi-source perception data is collected again.

[0014] Preferably, in step S6, correcting the map layer attributes of the target grid area includes: when the re-acquired lidar point cloud data forms a continuous geometric boundary in three consecutive frames, and the re-acquired millimeter-wave radar echo data forms a stable echo in three consecutive frames, and the grid area where the stable echo is located overlaps with the grid area occupied by the continuous geometric boundary, the corresponding overlapping grid area is corrected from a smoke masking layer to a solid obstacle layer. When the position drift of the re-acquired lidar point cloud data exceeds the preset drift threshold in three consecutive frames, and the re-acquired millimeter-wave radar echo data does not form a stable echo in three consecutive frames, keep the target grid area in the smoke cover layer. When the reacquired infrared thermal image data meets the conditions for writing to the thermal risk layer, the target grid area is written to the thermal risk layer. In step S8, maintaining the safe corridor for retreat includes: recording the grid areas that have been passed according to the actual time sequence of the fire-fighting robot, forming a sequence of grid areas that have been passed; For each raster region in the raster sequence that has passed through, record the corresponding smoke degradation confidence level, thermal risk status, and passage width. When the smoke degradation confidence of three consecutive grid regions in the grid sequence is lower than the preset return confidence threshold, or when any grid region in the three consecutive grid regions is written into the thermal risk layer, the three consecutive grid regions are marked as return restricted regions. In the subsequent obstacle avoidance navigation path planning, a continuous passable grid sequence between the current position of the fire-fighting robot and the safe exit position that does not pass through the restricted area for retreat is used as the safe retreat corridor.

[0015] The smoke environment perception and obstacle avoidance navigation system based on firefighting robots includes: the firefighting robot body, a multi-source perception module, a smoke degradation credibility calculation module, a layered environment map construction module, an active detection and control module, a map layer attribute correction module, an obstacle avoidance navigation module, and a return and withdrawal safety corridor maintenance module. The multi-source sensing module is installed on the fire-fighting robot body to collect smoke concentration data, visible light image data, infrared thermal image data, lidar point cloud data, lidar echo intensity data, millimeter-wave radar echo data, and robot pose data. The input end of the smoke degradation confidence calculation module is connected to the multi-source sensing module. The smoke degradation confidence calculation module is used to calculate the smoke degradation confidence of multiple directional sectors around the fire-fighting robot based on the multi-source sensing data. The input terminals of the layered environment map construction module are connected to the multi-source sensing module and the smoke degradation credibility calculation module, respectively. The layered environment map construction module is used to construct a layered environment map based on the smoke degradation credibility and multi-source sensing data. The layered environment map includes a physical obstacle layer, a smoke masking layer and a thermal risk layer. The input end of the active detection control module is connected to the layered environment map construction module, and the control output end of the active detection control module is connected to the walking control unit of the fire-fighting robot body. It is used to control the fire-fighting robot body to perform active detection actions when there is a target grid area with smoke degradation confidence lower than the preset detection confidence threshold in front of the obstacle avoidance navigation path, and the target grid area is located in the smoke obscuring layer and has not been written into the physical obstacle layer. The input of the map layer attribute correction module is connected to the multi-source sensing module and the active detection control module respectively. It is used to receive the multi-frame sensing results re-collected by the multi-source sensing module after active detection, and correct the map layer attributes of the corresponding target grid area according to the multi-frame sensing results. The input terminals of the obstacle avoidance and navigation module are connected to the layered environment map construction module and the map layer attribute correction module, respectively. The module is used to generate a comprehensive passage cost map based on the corrected layered environment map and plan the obstacle avoidance and navigation path of the fire-fighting robot. The input terminals of the retreat safety corridor maintenance module are connected to the obstacle avoidance navigation module and the map layer attribute correction module, respectively. It is used to record the smoke degradation confidence level and thermal risk status of the traversed areas, and maintain the retreat safety corridor based on the smoke degradation confidence level and thermal risk status of the traversed areas.

[0016] In summary, the present invention has the following main beneficial effects: By collecting smoke concentration data, visible light image data, infrared thermal image data, lidar point cloud data, lidar echo intensity data, millimeter-wave radar echo data, and robot pose data, and further calculating the smoke degradation reliability of multiple directional sectors around the firefighting robot, the firefighting robot does not directly rely on a single sensor to judge obstacles in dense smoke environments. Instead, it first judges the reliability of the perception data in each directional sector, thereby reducing the misjudgment of obstacle recognition results caused by smoke scattering, image degradation, and abnormal laser echoes.

[0017] By constructing a layered environmental map that includes a physical obstacle layer, a smoke-covered layer, and a thermal risk layer, and generating a comprehensive passage cost map based on the layered environmental map, physical obstacles, smoke-covered areas, and high-temperature risk areas participate in path planning at different map layers. This avoids equating smoke-covered areas directly with physical obstacles or ordinary passable areas, enabling firefighting robots to more reasonably distinguish detour areas, speed-reducing detection areas, and thermal risk avoidance areas in complex smoke environments.

[0018] By controlling the firefighting robot to perform active detection actions when there is a low-confidence smoke-covered area ahead of the obstacle avoidance navigation path, and correcting the map layer attributes based on the newly acquired multi-frame lidar point cloud data, millimeter-wave radar echo data, and infrared thermal image data, while maintaining a safe retreat corridor during the journey, the navigation path is updated only after the nature of the obstacle is confirmed. This can reduce the problem of the firefighting robot being unable to reliably retreat after entering the fire scene due to changes in smoke, increased thermal risk, or insufficient perception of the retreat path. Attached Figure Description

[0019] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1 refer to Figure 1 A smoke environment perception and obstacle avoidance navigation method based on firefighting robots includes the following steps: S1: When the firefighting robot moves within the target work area, it collects multi-source sensing data, including smoke concentration data, visible light image data, infrared thermal image data, lidar point cloud data, lidar echo intensity data, millimeter-wave radar echo data, and robot pose data. S2: Based on smoke concentration data, edge preservation in visible light image data, thermal gradient in infrared thermal image data, attenuation of lidar echo intensity data, drift of lidar point cloud data, and stability of millimeter-wave radar echo data, calculate the smoke degradation reliability of multiple directional sectors around the fire-fighting robot. S3: Based on the reliability of smoke degradation and multi-source perception data, construct a layered environment map around the fire-fighting robot. The layered environment map includes a physical obstacle layer, a smoke obscuring layer, and a thermal risk layer. S4: Generate a comprehensive passage cost map based on the physical obstacle layer, smoke cover layer, thermal risk layer and robot pose data, and plan the obstacle avoidance navigation path of the fire-fighting robot based on the comprehensive passage cost map; S5: When there is a target grid area with smoke degradation confidence level lower than the preset detection confidence level threshold ahead of the obstacle avoidance navigation path, and the target grid area is located in the smoke obscuring layer and has not been written into the physical obstacle layer, control the fire-fighting robot to perform active detection actions; the active detection actions include reducing the travel speed, adjusting the observation angle, and conducting short-distance probing when the grid area corresponding to the braking distance in front of the fire-fighting robot has not been written into the physical obstacle layer. S6: After performing active detection, re-acquire the lidar point cloud data, lidar echo intensity data, millimeter-wave radar echo data and infrared thermal image data corresponding to the target grid area, and correct the map layer attributes of the target grid area based on the re-acquired multi-frame perception results. S7: Regenerate the comprehensive access cost map based on the revised layered environment map and update the obstacle avoidance navigation path of the fire-fighting robot; S8: Record the smoke degradation confidence level and thermal risk status of the areas already passed during the firefighting robot's movement, and maintain a safe retreat corridor based on the smoke degradation confidence level and thermal risk status of the areas already passed. The safe retreat corridor serves as a constraint condition for subsequent obstacle avoidance navigation path planning.

[0022] This application applies to firefighting robots performing environmental perception, obstacle avoidance navigation, and return path maintenance in target work areas such as inside buildings, underground spaces, warehouse spaces, and factory passages where there are smoke obstructions, localized high temperatures, and uncertain obstacles.

[0023] The firefighting robot described in this embodiment includes a robot body, a walking control unit, a controller, a communication unit, and a multi-source sensing module mounted on the robot body. The multi-source sensing module includes a smoke concentration sensor, a visible light camera, an infrared thermal imager, a lidar, a millimeter-wave radar, and a pose detection component. The pose detection component includes an inertial measurement unit and an odometer. The smoke concentration sensor is used to collect smoke concentration data; the visible light camera is used to collect visible light image data; the infrared thermal imager is used to collect infrared thermal image data; the lidar is used to collect lidar point cloud data and lidar echo intensity data; the millimeter-wave radar is used to collect millimeter-wave radar echo data; and the pose detection component is used to collect the robot's pose data.

[0024] The data collected by the aforementioned sensors are timestamped by the controller and converted to the same robot coordinate system based on the robot's pose data. The robot coordinate system has its origin at the geometric center of the firefighting robot, its forward direction as the current travel direction, and its lateral direction perpendicular to the current travel direction as the horizontal direction. This unified coordinate system processing ensures that LiDAR point cloud data, millimeter-wave radar echo data, infrared thermal image data, and visible light image data can be mapped to the same spatial area, preventing inconsistencies in map layer writing positions due to different sensor installation locations.

[0025] Before the firefighting robot enters the target work area, the controller first completes the calibration parameter configuration. Calibration parameters include the number of edge pixels in a clear calibration state, calibration echo intensity, preset edge preservation threshold, preset detection confidence threshold, preset drift threshold, preset passage temperature threshold, preset thermal gradient threshold, preset detection angle, preset detection speed, preset detection distance, preset retreat confidence threshold, and braking distance. All of these parameters are derived from the firefighting robot's own sensor calibration, chassis braking performance testing, and product parameters of the firefighting robot itself and its components; no temporary on-site experience values ​​are used.

[0026] The number of edge pixels in the clear calibration state is obtained by edge detection after the visible light camera acquires images of the target area in a smoke-free or low-smoke environment. The calibration echo intensity is obtained by the lidar sampling multiple times at a fixed distance on a wall, door frame, or standard reflector in a calibration environment without significant smoke obstruction. The preset edge preservation threshold is determined by the number of edge pixels in images acquired by the visible light camera in both clear and smoke-simulated environments. When the edge preservation condition in the smoke-simulated environment drops to the point where door frames, corners, or device boundaries cannot be continuously identified, the corresponding edge preservation item is used as the preset edge preservation threshold.

[0027] The preset detection confidence threshold is determined based on the false detection rate of the firefighting robot in a smoke simulation calibration environment. When the smoke degradation confidence level of a certain sector is lower than this threshold, it indicates that the consistency between the lidar point cloud data, millimeter-wave radar echo data, and image data in that sector is insufficient to directly confirm a physical obstacle, and the controller triggers an active detection action. The preset drift threshold is determined based on the grid width and lidar ranging error. The preset drift threshold is not less than the maximum ranging error of the lidar within the calibration distance, and not greater than one grid width. This setting can eliminate the inherent ranging error of the lidar and avoid misjudging smoke scattering echoes with significant positional changes as stable obstacles.

[0028] The preset passage temperature threshold is determined based on the heat resistance rating of the fire-fighting robot's shell, the upper limit of the operating temperature of its internal electrical components, and the allowable temperature of its cables and seals. Specifically, the controller selects the lowest value among the above temperature limits and deducts a safety margin as the preset passage temperature threshold. The safety margin is determined based on the fire-fighting robot's operational safety factor. The preset thermal gradient threshold is determined based on the amount of temperature change the fire-fighting robot can withstand per unit travel distance, and is used to determine whether the fire-fighting robot will enter an area with excessively rapid temperature changes.

[0029] The preset detection angle is determined based on the firefighting robot's chassis steering capability, the LiDAR scanning field of view, and the effective detection range of the millimeter-wave radar. The preset detection speed is determined based on the firefighting robot's minimum stable travel speed, sensor refresh frequency, and controller processing cycle. The preset detection distance is determined based on the firefighting robot's braking distance, sensor near-range blind zone, and minimum safe passage distance. The preset retreat confidence threshold is determined based on the minimum smoke degradation confidence required for the firefighting robot to maintain positioning continuity and obstacle recognition continuity within the traversed path.

[0030] When the firefighting robot moves within the target work area, it shall perform the following steps.

[0031] S1. When the firefighting robot moves within the target work area, it collects multi-source sensing data. This multi-source sensing data includes smoke concentration data, visible light image data, infrared thermal image data, lidar point cloud data, lidar echo intensity data, millimeter-wave radar echo data, and robot pose data. The controller acquires this data according to a sampling period and binds the data within the same sampling period to the firefighting robot's current pose. For data with time delays, the controller synchronizes it based on timestamps, ensuring that data from different sensors corresponds to the robot's pose at the same or adjacent moments.

[0032] S2. Calculate the smoke degradation reliability of multiple directional sectors around the firefighting robot based on multi-source sensing data. Using the current direction of travel of the firefighting robot as the reference direction, the 360-degree area around the firefighting robot is divided into eight directional sectors, each corresponding to a 45-degree angle range. The controller statistically analyzes the smoke concentration data, visible light image data, infrared thermal image data, lidar point cloud data, lidar echo intensity data, and millimeter-wave radar echo data within each directional sector.

[0033] For the Each directional sector is used by the controller to convert the smoke concentration data within that sector into a smoke concentration attenuation term. The ratio of the number of edge pixels in the visible light image data to the number of edge pixels in the calibrated sharp state is used as the edge retention term. The temperature difference between adjacent sampling areas in the infrared thermal image data is normalized and used as the thermal gradient term. The decrease in lidar echo intensity data relative to the calibrated echo intensity is used as the laser echo attenuation term. The point cloud position offset at the same spatial location in multiple consecutive frames of LiDAR point cloud data is normalized and used as the point cloud drift term. The proportion of echo occurrences at the same spatial location in multiple consecutive frames of millimeter-wave radar echo data is used as a millimeter-wave stability term. .

[0034] Smoke concentration attenuation term Thermal gradient term Laser echo attenuation term and point cloud drift item Normalize the value according to the corresponding sensor range, the maximum calibrated change, or the calibration upper limit stored in the controller, so that its value is between 0 and 1; when the calculated value exceeds the calibration upper limit, the normalization result is set to 1. Edge-preserving terms. and millimeter wave stability Calculated according to the actual proportion, the value ranges from 0 to 1.

[0035] No. The reliability of smoke degradation in each directional sector is determined by the following formula: ; in, Indicates the first The confidence level of smoke degradation in each directional sector ranges from 0 to 1. The larger the value, the more reliable the perception result in that directional sector. This represents the term indicating the decrease in smoke concentration. Indicates edge-reserved items; Represents the thermal gradient term; This indicates the laser echo attenuation term; Indicates point cloud drift term; Represents the millimeter-wave stability term; to These represent fixed weights, corresponding to the smoke concentration attenuation term, edge preservation term, thermal gradient term, laser echo attenuation term, point cloud drift term, and millimeter-wave stabilization term, respectively. to All of them are non-negative numbers, and the sum of the six is ​​1.

[0036] The fixed weights are obtained by normalizing the reliability coefficients of each sensing item in the calibration environment. The reliability coefficients are obtained by repeated sampling in smoke-free, low-smoke, and high-smoke environments by the fire-fighting robot. Specifically, they are the ratio of the number of times the corresponding sensing item correctly reflects the target state under known calibration conditions to the total number of samples. The fixed weights are written into the controller before the fire-fighting robot enters the target work area and are not manually modified temporarily within the target work area.

[0037] The aforementioned smoke degradation reliability is not used to directly determine the existence of obstacles, but rather to evaluate the reliability of multi-source sensing data within that azimuth sector under smoke and thermal disturbance conditions. This processing avoids directly identifying smoke scattering echoes from lidar as physical obstacles, and also prevents the complete loss of access judgment criteria when visible light images degrade.

[0038] S3. Based on smoke degradation reliability and multi-source sensing data, construct a layered environmental map around the firefighting robot. The controller divides the area around the firefighting robot into grid regions. The grid size is determined based on the width of the firefighting robot body, minimum passage clearance, sensor spatial resolution, and controller computing power. Each grid region corresponds to a spatial unit within the target operating area. The layered environmental map includes a physical obstacle layer, a smoke masking layer, and a thermal risk layer.

[0039] The physical obstacle layer is used to record obstacle regions with stable geometric boundaries. For any grid region, when the LiDAR point cloud data within that grid region forms a continuous geometric boundary in three consecutive frames, the maximum positional offset of the continuous geometric boundary in the three consecutive frames does not exceed one grid width, and the millimeter-wave radar echo data shows stable echoes in the grid region for three consecutive frames, the controller writes that grid region into the physical obstacle layer. A continuous geometric boundary refers to the LiDAR point cloud forming a continuous distribution at adjacent scanning angles, corresponding to the boundary of a wall, door frame, equipment, collapsed object, step, or other physical structure. A stable echo means that the millimeter-wave radar detects an echo within the same grid region in three consecutive frames, and the echo position does not cross the grid region.

[0040] The smoke masking layer is used to record areas where perception credibility is reduced due to smoke but have not yet been confirmed as physical obstacles. For any grid area, when the edge preservation value in the visible light image data corresponding to the grid area is lower than a preset edge preservation threshold, there are near-range patch echoes in the lidar point cloud data, and the millimeter-wave radar echo data does not form three consecutive stable echoes within the grid area, the controller writes the grid area into the smoke masking layer. Near-range patch echoes refer to echo points present at multiple adjacent scanning angles within a preset distance range in front of the fire-fighting robot by the lidar, but the spatial positions of these echo points cannot form a continuous geometric boundary in three consecutive frames, and the positional offset between adjacent frames is greater than a preset drift threshold. The preset distance range is determined based on the effective near-range detection range of the lidar and the braking distance of the fire-fighting robot.

[0041] The thermal risk layer records areas where the highest temperature exceeds a preset passage temperature threshold or where the temperature difference between adjacent grid areas exceeds a preset thermal gradient threshold. For any grid area, when the highest temperature in the corresponding infrared thermal image data exceeds the preset passage temperature threshold, or when the temperature difference between the grid area and adjacent grid areas exceeds the preset thermal gradient threshold, the controller writes the grid area into the thermal risk layer. The thermal risk layer expresses the restriction of high-temperature areas on the passage of the firefighting robot. This area may not have physical obstacles, but it is still not treated as a normal passable area.

[0042] S4. Based on the physical obstacle layer, smoke cover layer, thermal risk layer, and robot pose data, a comprehensive passage cost map is generated, and the obstacle avoidance navigation path of the firefighting robot is planned based on the comprehensive passage cost map. When generating the comprehensive passage cost map, the controller sets grid areas in the physical obstacle layer as impassable markers, grid areas in the smoke cover layer as deceleration detection markers, and grid areas in the thermal risk layer as thermal risk avoidance markers. For grid areas not written into the physical obstacle layer, smoke cover layer, and thermal risk layer, the controller determines the corresponding passage cost based on the smoke degradation confidence level of the azimuth sector to which the grid area belongs; the lower the smoke degradation confidence level, the higher the corresponding passage cost. For grid areas in the physical obstacle layer, the controller directly sets them as impassable markers and does not consider them as candidate grids for ordinary paths.

[0043] When planning obstacle avoidance navigation paths, the controller does not use the shortest distance as the sole objective. Instead, it determines the obstacle avoidance navigation path while avoiding physical obstacle layers, minimizing thermal risk layer crossings, implementing deceleration detection in smoke-covered layers, and maintaining the continuity of the safe retreat corridor. Therefore, smoke-covered areas are not simply equated with physical obstacles, nor are they simply regarded as ordinary passable areas. Instead, they participate in navigation control as an independent map state that requires further exploration.

[0044] S5. When there is a target grid area ahead of the obstacle avoidance navigation path with a smoke degradation confidence level lower than the preset detection confidence level threshold, and the target grid area is located in the smoke obscuration layer and has not been written into the physical obstacle layer, the controller controls the firefighting robot to perform an active detection action. The active detection action is not a normal deceleration obstacle avoidance, but rather, when the target grid area has not yet been confirmed as a physical obstacle, it re-acquires multi-source perception data by changing the observation posture and short-distance probing.

[0045] The active detection actions include: controlling the firefighting robot to stop its straight-line movement before entering the target grid area and recording its current pose as the detection start pose; controlling the firefighting robot to reduce its speed to a preset detection speed; controlling the firefighting robot to turn to a first detection direction and a second detection direction, the first detection direction being the direction formed by a preset detection angle to the left relative to the detection start pose, and the second detection direction being the direction formed by a preset detection angle to the right relative to the detection start pose; acquiring three frames of lidar point cloud data, three frames of lidar echo intensity data, three frames of millimeter-wave radar echo data, and three frames of infrared thermal image data in the first and second detection directions, respectively; when the grid area corresponding to the braking distance in front of the firefighting robot has not been written into the physical obstacle layer, controlling the firefighting robot to move forward a preset detection distance along the original direction of travel and then stop, and acquiring the corresponding multi-source sensing data again.

[0046] S6. After performing the active detection action, re-acquire the lidar point cloud data, lidar echo intensity data, millimeter-wave radar echo data and infrared thermal image data corresponding to the target grid area, and correct the map layer attributes of the target grid area based on the re-acquired multi-frame perception results.

[0047] When the reacquired lidar point cloud data forms a continuous geometric boundary in three consecutive frames, and the reacquired millimeter-wave radar echo data forms a stable echo in three consecutive frames, and the grid area where the stable echo is located coincides with the grid area occupied by the continuous geometric boundary, the controller will correct the corresponding overlapping grid area from a smoke masking layer to a solid obstacle layer. This indicates that there is a solid obstacle with a stable spatial location in the area, rather than a temporary echo caused by smoke scattering.

[0048] When the positional drift of the reacquired LiDAR point cloud data exceeds a preset drift threshold across three consecutive frames, and the reacquired millimeter-wave radar echo data fails to form a stable echo across three consecutive frames, the controller keeps the target grid area within the smoke masking layer. This indicates that the LiDAR echo in this area is more likely caused by smoke, water mist, or high-temperature airflow, and cannot be directly written into the solid obstacle layer.

[0049] When the reacquired infrared thermal image data meets the conditions for writing to the thermal risk layer, the controller writes the target grid area to the thermal risk layer. The conditions for writing to the thermal risk layer are that the highest temperature value of the target grid area is greater than a preset passage temperature threshold, or the temperature difference between the target grid area and adjacent grid areas is greater than a preset thermal gradient threshold.

[0050] S7. Regenerate the comprehensive passage cost map based on the revised layered environment map and update the obstacle avoidance navigation path of the firefighting robot. If the target grid area is corrected to a physical obstacle layer, the controller excludes the area from the passable area and replans the obstacle avoidance navigation path. If the target grid area remains in the smoke cover layer and does not meet the thermal risk layer writing conditions, the controller allows the area as a candidate passable area, but limits the driving speed through the area to a preset detection speed or a safe speed lower than the normal cruising speed. If the target grid area is written into the thermal risk layer, the controller increases the passage cost of the area; when the thermal risk level reaches the prohibited passage conditions stored by the controller, the controller sets it as an impassable area.

[0051] S8. During the movement of the firefighting robot, record the smoke degradation confidence level and thermal risk status of the areas it has passed through, and maintain the safe retreat corridor based on the smoke degradation confidence level and thermal risk status of the areas it has passed through. The controller records the grid areas that the firefighting robot has passed through in the actual time sequence, forming a passed grid sequence, and records the corresponding smoke degradation confidence level, thermal risk status, and passage width for each grid area in the passed grid sequence.

[0052] The passage width is determined by the minimum lateral distance between the boundaries of the physical obstacle layers on both sides of the current grid area. When one or both boundaries have not yet been identified as physical obstacles, the passage width is calculated using the boundary of the smoke-masking layer as a temporary boundary, and is updated according to the corrected map layer attributes after active detection. If the passage width is less than the sum of the width of the fire-fighting robot's body and the preset safety margin, then the grid area is not considered part of the retreat safety corridor.

[0053] When the smoke degradation confidence level of three consecutive grid areas in a grid sequence is lower than the preset return confidence level threshold, or when any of the three consecutive grid areas is written into the thermal risk layer, the controller marks these three consecutive grid areas as a return restricted area. In subsequent obstacle avoidance navigation path planning, a continuous passable grid sequence between the fire robot's current position and the safety exit position that does not pass through the return restricted area is used as the return safety corridor. The return safety corridor is continuously updated as the fire robot moves, not simply recording the already traveled trajectory, but combining smoke degradation confidence level, thermal risk status, and passage width to determine whether the fire robot can return safely.

[0054] This embodiment also provides a smoke environment perception and obstacle avoidance navigation system based on a firefighting robot. The system includes a firefighting robot body, a multi-source perception module, a smoke degradation reliability calculation module, a layered environment map construction module, an active detection and control module, a map layer attribute correction module, an obstacle avoidance navigation module, and a return and withdrawal safety corridor maintenance module.

[0055] A multi-source sensing module is installed on the firefighting robot to collect smoke concentration data, visible light image data, infrared thermal image data, lidar point cloud data, lidar echo intensity data, millimeter-wave radar echo data, and robot pose data. The input of the smoke degradation confidence calculation module is connected to the multi-source sensing module to calculate the smoke degradation confidence of multiple directional sectors around the firefighting robot based on the multi-source sensing data.

[0056] The input terminals of the layered environment map construction module are connected to the multi-source sensing module and the smoke degradation credibility calculation module, respectively, to construct a layered environment map based on smoke degradation credibility and multi-source sensing data. The input terminal of the active detection control module is connected to the layered environment map construction module, and the control output terminal of the active detection control module is connected to the walking control unit of the fire-fighting robot. When there is a target grid area with smoke degradation credibility lower than a preset detection credibility threshold ahead of the obstacle avoidance navigation path, and the target grid area is located in the smoke obscuration layer and has not been written into the physical obstacle layer, the active detection action is controlled to be performed by the fire-fighting robot.

[0057] The input terminals of the map layer attribute correction module are connected to the multi-source perception module and the active detection control module, respectively. It receives multi-frame perception results re-collected by the multi-source perception module after active detection and corrects the map layer attributes of the corresponding target grid area based on these results. The input terminals of the obstacle avoidance and navigation module are connected to the layered environment map construction module and the map layer attribute correction module, respectively. It generates a comprehensive passage cost map based on the corrected layered environment map and plans the obstacle avoidance and navigation path for the firefighting robot. The input terminals of the return safety corridor maintenance module are connected to the obstacle avoidance and navigation module and the map layer attribute correction module, respectively. It records the smoke degradation reliability and thermal risk status of traversed areas and maintains the return safety corridor based on these parameters.

[0058] The aforementioned modules can be implemented by the processor, memory, and control program in the fire-fighting robot controller, or by an embedded industrial computer, edge computing unit, or multiple control boards working together. Data transmission between modules occurs via the fire-fighting robot's internal bus, data interface, or the controller's internal data channel. Data collected by the multi-source sensing module first enters the smoke degradation reliability calculation module and the layered environment map construction module. The output of the layered environment map construction module is simultaneously used by the active detection control module and the obstacle avoidance navigation module. After the active detection control module drives the fire-fighting robot to change its observation posture, the multi-source sensing module re-collects data from the target grid area. The map layer attribute correction module corrects the layered environment map based on the re-collected data, the obstacle avoidance navigation module updates the path based on the corrected layered environment map, and the return safety corridor maintenance module continuously updates the return safety corridor throughout the entire journey.

[0059] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A smoke environment perception and obstacle avoidance navigation method based on a firefighting robot, characterized in that, Includes the following steps: S1: When the firefighting robot moves within the target work area, it collects multi-source sensing data, including smoke concentration data, visible light image data, infrared thermal image data, lidar point cloud data, lidar echo intensity data, millimeter-wave radar echo data, and robot pose data. S2: Based on smoke concentration data, edge preservation in visible light image data, thermal gradient in infrared thermal image data, attenuation of lidar echo intensity data, drift of lidar point cloud data, and stability of millimeter-wave radar echo data, calculate the smoke degradation reliability of multiple directional sectors around the fire-fighting robot. S3: Based on the reliability of smoke degradation and multi-source perception data, construct a layered environment map around the fire-fighting robot. The layered environment map includes a physical obstacle layer, a smoke obscuring layer, and a thermal risk layer. S4: Generate a comprehensive passage cost map based on the physical obstacle layer, smoke cover layer, thermal risk layer and robot pose data, and plan the obstacle avoidance navigation path of the fire-fighting robot based on the comprehensive passage cost map; S5: When there is a target grid area with smoke degradation confidence level lower than the preset detection confidence level threshold ahead of the obstacle avoidance navigation path, and the target grid area is located in the smoke obscuring layer and has not been written into the physical obstacle layer, control the fire-fighting robot to perform active detection actions; the active detection actions include reducing the travel speed, adjusting the observation angle, and conducting short-distance probing when the grid area corresponding to the braking distance in front of the fire-fighting robot has not been written into the physical obstacle layer. S6: After performing active detection, re-acquire the lidar point cloud data, lidar echo intensity data, millimeter-wave radar echo data and infrared thermal image data corresponding to the target grid area, and correct the map layer attributes of the target grid area based on the re-acquired multi-frame perception results. S7: Regenerate the comprehensive access cost map based on the revised layered environment map and update the obstacle avoidance navigation path of the fire-fighting robot; S8: Record the smoke degradation confidence level and thermal risk status of the areas already passed during the firefighting robot's movement, and maintain a safe retreat corridor based on the smoke degradation confidence level and thermal risk status of the areas already passed. The safe retreat corridor serves as a constraint condition for subsequent obstacle avoidance navigation path planning.

2. The smoke environment perception and obstacle avoidance navigation method based on a fire-fighting robot according to claim 1, characterized in that, In step S2, the 360-degree area around the fire robot is divided into eight directional sectors, with the current direction of travel of the fire robot as the reference direction. Each directional sector corresponds to a 45-degree angle range. Statistical analysis was performed on smoke concentration data, visible light image data, infrared thermal image data, lidar point cloud data, lidar echo intensity data, and millimeter-wave radar echo data for each azimuth sector. Specifically, the smoke concentration data in the corresponding azimuth sector is converted into a smoke concentration attenuation term; the ratio of the number of edge pixels in the visible light image data to the number of edge pixels in the calibrated clear state is used as an edge retention term; the temperature difference between adjacent sampling areas in the infrared thermal image data is used as a thermal gradient term; the decrease ratio of the lidar echo intensity data relative to the calibrated echo intensity is used as a lidar echo attenuation term; the point cloud position offset at the same spatial location in multiple consecutive frames of lidar point cloud data is used as a point cloud drift term; and the proportion of echo occurrences at the same spatial location in multiple consecutive frames of millimeter-wave radar echo data is used as a millimeter-wave stability term. After normalizing the smoke concentration attenuation term, edge preservation term, thermal gradient term, laser echo attenuation term, point cloud drift term, and millimeter wave stability term, the smoke degradation confidence of the corresponding directional sector is obtained by weighted summation according to the fixed weights stored in the fire robot controller.

3. The smoke environment perception and obstacle avoidance navigation method based on a fire-fighting robot according to claim 2, characterized in that, In step S3, when constructing the layered environment map, the area around the fire-fighting robot is divided into grid areas; The physical obstacle layer is used to record obstacle areas with stable geometric boundaries, the smoke obscuring layer is used to record areas where the perception credibility is reduced due to smoke but has not yet been confirmed as physical obstacles, and the thermal risk layer is used to record areas where the highest temperature value is greater than a preset passage temperature threshold or the temperature difference between adjacent grid areas is greater than a preset thermal gradient threshold.

4. The smoke environment perception and obstacle avoidance navigation method based on a fire-fighting robot according to claim 3, characterized in that, When constructing the physical obstacle layer, for any grid area, if the LiDAR point cloud data within the grid area forms a continuous geometric boundary in three consecutive frames, the maximum positional offset of the continuous geometric boundary in the three consecutive frames does not exceed one grid width, and the millimeter-wave radar echo data has a stable echo in the grid area for three consecutive frames, then the grid area is written into the physical obstacle layer.

5. The smoke environment perception and obstacle avoidance navigation method based on a fire-fighting robot according to claim 4, characterized in that, When constructing the smoke masking layer, for any grid area, if the edge retention item in the visible light image data corresponding to the grid area is lower than the preset edge retention threshold, there is a near-range sheet echo in the lidar point cloud data, and the millimeter-wave radar echo data does not form three consecutive stable echoes in the grid area, then the grid area is written into the smoke masking layer. The preset edge preservation threshold and the near-range sheet echo determination threshold are both calibration parameters stored in the controller before the fire-fighting robot enters the target work area.

6. The smoke environment perception and obstacle avoidance navigation method based on a fire-fighting robot according to claim 5, characterized in that, When constructing the thermal risk layer, for any grid area, if the highest temperature value in the infrared thermal image data corresponding to the grid area is greater than the preset passage temperature threshold, or the temperature difference between the grid area and the adjacent grid area is greater than the preset thermal gradient threshold, the grid area is written into the thermal risk layer. The preset passage temperature threshold is determined based on the heat resistance rating of the fire robot's shell and the upper limit of the operating temperature of its internal electrical components, while the preset thermal gradient threshold is determined based on the amount of temperature change that the fire robot can withstand per unit travel distance.

7. The smoke environment perception and obstacle avoidance navigation method based on a fire-fighting robot according to claim 6, characterized in that, In step S4, when generating the comprehensive passage cost map, the grid area in the physical obstacle layer is set as an impassable marker, the grid area in the smoke obscuring layer is set as a deceleration detection marker, and the grid area in the thermal risk layer is set as a thermal risk avoidance marker. For grid regions that are not written into the entity obstacle layer, smoke masking layer, and thermal risk layer, the corresponding passage cost is determined based on the smoke degradation confidence level of the directional sector to which the grid region belongs. The lower the smoke degradation confidence level, the higher the corresponding passage cost.

8. The smoke environment perception and obstacle avoidance navigation method based on a fire-fighting robot according to claim 7, characterized in that, In step S5, the active detection action includes: controlling the fire-fighting robot to stop moving in a straight line before entering the target grid area, and recording the current pose as the detection start pose; Control the firefighting robot to reduce its preset detection speed; The firefighting robot is controlled to turn to the first detection direction and the second detection direction respectively. The first detection direction is the direction formed by turning to the left by a preset detection angle relative to the initial detection posture, and the second detection direction is the direction formed by turning to the right by a preset detection angle relative to the initial detection posture. Three frames of lidar point cloud data, three frames of lidar echo intensity data, three frames of millimeter-wave radar echo data, and three frames of infrared thermal image data were collected in the first and second detection directions, respectively. When the grid area corresponding to the braking distance in front of the fire-fighting robot is not written into the physical obstacle layer, the fire-fighting robot is controlled to move forward along the original direction of travel for a preset detection distance and then stop, and the corresponding multi-source perception data is collected again.

9. The smoke environment perception and obstacle avoidance navigation method based on a fire-fighting robot according to claim 8, characterized in that, In step S6, correcting the map layer attributes of the target grid area includes: when the re-acquired lidar point cloud data forms a continuous geometric boundary in three consecutive frames, and the re-acquired millimeter-wave radar echo data forms a stable echo in three consecutive frames, and the grid area where the stable echo is located overlaps with the grid area occupied by the continuous geometric boundary, the corresponding overlapping grid area is corrected from the smoke masking layer to the solid obstacle layer. When the position drift of the re-acquired lidar point cloud data exceeds the preset drift threshold in three consecutive frames, and the re-acquired millimeter-wave radar echo data does not form a stable echo in three consecutive frames, keep the target grid area in the smoke cover layer. When the reacquired infrared thermal image data meets the conditions for writing to the thermal risk layer, the target grid area is written to the thermal risk layer. In step S8, maintaining the safe corridor for retreat includes: recording the grid areas that have been passed according to the actual time sequence of the fire-fighting robot, forming a sequence of grid areas that have been passed; For each raster region in the raster sequence that has passed through, record the corresponding smoke degradation confidence level, thermal risk status, and passage width. When the smoke degradation confidence of three consecutive grid regions in the grid sequence is lower than the preset return confidence threshold, or when any grid region in the three consecutive grid regions is written into the thermal risk layer, the three consecutive grid regions are marked as return restricted regions. In the subsequent obstacle avoidance navigation path planning, a continuous passable grid sequence between the current position of the fire-fighting robot and the safe exit position that does not pass through the restricted area for retreat is used as the safe retreat corridor.

10. A smoke environment perception and obstacle avoidance navigation system based on a fire-fighting robot, applicable to the smoke environment perception and obstacle avoidance navigation method based on a fire-fighting robot as described in any one of claims 1-9, characterized in that, include: The firefighting robot body, multi-source perception module, smoke degradation reliability calculation module, layered environment map construction module, active detection and control module, map layer attribute correction module, obstacle avoidance and navigation module, and return and withdrawal safety corridor maintenance module; The multi-source sensing module is installed on the fire-fighting robot body to collect smoke concentration data, visible light image data, infrared thermal image data, lidar point cloud data, lidar echo intensity data, millimeter-wave radar echo data, and robot pose data. The input end of the smoke degradation confidence calculation module is connected to the multi-source sensing module. The smoke degradation confidence calculation module is used to calculate the smoke degradation confidence of multiple directional sectors around the fire-fighting robot based on the multi-source sensing data. The input terminals of the layered environment map construction module are connected to the multi-source sensing module and the smoke degradation credibility calculation module, respectively. The layered environment map construction module is used to construct a layered environment map based on the smoke degradation credibility and multi-source sensing data. The layered environment map includes a physical obstacle layer, a smoke masking layer and a thermal risk layer. The input end of the active detection control module is connected to the layered environment map construction module, and the control output end of the active detection control module is connected to the walking control unit of the fire-fighting robot body. It is used to control the fire-fighting robot body to perform active detection actions when there is a target grid area with smoke degradation confidence lower than the preset detection confidence threshold in front of the obstacle avoidance navigation path, and the target grid area is located in the smoke obscuring layer and has not been written into the physical obstacle layer. The input of the map layer attribute correction module is connected to the multi-source sensing module and the active detection control module respectively. It is used to receive the multi-frame sensing results re-collected by the multi-source sensing module after active detection, and correct the map layer attributes of the corresponding target grid area according to the multi-frame sensing results. The input terminals of the obstacle avoidance and navigation module are connected to the layered environment map construction module and the map layer attribute correction module, respectively. The module is used to generate a comprehensive passage cost map based on the corrected layered environment map and plan the obstacle avoidance and navigation path of the fire-fighting robot. The input terminals of the retreat safety corridor maintenance module are connected to the obstacle avoidance navigation module and the map layer attribute correction module, respectively. It is used to record the smoke degradation confidence level and thermal risk status of the traversed areas, and maintain the retreat safety corridor based on the smoke degradation confidence level and thermal risk status of the traversed areas.

Images