Fire scene map construction method and system integrating lidar / vision / UWB

Abstract

The present invention relates to a laser radar / vision / UWB-integrated fire scene map construction method and system. The laser radar / vision / UWB-integrated fire scene map construction method provided by the present invention utilizes UWB positioning to accurately construct an initial fire scene map, which is then mutually corrected with a self-constructed map using laser radar information / video data. This map is then integrated into a graph optimization framework and loopback detection is added. This enables the timely construction of a two-dimensional fire scene map even without an indoor map, effectively overcoming the limitations of existing technologies for constructing internal fire scene maps and improving the accuracy of internal fire scene map construction.

Description

Technical Field

[0001] The present invention relates to the field of map construction technology, and in particular to a method and system for constructing a fire scene map integrating lidar / vision / UWB. Background Art

[0002] When urban fires occur, firefighters often need to conduct in-depth firefighting and rescue operations, putting their own safety at risk. Fire scene maps are often needed to locate firefighters in distress and provide assistance. Currently, most existing buildings lack access to indoor maps in advance of a fire, leaving command centers without a reference map. Furthermore, most current firefighter positioning systems only provide positioning capabilities, without map display capabilities. This significantly limits the ability to interpret the location of firefighters in distress. Wireless communication alone between the command center and firefighters makes it difficult to develop effective rescue plans, which can easily result in casualties.

[0003] Most existing firefighter positioning systems primarily locate firefighters, lacking a map display of the fire scene, which limits interpretation of firefighter positions. Some positioning systems claim to have maps simply by pre-deploying positioning infrastructure and mapping sensors within numerous buildings to collect data and then pre-create them offline. Due to the random and sudden nature of building fires, these systems cannot truly self-build maps in real time, making them less practical. Furthermore, existing SLAM (Simultaneous Local Area Mapping) mapping systems, when applied to fire scenes, suffer from limitations such as a limited number of sensors and a lack of loopback optimization, resulting in inaccurate maps.

[0004] To solve the above problems, firefighters are required to wear a specific positioning device when entering a fire scene for rescue. This device can instantly obtain an accurate internal map of the fire scene. Once the firefighters themselves encounter danger, the command center can view the exact location of the firefighters in distress based on the map and notify nearby firefighters to rescue them. This can greatly protect the lives of firefighters.

[0005] Firefighters wear specialized positioning devices, primarily including the following: The first type utilizes ultra-wideband positioning technology, which offers advantages such as high accuracy, excellent anti-interference capabilities, and strong penetration. Its ultra-high bandwidth and nanometer-scale narrow pulse waves effectively reduce multipath interference, making it ideal for locating firefighters in complex fire environments. However, these devices lack the ability to display fire scene maps, making rescue operations difficult in complex environments.

[0006] The second type is a device that uses simultaneous positioning and mapping technology for positioning. Simultaneous positioning and mapping technology uses various sensors to collect current environmental information and construct a map of the surrounding environment through different algorithm frameworks. Among them, using lidar and camera to build maps are two widely used map building methods. Lidar has the advantages of good real-time mapping and a wide scanning range. Monocular cameras have the advantages of low price, small size, and easy portability. However, these two sensors also have the following shortcomings: (1) Monocular cameras have difficulty in obtaining three-dimensional information of objects and are greatly affected by the light in the environment. During the movement of lidar, violent movement will cause serious distortion; (2) When scanning and building maps in an indoor environment, the error will become larger and larger as the number of scans increases, eventually causing the entire map to be distorted.

[0007] In response to the above-mentioned problems, Chinese patent CN201610856938.2 provides a map construction method based on UWB indoor positioning technology and laser radar, which uses UWB and laser radar dual sensors to construct maps, making the map construction more accurate. Chinese patent CN201710849238.5 provides a pedestrian following system and method based on hybrid positioning of UWB and laser radar. Pedestrians are initially located and identified using UWB, and then accurately located and identified using laser radar, meeting the requirements for precise positioning of pedestrians in complex environments. It has the characteristics of good accuracy and strong stability. However, these two Chinese invention patents focus on pedestrian positioning and do not involve research on real-time map construction. Chinese patent CN202010847107.5 provides an indoor pedestrian following and obstacle avoidance method based on UWB and LiDAR. It uses the AGV's onboard UWB module to obtain pedestrian coordinate information, and uses radar to obtain obstacle information, which is uploaded to the host computer in real time for obstacle avoidance path planning. This can achieve real-time tracking of pedestrians in complex environments, but it cannot obtain indoor maps in real time. Chinese patent CN202111126574.X provides an AGV positioning system and method based on ultra-wideband and laser SLAM composite navigation technology. It uses UWB modules and LiDAR module devices for data fusion, improving the state accumulation error caused by occlusion problems. However, because the LiDAR is placed inside the AGV body, the radar will lose positioning information when the surrounding environment changes significantly.

[0008] Based on this, providing a fire scene map construction method or system to enable instant self-construction of a two-dimensional fire scene map in the absence of an indoor map has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0009] The purpose of the present invention is to provide a fire scene map construction method and system that integrates lidar / vision / UWB, which can realize the timely construction of a two-dimensional map of the fire scene in the absence of an indoor map, effectively making up for the limitations of the existing technology for constructing internal maps of the fire scene and improving the accuracy of internal map construction of the fire scene.

[0010] To achieve the above object, the present invention provides the following solutions:

[0011] A method for constructing a fire scene map integrating lidar / vision / UWB, comprising:

[0012] Obtain UWB positioning information, lidar information and video data information;

[0013] Based on the UWB positioning information, a connectivity fusion algorithm is used to construct an initial map of the fire scene;

[0014] Based on the laser radar information, an ICP algorithm is used to determine the laser radar posture information;

[0015] Constructing a planar local map based on the position information of the laser radar;

[0016] Extracting key frames from the video data information, and determining camera pose information based on the key frames;

[0017] Obtaining a three-dimensional point cloud map based on the camera's position information, and performing dimensionality reduction processing on the three-dimensional point cloud map to obtain a two-dimensional point cloud map;

[0018] Fusing the initial fire scene map, the planar local map, and the two-dimensional point cloud map to obtain a local grid map;

[0019] Performing loop closure detection on the local grid map;

[0020] The position information of the laser radar during the loop detection process is optimized to obtain a global grid map; and the global grid map is used as a fire scene map.

[0021] Preferably, the method of constructing an initial fire scene map based on the UWB positioning information using a connectivity fusion algorithm specifically includes:

[0022] Building a UWB positioning base station and obtaining the location of the UWB positioning base station;

[0023] Determine the location information of the firefighter based on the location of the UWB positioning base station;

[0024] generating a movement trajectory of the firefighter based on the position information of the firefighter;

[0025] Determining whether two firefighters belong to the same connected area based on the location information of the firefighters;

[0026] When two firefighters belong to the same connected area, an open area is generated based on the movement trajectories of the two firefighters; the open area is the area where the firefighters enter the detection area;

[0027] When two firefighters do not belong to the same connected area, a gray area is generated according to the movement trajectories of the two firefighters; the gray area is an obstacle area;

[0028] An initial fire scene map is generated based on the open area and the gray area.

[0029] Preferably, determining whether two firefighters belong to the same connected area based on the location information of the firefighters specifically includes:

[0030] Take the locations of the two firefighters as target points and obtain the connectivity identifiers of the two target points;

[0031] When the connectivity flags of the two target points are equal and the location distance between the two target points meets the preset distance, it is determined that the two firefighters belong to the same connected area;

[0032] When the connectivity identifiers of the two target points are not equal or the position distance between the two target points does not meet the preset distance, it is determined that the two firefighters do not belong to the same connectivity area.

[0033] Preferably, when two firefighters belong to the same connected area, an open area is obtained by horizontally expanding a specific distance with the central axis of the movement trajectories of the two firefighters as the center.

[0034] Preferably, performing loop detection on the local grid map specifically includes:

[0035] extracting a scan frame from the local grid map;

[0036] Matching the scan frame with the laser radar information to obtain a matching result;

[0037] When the matching result meets the preset conditions, the loop detection is completed.

[0038] Preferably, the optimizing the position information of the laser radar during the loop detection process to obtain a global grid map specifically includes:

[0039] The GTSAM optimization library is used to optimize the posture information of the lidar during the loop detection process to obtain a global grid map.

[0040] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0041] The fire scene map construction method integrating lidar / vision / UWB provided by the present invention uses UWB positioning to accurately construct an initial fire scene map, which is then mutually corrected with a self-constructed map using lidar information / video data information. The map is integrated and loop detection is added in a graph optimization framework. This method can realize the timely construction of a two-dimensional fire scene map in the absence of an indoor map, effectively making up for the limitations of existing technologies in constructing internal fire scene maps and improving the accuracy of internal fire scene map construction.

[0042] Corresponding to the above-mentioned method for constructing a fire scene map integrating lidar / vision / UWB, the present invention also provides the following implementation system:

[0043] Among them, a fire scene map construction system integrating lidar / vision / UWB includes:

[0044] Information acquisition module, used to obtain UWB positioning information, lidar information and video data information;

[0045] A first map construction module is used to construct an initial map of the fire scene based on the UWB positioning information using a connectivity fusion algorithm;

[0046] A first pose determination module is used to determine the pose information of the laser radar using an ICP algorithm based on the laser radar information;

[0047] A second map construction module is used to construct a planar local map based on the position information of the laser radar;

[0048] a second posture determination module, configured to extract key frames from the video data information and determine the posture information of the camera based on the key frames;

[0049] a third map construction module, configured to obtain a three-dimensional point cloud map based on the camera's position information, and perform dimensionality reduction processing on the three-dimensional point cloud map to obtain a two-dimensional point cloud map;

[0050] A fourth map construction module is used to fuse the initial fire scene map, the planar local map and the two-dimensional point cloud map to obtain a local grid map;

[0051] A loop detection module, configured to perform loop detection on the local grid map;

[0052] The fifth map construction module is used to optimize the position information of the laser radar during the loop detection process to obtain a global grid map; and use the global grid map as the fire scene map.

[0053] Another fire scene map construction system that integrates lidar / vision / UWB includes:

[0054] The drone is equipped with a UWB positioning base station and a Beidou differential receiving module to obtain the location information of the UWB positioning base station;

[0055] A portable device equipped with a lidar and a monocular camera to obtain lidar information and video data;

[0056] The remote device is wirelessly connected to the UAV and the portable device respectively, and is used to construct a fire scene map according to the fire scene map construction method provided by the present invention based on the location information, the lidar information and the video data information, and display the fire scene map.

[0057] The technical effects achieved by the fire scene map construction system integrating lidar / vision / UWB provided by the present invention are the same as those achieved by the fire scene map construction method provided above, so they will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0059] Figure 1 Flowchart of the fire scene map construction method integrating lidar / vision / UWB provided by the present invention;

[0060] Figure 2 This is a structural diagram of a fire scene map construction system using integrated lidar / vision / UWB provided by the present invention;

[0061] Figure 3 This is a diagram of the fire scene positioning and map construction implementation architecture provided by the present invention;

[0062] Figure 4 This is a flowchart of the fire scene map construction method using the fusion of lidar / vision / UWB provided by the present invention;

[0063] Figure 5 This is a framework diagram of the connectivity fusion algorithm provided by the present invention;

[0064] Figure 6 A schematic diagram of the experimental layout scenario provided by the present invention;

[0065] Figure 7 A map constructed using the Hector algorithm of the conventional lidar provided by the present invention;

[0066] Figure 8The initial fire scene map is constructed using the fire scene map construction method that integrates lidar / vision / UWB provided by the present invention;

[0067] Figure 9 This is the final map constructed using the fire scene map construction method that integrates lidar / vision / UWB provided by the present invention. DETAILED DESCRIPTION

[0068] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0069] The purpose of the present invention is to provide a fire scene map construction method and system that integrates lidar / vision / UWB, which can realize the timely construction of a two-dimensional map of the fire scene in the absence of an indoor map, effectively making up for the limitations of the existing technology for constructing internal maps of the fire scene and improving the accuracy of internal map construction of the fire scene.

[0070] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0071] like Figure 1 As shown, the fire scene map construction method provided by the present invention that integrates laser radar / vision / UWB includes:

[0072] Step 100: Obtain UWB positioning information, lidar information, and video data information.

[0073] Step 101: Based on the UWB positioning information, a connectivity fusion algorithm is used to construct an initial map of the fire scene. The implementation process of this step is as follows:

[0074] Step 1011: Build a UWB positioning base station and obtain the location of the UWB positioning base station.

[0075] Step 1012: Determine the location information of the firefighter based on the location of the UWB positioning base station.

[0076] Step 1013: Generate the firefighter's movement trajectory based on the firefighter's location information.

[0077] Step 1014: Determine whether the two firefighters belong to the same connected area based on the firefighters' location information. For example, using the locations of the two firefighters as target points, obtain connectivity identifiers for the two target points. If the connectivity identifiers of the two target points are equal and the distance between the two target points meets a preset distance, the two firefighters are determined to belong to the same connected area. If the connectivity identifiers of the two target points are unequal or the distance between the two target points does not meet the preset distance, the two firefighters are determined not to belong to the same connected area.

[0078] Step 1015: When two firefighters belong to the same connected area, an open area is obtained by horizontally expanding a specific distance (e.g., 1 meter) from the central axis of the two firefighters' movement trajectories. The open area is the area where the firefighters enter the detection.

[0079] Step 1016: When the two firefighters do not belong to the same connected area, a gray area is generated based on the movement trajectories of the two firefighters. The gray area is an obstacle area.

[0080] Step 1017: Generate an initial fire scene map based on the open area and the gray area.

[0081] Step 102: Based on the lidar information, the ICP algorithm is used to determine the position information of the lidar.

[0082] Step 103: Construct a planar local map based on the laser radar's position information.

[0083] Step 104: extract key frames from the video data information, and determine the camera's position information based on the key frames.

[0084] Step 105: Obtain a three-dimensional point cloud map based on the camera's position information, and perform dimensionality reduction processing on the three-dimensional point cloud map to obtain a two-dimensional point cloud map.

[0085] Step 106: Fuse the initial fire scene map, the planar local map, and the two-dimensional point cloud map to obtain a local grid map.

[0086] Step 107: Perform loop closure detection on the local grid map. The implementation process of this step is as follows:

[0087] Step 1071: Extract the scanned frame in the local grid map.

[0088] Step 1072: Match the scan frame with the laser radar information to obtain a matching result.

[0089] Step 1073: When the matching result meets the preset conditions, the loop detection is completed. Alternatively, in order to improve the accuracy of the map, the number of loop detections can be set according to actual needs.

[0090] Step 108: Optimize the LiDAR pose information during the loop closure detection process to obtain a global grid map. Use the global grid map as the fire scene map. For example, the GTSAM optimization library can be used to optimize the LiDAR pose information during the loop closure detection process to obtain the global grid map.

[0091] Corresponding to the above-mentioned method for constructing a fire scene map integrating lidar / vision / UWB, the present invention also provides the following implementation system:

[0092] One of the fire scene map construction systems that integrates lidar / vision / UWB, such as Figure 2 As shown, the system includes:

[0093] Information acquisition module 1 is used to obtain UWB positioning information, lidar information and video data information.

[0094] The first map construction module 2 is used to construct an initial map of the fire scene based on UWB positioning information using a connectivity fusion algorithm.

[0095] The first pose determination module 3 is used to determine the pose information of the lidar based on the lidar information using the ICP algorithm.

[0096] The second map construction module 4 is used to construct a planar local map based on the position information of the laser radar.

[0097] The second posture determination module 5 is used to extract key frames from the video data information and determine the posture information of the camera based on the key frames.

[0098] The third map construction module 6 is used to obtain a three-dimensional point cloud map based on the camera's posture information, and perform dimensionality reduction processing on the three-dimensional point cloud map to obtain a two-dimensional point cloud map.

[0099] The fourth map construction module 7 is used to fuse the initial fire scene map, the planar local map and the two-dimensional point cloud map to obtain a local grid map.

[0100] The loop detection module 8 is used to perform loop detection on the local grid map.

[0101] The fifth map construction module 9 is used to optimize the position information of the laser radar during the loop detection process to obtain a global grid map, and use the global grid map as the fire scene map.

[0102] Another fire scene map construction system that integrates lidar / vision / UWB includes:

[0103] The drone is equipped with a UWB positioning base station and a Beidou differential receiving module to obtain the location information of the UWB positioning base station.

[0104] A portable device equipped with a lidar and a monocular camera to obtain lidar information and video data information.

[0105] The remote device is wirelessly connected to the drone and the portable device respectively, and is used to construct a fire scene map based on the location information, lidar information and video data information using the above-mentioned fire scene map construction method provided by the present invention, and display the fire scene map.

[0106] The following is Figure 3 Taking the implementation architecture shown as an example, the specific implementation process of the fire scene map construction method and system for integrating lidar / vision / UWB provided by the present invention is described. In actual use, the specific components used in this embodiment are not used as specific limitations of the technical solution provided by the present invention.

[0107] In this embodiment, the fire command center is the remote device.

[0108] like Figure 4 As shown, in this embodiment, based on Figure 3 The implementation architecture shown in the figure shows the implementation process of the fire scene map construction method integrating lidar / vision / UWB.

[0109] Step 1: Build a UWB positioning base station

[0110] During the rescue process, 4-6 drones are equipped with UWB positioning base stations and Beidou differential receiving modules. They are remotely controlled to hover around the burning building where UWB signal transmission is conducive. The position of the outdoor UWB base station is located through the Beidou satellite positioning system, and then the relative position information of the firefighters is obtained through coordinate conversion.

[0111] Step 2: Collect multi-sensor data

[0112] The portable device worn by firefighters integrates a lidar sensor, a monocular camera and a UWB positioning tag to obtain the lidar's scanning ranging information, environmental information and distance information respectively.

[0113] Step 3: Build an initial map of the fire scene

[0114] Firefighters enter the fire scene from different entrances, and the walking trajectory information of all firefighters is obtained through UWB positioning tags. The positioning trajectory of the firefighters is processed by the mean preprocessing method to alleviate the problem of positioning coordinate jump. In order to obtain the initial map of the fire scene, the present invention adopts a connectivity fusion algorithm. In this embodiment, based on the connectivity fusion algorithm, the two sides of the central axis of the positioning trajectory of all firefighter tags are extended 1 meter outward as open space. The space that can be entered is regarded as open space, and the space that cannot be entered is regarded as walls and other obstacles. When multiple firefighters enter the fire scene, the larger the range of activities of each firefighter, the larger the open space between them, and the more complete the initial map of the fire scene obtained. The framework of the connectivity fusion algorithm is as follows: Figure 5 shown.

[0115] The implementation principle of the connectivity fusion algorithm is:

[0116] The displacement difference of each target node from time t to time t+1 is relatively stable, which can be expressed as:

[0117]

[0118] Among them, U th is the upper limit of the displacement threshold, which is set to 2.1 m in this embodiment based on the walking speed of an adult of 0.6 to 2.1 m / s. (x, y) are the position coordinates of the firefighter.

[0119] Collect the indoor walking trajectory data of all firefighters (the location points of each step are connected to form a trajectory) and process the UWB positioning data to form an indoor trajectory in chronological order. The indoor trajectory is a continuous point set. It includes the coordinates of different target nodes i and j at different times t. The interconnectedness of elements i and j in set S can be expressed as target points i and j belonging to the same connected region. C(i) and C(j) represent the connectivity identifiers of the two targets, and D(i, j) represents the distance between the two target points. The connectivity probability between two points in the same connected region among all points with a distance d is recorded as P, that is:

[0120] τ(i,j)=P(C(i)=C(j)≠0|D(i,j)=d) (2)

[0121] When the position information between different firefighters satisfies formula (2), the two target points are considered to be connected. After obtaining these positioning trajectories, the above connectivity processing method is used, and so on, to obtain the connected areas between all firefighters.

[0122] Step 4: Build a LiDAR map

[0123] The firefighter's posture is estimated using the ICP algorithm using LiDAR scanning data and then further optimized.

[0124] Step 5: Build a monocular camera point cloud map

[0125] The monocular camera's motion is captured through the video stream, which then extracts image information between adjacent frames and performs keyframe extraction to determine the camera's motion. The 3D point cloud map generated by the monocular camera is projected onto a 2D laser plane through dimensionality reduction. This is then combined with the LiDAR point cloud data as radar data for subsequent scan matching, improving the accuracy of firefighter pose estimation.

[0126] Then, the local grid map is obtained by fusing the initial fire scene map, the lidar map and the monocular camera point cloud map.

[0127] Step 6: Loop Detection

[0128] Extract feature points from all scanned frames, perform scan matching, and complete loop detection. The purpose of loop detection is to obtain globally consistent trajectories and maps, reduce the cumulative errors generated during the back-end optimization process, and obtain a globally consistent raster map.

[0129] Step 7: Graph Optimization

[0130] Construct an optimization graph and input it into the GTSAM optimization library for calculation to obtain the processed optimization graph. This optimization library does not optimize the pose outside the loop, but only optimizes the pose within the loop.

[0131] 1. Assume that the source point cloud and the target point cloud are represented by U = {u1,u2,...,u N}, V={v1,v2,...,v N} means that the center of gravity of the two point cloud sets is calculated first

[0132] 2. The error function F(x) is:

[0133]

[0134] Among them, x i and x j Represents the position information of vertices i and j, that is, the position of the firefighter at different times. ij is the actual observation value obtained by the sensor between vertices i and j. C represents the set of all constraints of the lidar. Ω ij is the information matrix of the constraints. ij (x i ,x j ,z ij) is the difference between the predicted value and the measured value, that is, the error function. At this point, the graph optimization problem is finally transformed into the problem of optimal solution, the main purpose of which is to find x that meets the standard * Make the objective function F(x) reach the minimum value.

[0135] 3. Construct the objective function E(R, T) using the rotation matrix R and the translation matrix T. Set a threshold and determine whether E(R, T) is less than the threshold. If so, the loop is closed and the map construction ends. If not, R and T are respectively inserted into the source point cloud U to obtain a new set of points M. Then, M is matched with the target point cloud V to solve for the new R and T. After repeated iterations, convergence is achieved, and the final optimal transformation matrix is obtained.

[0136] Match the two sets of point clouds and minimize the objective function by rotating and translating the target point cloud V:

[0137]

[0138] Where N represents the number of point clouds, R and T represent the rotation matrix and translation matrix between two point clouds, respectively.

[0139] 4. Since the graph optimization front-end completes the construction of the front-end graph by taking the postures of the firefighters at different times and the positions of the UWB tags as vertices, and since the front-end generates certain errors when constructing the transformation matrix, the postures of the nodes in the graph are not globally optimal in the subsequent graph construction process. It is necessary to continuously adjust the relationship between the vertices and edges to meet the globally optimal estimated posture. Therefore, the graph constructed by the front-end needs to be optimized at the back-end. Assume that the posture of the mobile firefighter is X, the observed value of the sensor is Z, and the predicted value of the sensor is f(x). The principle of the graph optimization back-end is to use the nonlinear least squares method to obtain the optimal X, so that the error function e between the predicted value and the observed value of the system is i (x) is minimized, and the optimal pose estimation is completed, which is expressed as:

[0140] e i (x) = f i (x)-z i (5)

[0141] Based on the above description, the present invention uses three sensors, lidar, monocular camera, and UWB, to respectively collect environmental information, image information, and positioning trajectory of the firefighters around them for map construction. The UWB data is first subjected to connectivity fusion calculation to obtain an initial indoor map, and then the three-dimensional point cloud map obtained by the monocular camera is projected onto the lidar plane. Multiple sensors collect data to make the constructed map more accurate.

[0142] In addition, for map fusion, the present invention uses a graph optimization algorithm to fuse the data information of the three sensors, adds loop detection information to make the firefighter's posture information more accurate, and thus obtains an optimized indoor map.

[0143] In order to verify the mapping performance of the present invention, a large corridor environment scene (such as Figure 6 The experiment was conducted under the following conditions (as shown in the figure), and five different locations were marked in the experimental environment to facilitate error analysis. The map constructed by the Hector algorithm of the traditional lidar is as follows Figure 7 As shown, the initial fire scene map constructed by the method of the present invention is as follows Figure 8 As shown in the figure, the final fire scene map is as follows Figure 9 As shown, through comparative analysis of experimental results, it is found that the map constructed by the present invention is more accurate due to the correction of closed-loop detection, the obstacles are clearer, and the environmental information is reflected more realistically.

[0144] As shown in Table 1, due to the single sensor factor, the average error of the LiDAR mapping algorithm in the experiment was 0.781m. The error of the proposed method decreased by 0.551m, a 70.5% reduction compared to LiDAR mapping alone. This experimental result demonstrates that the proposed mapping algorithm can produce more accurate maps.

[0145] Table 1 Map construction accuracy analysis table

[0146]

[0147] Based on the above description, once a fire breaks out in a building, after the fire command vehicle arrives at the scene, firefighters wearing portable devices enter the scene from different entrances. During the rescue operation, four to six drones equipped with UWB positioning base stations and BeiDou differential receiver modules can be used to control the drones to hover around the burning building, where UWB signal transmission is facilitated. The UWB positioning base stations and BeiDou differential receiver modules meet the normal payload range of the drones. The outdoor UWB base stations are located using the BeiDou satellite positioning system, and then the firefighters' relative positions are obtained through coordinate conversion. The portable devices worn by firefighters integrate a lidar sensor, a monocular camera, and a UWB positioning tag. These devices capture lidar maps, image information of the surrounding environment, and all firefighters' movement trajectories. A loop closure detection algorithm based on local submap matching is designed. This algorithm constructs submaps using lidar point cloud data. Successive scans are aligned with the submap using nonlinear optimization, and loop closure detection is performed to improve mapping accuracy, ultimately producing a map that can be used as a reference for the command center. When firefighters themselves encounter danger, the fire commander responsible for coordinating and dispatching tasks can view the firefighters' location and surrounding environment information in real time based on the map in the background, and promptly notify nearby firefighters to provide rescue, thereby reducing the casualty rate of firefighters.

[0148] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0149] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A fire scene map construction method integrating laser radar / vision / UWB, characterized in that: include: Obtain UWB positioning information, lidar information and video data information; Based on the UWB positioning information, a connectivity fusion algorithm is used to construct an initial map of the fire scene; Based on the laser radar information, an ICP algorithm is used to determine the laser radar posture information; Constructing a planar local map based on the position information of the laser radar; Extracting key frames from the video data information, and determining camera pose information based on the key frames; Obtaining a three-dimensional point cloud map based on the camera's position information, and performing dimensionality reduction processing on the three-dimensional point cloud map to obtain a two-dimensional point cloud map; Fusing the initial fire scene map, the planar local map, and the two-dimensional point cloud map to obtain a local grid map; Performing loop closure detection on the local grid map; Optimizing the position information of the laser radar during the loop detection process to obtain a global grid map; using the global grid map as a fire scene map; Based on the UWB positioning information, a connectivity fusion algorithm is used to construct an initial map of the fire scene, specifically including: Building a UWB positioning base station and obtaining the location of the UWB positioning base station; Determine the location information of the firefighter based on the location of the UWB positioning base station; generating a movement trajectory of the firefighter based on the position information of the firefighter; Determining whether two firefighters belong to the same connected area based on the location information of the firefighters; When two firefighters belong to the same connected area, an open area is generated based on the movement trajectories of the two firefighters; the open area is the area where the firefighters enter the detection area; When two firefighters do not belong to the same connected area, a gray area is generated according to the movement trajectories of the two firefighters; the gray area is an obstacle area; An initial fire scene map is generated based on the open area and the gray area.

2. The method for constructing a fire scene map integrating laser radar / vision / UWB according to claim 1, characterized in that: The determining whether two firefighters belong to the same connected area based on the location information of the firefighters specifically includes: Take the locations of the two firefighters as target points and obtain the connectivity identifiers of the two target points; When the connectivity flags of the two target points are equal and the location distance between the two target points meets the preset distance, it is determined that the two firefighters belong to the same connected area; When the connectivity identifiers of the two target points are not equal or the position distance between the two target points does not meet the preset distance, it is determined that the two firefighters do not belong to the same connectivity area.

3. The method for constructing a fire scene map integrating laser radar / vision / UWB according to claim 1, characterized in that: When two firefighters belong to the same connected area, the open area is obtained by horizontally expanding a specific distance with the central axis of the two firefighters' movement trajectories as the center.

4. The method for constructing a fire scene map integrating laser radar / vision / UWB according to claim 1, characterized in that: The performing loop detection on the local grid map specifically includes: extracting a scan frame from the local grid map; Matching the scan frame with the laser radar information to obtain a matching result; When the matching result meets the preset conditions, the loop detection is completed.

5. The method for constructing a fire scene map integrating laser radar / vision / UWB according to claim 1, characterized in that: The optimization of the laser radar's posture information during the loop detection process to obtain a global grid map specifically includes: The GTSAM optimization library is used to optimize the posture information of the lidar during the loop detection process to obtain a global grid map.

6. A fire scene map construction system integrating laser radar / vision / UWB, characterized in that: The laser radar / vision / UWB fusion fire scene map construction system is used to implement the laser radar / vision / UWB fusion fire scene map construction method according to any one of claims 1 to 5; the system includes: Information acquisition module, used to obtain UWB positioning information, lidar information and video data information; A first map construction module is used to construct an initial map of the fire scene based on the UWB positioning information using a connectivity fusion algorithm; A first pose determination module is used to determine the pose information of the laser radar using an ICP algorithm based on the laser radar information; A second map construction module is used to construct a planar local map based on the position information of the laser radar; a second posture determination module, configured to extract key frames from the video data information and determine the posture information of the camera based on the key frames; a third map construction module, configured to obtain a three-dimensional point cloud map based on the camera's position information, and perform dimensionality reduction processing on the three-dimensional point cloud map to obtain a two-dimensional point cloud map; A fourth map construction module is used to fuse the initial fire scene map, the planar local map and the two-dimensional point cloud map to obtain a local grid map; A loop detection module, configured to perform loop detection on the local grid map; The fifth map construction module is used to optimize the position information of the laser radar during the loop detection process to obtain a global grid map; and use the global grid map as the fire scene map.

7. A fire scene map construction system integrating laser radar / vision / UWB, characterized in that: include: The drone is equipped with a UWB positioning base station and a Beidou differential receiving module to obtain the location information of the UWB positioning base station; A portable device equipped with a lidar and a monocular camera to obtain lidar information and video data; A remote device is wirelessly connected to the drone and the portable device respectively, and is used to construct a fire scene map based on the location information, the lidar information and the video data information using the fire scene map construction method according to any one of claims 1 to 5, and display the fire scene map.

Images