A method and system for constructing an accident disaster semantic map based on multi-source information fusion, and a computer readable storage medium

Abstract

The application discloses a kind of based on multi-source information fusion accident disaster semantic map construction method, system and computer readable storage medium, method includes the following steps: time synchronization is carried out to multi-source data, space alignment, denoising and smoothing processing;Based on the multi-source data after pre-processing, carry out semantic feature extraction, semantic feature includes geometric feature, visual semantic feature and environmental semantic feature;Using hierarchical fusion method, first fusion geometric feature and visual semantic feature, generate preliminary fusion feature set, again secondary fusion is carried out to preliminary fusion feature set and environmental semantic feature, generate secondary fusion feature set including geometric feature, visual semantic information and environmental semantic information;With the three-dimensional point cloud geometry map constructed by synchronous positioning and mapping method as carrier, semantic label in secondary fusion feature set is mapped to the corresponding three-dimensional position of geometry map, constructs three-dimensional semantic map.The application can improve the comprehensiveness of rescue information under accident disaster environment.

Description

Technical Field

[0001] This invention relates to the field of disaster emergency response technology, and in particular to a method, system, and computer-readable storage medium for constructing an accident disaster semantic map based on multi-source information fusion. Background Technology

[0002] Following accidents and disasters such as fires and chemical plant leaks, the scene typically presents a complex and dangerous environment characterized by high temperatures, toxicity, and dense smoke. Existing maps are insufficient to address the complex changes at accident and disaster sites, while traditional manual exploration methods suffer from low efficiency and high risk. In recent years, mobile robot technology based on Simultaneous Localization and Mapping (SLAM) has provided an effective means for autonomous detection and mapping of accident and disaster environments.

[0003] However, existing SLAM methods primarily focus on constructing environmental geometry, outputting only point cloud maps or occupancy grid maps. They lack consideration for hazardous factors at accident and disaster sites, such as high temperatures, toxic and harmful gases, and smoke, making it difficult to support risk assessment, command and decision-making, and safe route planning in accident and disaster relief. Furthermore, while some semantic SLAM methods incorporate target recognition information, their semantic information is largely limited to object category recognition such as tables, chairs, windows, and doors, lacking consideration for disaster-related information closely related to trapped personnel, safe passages, and hazardous factors, thus failing to generate comprehensive semantic risk maps for accidents and disasters.

[0004] Therefore, existing methods construct semantic maps with limited information dimensions and lack environmental hazard information. This paper proposes a method for constructing accident and disaster semantic maps that integrates geometric structure information, visual semantic information, and environmental semantic information, in order to improve the comprehensiveness of rescue information in accident and disaster environments and enhance command and decision support for disaster relief. Summary of the Invention

[0005] This invention aims to at least solve one of the technical problems existing in the prior art. To this end, this invention proposes a method, system, and computer-readable storage medium for constructing an accident and disaster semantic map based on multi-source information fusion. The method for constructing the accident and disaster semantic map can improve the comprehensiveness of rescue information in accident and disaster environments.

[0006] The present invention also proposes an accident and disaster semantic map construction system based on multi-source information fusion.

[0007] The present invention also proposes a computer-readable storage medium.

[0008] The method for constructing an accident and disaster semantic map based on multi-source information fusion according to the first aspect of the present invention includes the following steps: S1: Preprocess the multi-source data. Using the timestamp of the inertial measurement unit as a reference, synchronize and align the original multi-source data collected by the vision sensor, lidar, inertial measurement unit and environmental sensor in time and space, and perform noise reduction and smoothing on the data. S2: Based on the preprocessed multi-source data, semantic features are extracted, including geometric features, visual semantic features, and environmental semantic features; S3: Using a layered fusion method, the geometric features and the visual semantic features are first fused to generate a preliminary fused feature set. Then, the preliminary fused feature set is fused with the environmental semantic features to generate a secondary fused feature set containing geometric features, visual semantic information and environmental semantic information. S4: Using the 3D point cloud geometric map constructed by the synchronous positioning and mapping method as a carrier, the semantic labels in the secondary fusion feature set are mapped to the corresponding 3D positions on the geometric map to construct a 3D semantic map.

[0009] Alternatively, methods for achieving time synchronization and spatial alignment include: Using the inertial measurement unit timestamp as a reference, the time series of other multi-source sensors are linearly interpolated and the time offset is corrected. LVI-SAM is used to estimate the robot's pose in the global coordinate system in real time. Combined with the extrinsic parameter matrix between the sensor and the robot body, the observation data of different sensors in their respective coordinate systems are mapped to a unified global coordinate system through coordinate transformation.

[0010] Optionally, methods for semantic information extraction include: Geometric feature extraction: From the preprocessed laser point cloud data, the covariance matrix of the local spatial distribution is calculated using a feature extraction method based on neighborhood statistics. The local structure information of the point cloud is obtained through eigenvalue decomposition. Curvature parameters are calculated based on eigenvalues, and edge points and planar points are selected to form a geometric feature set. Visual semantic information extraction: The YOLOv8 model is used to perform target detection on the preprocessed image data, and the target bounding box information, target type and detection confidence are output to form a visual semantic feature set. The target type includes at least trapped personnel, flames and safety passages. Environmental semantic information extraction: Based on the preset hazard level classification rules and threshold determination methods, the hazard level of the preprocessed environmental state information is determined and hazard semantic labels are generated. The hazard semantic labels are associated with the corresponding three-dimensional positions by combining the pose information of the environmental sensors to generate a hazard semantic feature set.

[0011] Optionally, in the extraction of environmental semantic information, the preset hazard level classification rules for carbon monoxide gas concentration are as follows: less than 50 ppm is safety level 0, 50~100 ppm is mild hazard level 1, 200~800 ppm is medium to high hazard level 2, and greater than 800 ppm is severe hazard level 3; the formula for calculating the detection confidence of hazard information is: Where is the warning lower limit for the k-th type of dangerous semantics, and is the danger confirmation threshold for the k-th type of dangerous semantics.

[0012] Optionally, the specific steps of the first-layer fusion in the multi-source information fusion include: The pixel coordinates within the detected 2D bounding box are back-projected onto the camera normalization plane through the camera intrinsic parameter matrix, and then converted to polar coordinates after normalization. Project the coordinates of the nearby laser point cloud onto the camera plane and convert them to polar coordinates; By using KD-Tree to search for the polar coordinates of visual semantic features, the nearest multiple depth points are found on the sphere, and the visual depth is obtained through visual feature rays to form the projection position of the visual semantic target in the point cloud map. Each point cloud is labeled with visual semantic information on the point cloud map to generate a preliminary fusion feature set.

[0013] Optionally, the specific steps of the second-layer fusion in the multi-source information fusion include: based on the environmental semantic feature set, filtering all environmental data that exceed the danger warning threshold, adding the danger type, spatial location and danger level of the environmental danger data to the preliminary fusion feature set, and generating a secondary fusion feature set containing geometric features, visual semantic information and environmental semantic information.

[0014] According to a second aspect of the present invention, an accident and disaster semantic map construction system based on multi-source information fusion is used to perform the above-described method, the system comprising: Data preprocessing unit: used to perform time synchronization, spatial alignment, and noise reduction and smoothing on multi-source raw data collected by vision sensors, LiDAR, inertial measurement units, and environmental sensors; Semantic information extraction unit: includes a geometric feature extraction module, a visual semantic information extraction module, and an environmental semantic information extraction module, which are used to extract geometric features, visual semantic features, and environmental semantic features from the preprocessed data, respectively. Multi-source information fusion unit: Used to employ a hierarchical fusion method to first fuse geometric features and visual semantic features, and then fuse the fusion result with environmental semantic features to generate a secondary fusion feature set; Semantic map construction unit: Used to map semantic labels to corresponding positions on the geometric map based on a 3D point cloud geometric map constructed by simultaneous localization and mapping method, and to construct and output a 3D semantic map.

[0015] Optionally, the visual semantic information extraction module adopts the YOLOv8 model to identify and locate trapped personnel, flames, and safety passage targets in accident and disaster scenarios, and assigns corresponding visual semantic labels; the environmental semantic information extraction module determines the level of at least one environmental hazard quantity in ambient temperature, toxic gases, and smoke according to preset hazard level classification rules and threshold determination methods, and generates hazard semantic labels.

[0016] Optionally, the three-dimensional semantic map is marked with the location of trapped personnel, the location of safe passages, and information on dangerous areas, which is used to provide data support for the formulation of accident and disaster relief plans.

[0017] According to a third aspect of the present invention, a computer-readable storage medium has a computer program stored thereon, which, when executed by a processor, implements the above-described method for constructing an accident and disaster semantic map based on multi-source information fusion.

[0018] The method, system, and computer-readable storage medium for constructing accident and disaster semantic maps based on multi-source information fusion of the present invention have at least the following beneficial effects: 1. Semantic maps that integrate multi-source information not only include the geometric structure information of the accident site, but also the semantic information of targets such as trapped personnel, as well as information on dangerous physical quantities such as high temperature, toxic gas, and smoke. This improves the diversity and comprehensiveness of rescue information in the semantic map of accidents and disasters, thereby serving accident and disaster rescue decision-making.

[0019] 2. Applicable to various disaster scenarios such as fire, explosion, and chemical leak, with strong versatility and expandability. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart of a method for constructing an accident and disaster semantic map according to an embodiment of the present invention; Figure 2 This is a schematic diagram of an accident and disaster semantic map construction system according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a multi-source information fusion process according to an embodiment of the present invention. Detailed Implementation

[0022] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0023] The following disclosure provides numerous different embodiments or examples for implementing various structures of the invention. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the invention. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed. Additionally, examples of various specific processes and materials are provided in this invention; however, those skilled in the art will recognize the applicability of other processes and / or the use of other materials.

[0024] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0025] refer to Figure 1 A method for constructing an accident and disaster semantic map based on multi-source information fusion includes the following steps: S1: Preprocess the multi-source data. Using the timestamp of the inertial measurement unit as a reference, synchronize and align the original multi-source data collected by the vision sensor, lidar, inertial measurement unit and environmental sensor in time and space, and perform noise reduction and smoothing on the data. S2: Based on the preprocessed multi-source data, semantic features are extracted, including geometric features, visual semantic features, and environmental semantic features; S3: A hierarchical fusion method is adopted. First, geometric features and visual semantic features are fused to generate a preliminary fused feature set. Then, the preliminary fused feature set is fused with environmental semantic features to generate a secondary fused feature set containing geometric features, visual semantic information and environmental semantic information. S4: Using a 3D point cloud geometric map constructed by simultaneous localization and mapping (SLAM) as a carrier, the semantic labels in the secondary fusion feature set are mapped to the corresponding 3D positions on the geometric map to construct a 3D semantic map.

[0026] In step S1, the multi-source data is preprocessed, which mainly involves performing spatiotemporal synchronization and standardization on the raw data such as image sequences, point cloud data, pose, and environmental data collected by multiple source sensors, including visual sensors, lidar, inertial measurement units (IMUs), and environmental sensors.

[0027] Specifically, this includes: (1) using the IMU timestamp as a reference, linearly interpolating the time series of other multi-source sensors and correcting for time offset, so as to achieve synchronization of multi-source data at the same time.

[0028] (2) Select image data and environmental data that are closest to the current point cloud data based on the timestamp, and use LVI-SAM (Lidar-Visual-Inertial Simultaneous Localization and Mapping) to estimate the robot's pose in the global coordinate system in real time. Combining the extrinsic parameter matrix between the sensor and the machine body The observation data from different sensors in their respective coordinate systems Through coordinate transformation: Mapped to a unified global coordinate system This enables spatial alignment of data from multiple sensor sources.

[0029] (3) To reduce the impact of noise, for laser point cloud data, based on the high-frequency attitude information provided by the IMU, time interpolation and motion compensation are performed on each point during the scanning process to achieve point cloud distortion removal, and curvature feature constraints are combined to suppress the influence of outliers; for image data, camera intrinsic parameters are used to correct image distortion, and a feature point tracking mechanism is used to improve the stability of visual information; for environmental sensor data, a method is adopted... The principle is to remove abnormal data jumps caused by sensor drift and sudden interference, and to smooth the data using a moving average method to remove noise outliers and retain the true changes in the disaster. The preprocessed data is then used as input for the subsequent semantic information extraction unit.

[0030] In step S2, based on the preprocessed multi-source data, semantic features are extracted. Geometric features are extracted from geometric data to construct a preliminary map, target semantic features are extracted from visual semantic data, and hazard semantic features are extracted from hazardous environment data, thereby realizing the extraction and preliminary association of semantic information of the three types of features.

[0031] Geometric Feature Extraction: From the preprocessed laser point cloud data, a feature extraction method based on neighborhood statistics is used to calculate the covariance matrix of the local spatial distribution. Then, local information of the point cloud structure is obtained through eigenvalue decomposition. Furthermore, curvature parameters are calculated based on eigenvalues ​​to filter out edge points and planar points, ultimately forming the geometric feature set of LVI-SAM. in, N represents the spatial coordinates in the 3D point cloud robot base coordinate system, and N is the total number of geometric points.

[0032] Visual semantic information extraction: The YOLOv8 model is used, and the input image is... The model first extracts multi-scale feature maps using a convolutional neural network: in, This represents the feature extraction function of the l-th convolutional layer. Subsequently, object detection is performed on the feature map, outputting the center coordinates of the visual semantic target bounding box. ,width Height h, target type C, and detection confidence This forms a visual semantic feature set: in, Let m be the category label for the m-th visual semantic target. Here, M represents the pixel coordinates of the visual semantic target, and M represents the total number of visual semantic targets. The YOLO network is used to identify and locate key targets in accident and disaster scenarios, including trapped personnel, flames, and safety passages, and assigns them corresponding semantic labels.

[0033] Environmental semantic information extraction: Semantic annotation is performed on the preprocessed environmental state information based on preset hazard level classification and threshold determination methods. According to disaster type and emergency rescue needs, and referring to national safety standards and disaster relief specifications, hazard level classification rules are preset for environmental hazard quantities such as ambient temperature, toxic gases, and smoke. Taking carbon monoxide as an example, a gas concentration of less than 50 ppm is considered safe, 50-100 ppm is considered slightly dangerous, 200-800 ppm is considered medium to high dangerous, and greater than 800 ppm is considered severely dangerous, with hazard levels classified as 0, 1, 2, and 3 respectively. Simultaneously, a threshold determination method is used to calculate the confidence level of the current environmental hazard information. The detection confidence level of the hazard information is: in, This is the warning threshold for the k-th type of dangerous semantics; anything below this value is considered safe. It is the danger confirmation threshold for the k-th type of dangerous semantic, exceeding... The data is identified as hazardous. Based on pre-defined hazard level classification rules and confidence calculations, the processed environmental data is assessed for hazard level, generating hazard semantic labels. Combining the pose information from environmental sensors, the hazard semantic labels are associated with their corresponding 3D locations, ultimately generating a hazard semantic feature set. in, Let k be the type of the dangerous semantic. It is classified as a dangerous level. The coordinates of the robot's base coordinate system for the hazard information collection points. The set of semantic features represents the confidence level for detecting hazard information, where K is the total number of hazard information collection points. This set of semantic features will serve as the data support for subsequent multi-source information fusion units.

[0034] Reference Figure 3 In step S3, a hierarchical fusion method is used to achieve the unified fusion of geometric information, visual semantic information and environmental semantic information. First, the geometric features and visual semantic features are initially fused, and then the fusion result is fused with the hazard semantic features for a second time.

[0035] In the first layer of fusion, the pixel coordinates of all previously detected 2D bounding boxes are combined. Then, it is back-projected onto the camera normalized plane through the camera intrinsic parameter matrix K: Then, after normalization, convert to polar coordinates. Similarly, the coordinates of nearby laser point clouds. Projected onto the camera plane and then converted to polar coordinates Then, the polar coordinates of the visual semantic features are searched using a KD-Tree to find the three nearest depth points on the sphere, and the visual depth is obtained through the visual feature rays. This forms the projection position of the visual semantic target in the point cloud map. Finally, visual semantic information labels are assigned to each point cloud on the point cloud map, generating a preliminary fusion feature set. .

[0036] In the second layer of fusion, based on the extracted environmental semantic information, all environmental data exceeding the danger warning threshold are filtered out, and the danger type is categorized. Spatial location of environmental hazard data and hazard level Add to Finally, a secondary fusion feature set containing geometric features, visual semantic information, and environmental semantic information is generated. .

[0037] In step S4, based on the above Using a high-precision 3D point cloud geometric map constructed with LVI-SAM as a carrier, the fused visual semantic tags and hazard semantic tags are mapped to the corresponding 3D locations on the geometric map to construct a 3D semantic map. At the same time, the semantic map is visualized and output in various forms such as point cloud format and raster format, supporting synchronous display on mobile rescue platforms and emergency command centers. By using key rescue information such as the location of trapped personnel, passage location, and danger zone marked on the map, data support is provided for the formulation of rescue plans.

[0038] A system for constructing accident and disaster semantic maps based on multi-source information fusion, used to execute the above method, includes: Data preprocessing unit: used to perform time synchronization, spatial alignment, and noise reduction and smoothing on multi-source raw data collected by vision sensors, LiDAR, inertial measurement units, and environmental sensors; Semantic information extraction unit: includes a geometric feature extraction module, a visual semantic information extraction module, and an environmental semantic information extraction module, which are used to extract geometric features, visual semantic features, and environmental semantic features from the preprocessed data, respectively. Multi-source information fusion unit: Used to employ a hierarchical fusion method to first fuse geometric features and visual semantic features, and then fuse the fusion result with environmental semantic features to generate a secondary fusion feature set; Semantic map construction unit: Using a 3D point cloud geometric map constructed by synchronous positioning and mapping method as a carrier, semantic labels are mapped to the corresponding positions on the geometric map to construct and output a 3D semantic map. The 3D semantic map is marked with the location of trapped personnel, the location of safe passages and information on dangerous areas, which is used to provide data support for the formulation of accident and disaster relief plans.

[0039] The visual semantic information extraction module uses the YOLOv8 model to identify and locate trapped personnel, flames, and safety passage targets in accident and disaster scenarios, and assigns corresponding visual semantic labels. The environmental semantic information extraction module determines the level of at least one environmental hazard quantity in ambient temperature, toxic gases, and smoke according to preset hazard level classification rules and threshold determination methods, and generates hazard semantic labels.

[0040] A computer-readable storage medium, wherein a computer program, when executed by a processor, implements the above-described method for constructing an accident and disaster semantic map based on multi-source information fusion.

[0041] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for constructing an accident and disaster semantic map based on multi-source information fusion, characterized in that, Includes the following steps: S1: Preprocess the multi-source data. Using the timestamp of the inertial measurement unit as a reference, synchronize and align the original multi-source data collected by the vision sensor, lidar, inertial measurement unit and environmental sensor in time and space, and perform noise reduction and smoothing on the data. S2: Based on the preprocessed multi-source data, semantic features are extracted, including geometric features, visual semantic features, and environmental semantic features; S3: Using a layered fusion method, the geometric features and the visual semantic features are first fused to generate a preliminary fused feature set. Then, the preliminary fused feature set is fused with the environmental semantic features to generate a secondary fused feature set containing geometric features, visual semantic information and environmental semantic information. S4: Using the 3D point cloud geometric map constructed by the synchronous positioning and mapping method as a carrier, the semantic labels in the secondary fusion feature set are mapped to the corresponding 3D positions on the geometric map to construct a 3D semantic map.

2. The method for constructing an accident and disaster semantic map based on multi-source information fusion according to claim 1, characterized in that, Methods for achieving time synchronization and spatial alignment include: Using the inertial measurement unit timestamp as a reference, the time series of other multi-source sensors are linearly interpolated and the time offset is corrected. LVI-SAM is used to estimate the robot's pose in the global coordinate system in real time. Combined with the extrinsic parameter matrix between the sensor and the robot body, the observation data of different sensors in their respective coordinate systems are mapped to a unified global coordinate system through coordinate transformation.

3. The method for constructing an accident and disaster semantic map based on multi-source information fusion according to claim 1, characterized in that, Methods for semantic information extraction include: Geometric feature extraction: From the preprocessed laser point cloud data, the covariance matrix of the local spatial distribution is calculated using a feature extraction method based on neighborhood statistics. The local structure information of the point cloud is obtained through eigenvalue decomposition. Curvature parameters are calculated based on eigenvalues, and edge points and planar points are selected to form a geometric feature set. Visual semantic information extraction: The YOLOv8 model is used to perform target detection on the preprocessed image data, and the target bounding box information, target type and detection confidence are output to form a visual semantic feature set. The target type includes at least trapped personnel, flames and safety passages. Environmental semantic information extraction: Based on the preset hazard level classification rules and threshold determination methods, the hazard level of the preprocessed environmental state information is determined and hazard semantic labels are generated. The hazard semantic labels are associated with the corresponding three-dimensional positions by combining the pose information of the environmental sensors to generate a hazard semantic feature set.

4. The method for constructing an accident and disaster semantic map based on multi-source information fusion according to claim 3, characterized in that, In the extraction of environmental semantic information, the preset hazard level classification rules for carbon monoxide gas concentration are as follows: less than 50 ppm is safety level 0, 50~100 ppm is mild hazard level 1, 200~800 ppm is medium to high hazard level 2, and greater than 800 ppm is severe hazard level 3; the formula for calculating the detection confidence of hazard information is: in, This is the lower limit of the warning for the k-th type of dangerous semantics. It is the danger confirmation threshold for the k-th type of dangerous semantic.

5. The method for constructing an accident and disaster semantic map based on multi-source information fusion according to claim 1, characterized in that, The specific steps of the first-layer fusion in the multi-source information fusion include: The pixel coordinates within the detected 2D bounding box are back-projected onto the camera normalization plane through the camera intrinsic parameter matrix, and then converted to polar coordinates after normalization. Project the coordinates of the nearby laser point cloud onto the camera plane and convert them to polar coordinates; By using KD-Tree to search for the polar coordinates of visual semantic features, the nearest multiple depth points are found on the sphere, and the visual depth is obtained through visual feature rays to form the projection position of the visual semantic target in the point cloud map. Each point cloud is labeled with visual semantic information on the point cloud map to generate a preliminary fusion feature set.

6. The method for constructing an accident and disaster semantic map based on multi-source information fusion according to claim 1, characterized in that, The specific steps of the second-layer fusion in the multi-source information fusion include: based on the environmental semantic feature set, filtering all environmental data that exceed the danger warning threshold, adding the danger type, spatial location and danger level of the environmental danger data to the preliminary fusion feature set, and generating a secondary fusion feature set containing geometric features, visual semantic information and environmental semantic information.

7. A system for constructing semantic maps of accidents and disasters based on multi-source information fusion, used to execute the method according to any one of claims 1 to 6, characterized in that, include: Data preprocessing unit: used to perform time synchronization, spatial alignment, and noise reduction and smoothing on multi-source raw data collected by vision sensors, LiDAR, inertial measurement units, and environmental sensors; Semantic information extraction unit: includes a geometric feature extraction module, a visual semantic information extraction module, and an environmental semantic information extraction module, which are used to extract geometric features, visual semantic features, and environmental semantic features from the preprocessed data, respectively. Multi-source information fusion unit: Used to employ a hierarchical fusion method to first fuse geometric features and visual semantic features, and then fuse the fusion result with environmental semantic features to generate a secondary fusion feature set; Semantic map construction unit: Used to map semantic labels to corresponding positions on the geometric map based on a 3D point cloud geometric map constructed by simultaneous localization and mapping method, and to construct and output a 3D semantic map.

8. The accident and disaster semantic map construction system based on multi-source information fusion according to claim 7, characterized in that, The visual semantic information extraction module uses the YOLOv8 model to identify and locate trapped personnel, flames, and safety passage targets in accident and disaster scenarios, and assigns corresponding visual semantic labels. The environmental semantic information extraction module determines the level of at least one environmental hazard quantity in ambient temperature, toxic gases, and smoke according to preset hazard level classification rules and threshold determination methods, and generates hazard semantic labels.

9. The accident and disaster semantic map construction system based on multi-source information fusion according to claim 7, characterized in that, The three-dimensional semantic map is marked with the location of trapped personnel, the location of safe passages, and information on dangerous areas, which is used to provide data support for the formulation of accident and disaster relief plans.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the accident and disaster semantic map construction method based on multi-source information fusion as described in any one of claims 1 to 6.

Images