Fire safety hazard identification and labeling system based on visual understanding

Abstract

The application discloses a fire safety hidden danger identification and marking system based on visual understanding and relates to the technical field of computer vision, which comprises the following steps: generating a three-dimensional semantic field by using a multilayer perception network, calculating the distance and relative azimuth angle between entities to match the spatial relation predicate, constructing a scene graph, inputting a preset hidden danger judgment logic tree, obtaining an output result, constructing an Eikonal equation and solving it, calculating the evacuation time increment of the evacuation time field relative to the unobstructed reference scene, and outputting a hidden danger quantitative value.The three-dimensional semantic field containing geometric, color and semantic information is constructed, and when judging the hidden dangers such as blocked safety exits or blocked fire equipment, the application is no longer limited by the limitations of single-view visual features, but uses the three-dimensional spatial topological relationship between entities for deep reasoning, constructs an evacuation time field by solving the Eikonal equation, and greatly improves the fine level of fire management and the actual effectiveness of hidden danger management.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a fire safety hazard identification and labeling system based on visual understanding. Background Technology

[0002] In recent years, with the in-depth development of smart city and Internet of Things technologies, the fire safety field is undergoing a major transformation from passive defense to proactive intelligent supervision. This places higher demands on the digital and refined perception capabilities of scene environments. In modern building spaces, fire safety is not only about monitoring single fires, but also involves the identification of routine violations and the assessment of potential risk sources. These hidden dangers often have significant spatial geometric attributes, and their existence depends on their relative position and topological relationship in three-dimensional space. The evolution of visual perception technology has provided a technical foundation for high-fidelity reconstruction of fire scenes, making it possible to obtain geometric structural information and semantic attributes in complex environments through non-contact means. This ability to map from two-dimensional images to three-dimensional semantic space creates the prerequisites for establishing an intelligent fire hazard investigation system with spatial understanding capabilities, and promotes fire management towards having three-dimensional spatial cognition and quantitative assessment capabilities.

[0003] Currently, Chinese invention patent CN113239832B discloses a method and system for intelligent hazard identification based on image recognition, including: acquiring images of a stacked goods area, the images including safety passage signs, goods, and safety passages; performing image segmentation on the images to obtain a goods segmentation map and a safety passage sign segmentation map; obtaining the directional similarity of safety passages based on the angle between the safety passages and safety passage signs in the current area; obtaining the width compliance of safety passages by calculating the effective width of safety passages, and obtaining the passage standardization of safety passages by combining the directional similarity; calculating the goods stability based on the number and area of ​​the difference areas; and calculating the safety index of the current area based on the passage standardization and goods stability. This invention can assess the fire hazard risk caused by the disorderly stacking of goods in the current area, reminding enterprise staff to deal with it in time and prevent disasters. However, in the prior art, two-dimensional visual recognition technology is difficult to construct a complete three-dimensional semantic expression of the scene and cannot effectively use the three-dimensional spatial topological relationship between objects for deep reasoning. When judging whether the safety exit is blocked or whether the fire equipment is blocked, it often can only rely on the visual features of a single viewpoint, lacking accurate distance measurement and spatial orientation analysis based on three-dimensional voxels, resulting in a lack of spatial credibility in the recognition results. Existing hazard assessment methods mostly stay at the level of qualitative judgment, only outputting a binary result or simple label of whether there is a hazard, lacking a physical quantification mechanism for the severity of the hazard, and unable to calculate key indicators such as the specific incremental impact of blocked evacuation routes on the escape time of personnel. As a result, fire rectification decisions lack data support and it is difficult to achieve the leap from simple visual perception to in-depth physical assessment and decision support. Summary of the Invention

[0004] The technical problem solved by this invention is that two-dimensional visual recognition technology is difficult to construct a complete three-dimensional semantic expression of a scene, and cannot effectively utilize the three-dimensional spatial topological relationships between objects for deep reasoning. When judging whether a safety exit is blocked or whether fire-fighting equipment is obstructed, it often relies on single-view visual features and lacks accurate distance measurement and spatial orientation analysis based on three-dimensional voxels, resulting in a lack of spatial credibility in the recognition results. Existing hazard assessment methods mostly remain at the level of qualitative judgment, only outputting binary results or simple labels indicating whether a hazard exists, lacking a physical quantification mechanism for the severity of the hazard, and unable to calculate key indicators such as the specific incremental impact of blocked evacuation routes on personnel escape time. This results in a lack of data support for fire rectification decisions and makes it difficult to achieve the leap from simple visual perception to in-depth physical assessment and decision support.

[0005] To address the aforementioned technical problems, this invention provides the following technical solution: a fire safety hazard identification and labeling system based on visual understanding, comprising a construction module, a generation module, a reasoning module, and an evaluation module:

[0006] The construction module is used to acquire continuous images of fire scene, calculate the coordinates of spatial sampling points and the direction of target ray by combining camera pose parameters, calculate volume density value, color value and semantic category probability distribution by using multilayer perceptron network, and generate a three-dimensional semantic field through integral training.

[0007] The generation module is used to extract effective voxels and divide entity objects, calculate the distance and relative orientation angle between entities, match spatial relationship predicates, and construct a scene graph;

[0008] The reasoning module is used to input the entity category attributes and spatial topology relationships of the scene graph into a preset hidden danger judgment logic tree, traverse each node for matching, and obtain the output result;

[0009] The evaluation module is used to construct and solve the Eikonal equation based on the three-dimensional semantic field when the output result indicates the existence of a hidden danger, obtain the evacuation time field, calculate the evacuation time increment of the evacuation time field relative to the barrier-free benchmark scene, and output the hazard quantification value.

[0010] As a preferred embodiment of the fire safety hazard identification and labeling system based on visual understanding described in this invention, the construction module is used to acquire continuous images of the fire scene as input images;

[0011] Feature extraction and matching are performed on the input image, and camera pose parameters are calculated, including the camera center position and rotation matrix;

[0012] Based on the pixel coordinates of the input image and the preset camera intrinsic parameter matrix, the direction vector of each pixel in the camera coordinate system is calculated, and the direction vector is converted into the target ray direction in world coordinates according to the camera pose parameters.

[0013] Spatial location points are selected at equal intervals along the direction of the target ray, and the coordinates of each spatial location point are determined as the coordinates of the spatial sampling point.

[0014] The spatial sampling point coordinates and the target ray direction are input into a multilayer perceptron network to obtain the volume density value, color value and semantic category probability distribution corresponding to the spatial sampling point coordinates.

[0015] Based on the volume density value and color value, the spatial sampling points are integrated along the target ray direction to generate a predicted image;

[0016] Based on the volume density value and semantic category probability distribution, the spatial sampling points are integrated along the target ray direction to generate a predicted semantic map;

[0017] Calculate the mean square error of pixel color between the predicted image and the corresponding single frame image in the input image, and calculate the cross-entropy loss value of the semantic labels between the predicted semantic map and the corresponding single frame image in the input image.

[0018] Based on the mean squared error and cross-entropy loss value, the weight parameters of the multilayer perceptron network are updated using the backpropagation algorithm until the mean squared error is less than a preset error threshold, thereby generating a three-dimensional semantic field.

[0019] As a preferred embodiment of the fire safety hazard identification and labeling system based on visual understanding described in this invention, the generation module is used to sample within the spatial range of the three-dimensional semantic field according to a preset grid spacing to obtain grid sampling points and acquire the volume density value of each grid sampling point.

[0020] Grid sampling points with a volume density value greater than a preset density threshold are considered as valid voxels;

[0021] Extract the probability of each category from the semantic category probability distribution corresponding to each valid voxel, take the category corresponding to the largest probability value as the target semantic category, and write the label of the target semantic category into the attribute information of the valid voxel as the entity category attribute;

[0022] Traverse each valid voxel, group together valid voxels with the same semantic category and spatially adjacent, classify and identify them as the same entity object;

[0023] Traverse the valid voxel sets contained in each of the two entity objects, calculate the Euclidean distance between the nearest points of the two sets as the minimum distance, and calculate the horizontal projection angle of the line connecting the center points of the two entity objects in the world coordinate system.

[0024] The horizontal projection angle is taken as the relative azimuth angle;

[0025] If the minimum distance value is less than the preset distance threshold, the relative orientation angle and the vertical height difference between the two entity objects are input into the preset rule set to match and determine the spatial relationship predicate between the entity objects;

[0026] If the minimum distance value is greater than or equal to the preset distance threshold, it is determined that there is no spatial relationship between the entity objects, and the matching stops;

[0027] Each entity object is mapped to an independent node in the scene graph, and the spatial relationship predicate is used as the attribute information of the connecting edges of each entity object.

[0028] The independent nodes corresponding to the entity objects with spatial relation predicates are taken as associated node pairs, and directed connection edges are generated between each associated node pair.

[0029] The directed connecting edges are used to topologically associate the corresponding independent nodes to generate a graph data structure, which is then used as the scene graph.

[0030] As a preferred embodiment of the fire safety hazard identification and labeling system based on visual understanding described in this invention, the reasoning module is used to obtain the entity category attributes of entity objects in the scene graph and the spatial topological relationships between entity objects;

[0031] Input the entity category attributes and spatial topological relationships into a preset hidden danger judgment logic tree;

[0032] Obtain the root node of the preset hidden danger judgment logic tree as the current judgment node, and match the entity category attribute and spatial topology relationship with the judgment conditions stored in the current judgment node;

[0033] If the entity category attribute and spatial topology relationship satisfy the judgment condition of the current judgment node, then jump to the child node pointed to by the current judgment node, update the child node to the current judgment node, and repeat the matching until the leaf node of the preset hidden danger judgment logic tree is reached.

[0034] If the entity category attribute and spatial topology relationship do not meet the judgment conditions of the current judgment node, then jump to the negative branch node pointed to by the current judgment node, update the negative branch node to the current judgment node, and repeat the matching until the leaf node of the preset hidden danger judgment logic tree is reached.

[0035] Read the conclusion identifier stored in the leaf node, generate a Boolean value indicating whether there is a fire safety hazard and a corresponding hazard type label based on the conclusion identifier, and output the Boolean value and the corresponding hazard type label.

[0036] As a preferred embodiment of the visual understanding-based fire safety hazard identification and labeling system of the present invention, the evaluation module is used to receive the output results;

[0037] If the Boolean value indicates a potential risk, then read the volume density value in the three-dimensional semantic field;

[0038] A three-dimensional velocity field is constructed based on the volume density value in the three-dimensional semantic field. Voxels whose semantic categories belong to the preset safety exit category are extracted from the three-dimensional semantic field, and the set of spatial coordinates of the voxels belonging to the preset safety exit category is used as the safety exit location.

[0039] The potential energy value of the safety exit location in the potential field calculation space is set to zero, which serves as a zero potential energy boundary condition.

[0040] Based on the three-dimensional velocity field and the zero potential energy boundary condition, the Eikonal equation is constructed. The evacuation time field is obtained by solving the Eikonal equation. The evacuation time increment of the evacuation time field relative to the barrier-free reference scene is calculated, and the hazard quantification value is output.

[0041] As a preferred embodiment of the visual understanding-based fire safety hazard identification and labeling system of the present invention, the logic of constructing a three-dimensional velocity field based on the volume density value in the three-dimensional semantic field includes:

[0042] The volume density values ​​in the volume density distribution data are mapped to spatial movement speed values, wherein the spatial movement speed values ​​in the obstacle area are set to zero.

[0043] The volume density values ​​in the three-dimensional semantic field are mapped to spatial movement speed values, and the regions in the three-dimensional semantic field with volume density values ​​greater than the preset obstacle density are designated as obstacle regions, and the spatial movement speed values ​​of the obstacle regions are set to zero.

[0044] As a preferred embodiment of the visual understanding-based fire safety hazard identification and labeling system described in this invention, the logic for solving the Eikonal equation to obtain the evacuation time field includes:

[0045] Based on the coordinate range of the three-dimensional semantic field, the three-dimensional space is discretized into a grid, and the discrete grid nodes are used as initial nodes and added to the sequence of nodes to be solved.

[0046] Repeatedly propagate the time between the grid nodes until the sequence of nodes to be solved is empty. Then, map the time attribute values ​​stored in each discrete grid node back to the three-dimensional space to generate an evacuation time field.

[0047] As a preferred embodiment of the visual understanding-based fire safety hazard identification and labeling system of the present invention, the logic of time propagation of the grid nodes includes:

[0048] The discrete grid node with the smallest time attribute value is extracted from the sequence of nodes to be solved as the currently determined node. Each spatial neighbor node of the currently determined node is obtained as the node to be updated. For each node to be updated, the finite difference scheme of the first-order upwind scheme is used. Based on the time attribute values ​​of the determined neighbor nodes in each direction of the node to be updated, combined with the spatial movement velocity value of the node to be updated in the three-dimensional velocity field, the first-order difference discrete equation corresponding to the Eikonal equation is solved based on the fast travel method to obtain the candidate arrival time of the node to be updated.

[0049] Obtain the spatial neighbor nodes of the currently processed discrete grid node as the nodes to be updated;

[0050] The cumulative arrival time from each node to be updated to the safety exit location is calculated based on the local travel time to obtain the estimated time, and the time attribute value of the corresponding node to be updated is updated using the estimated time.

[0051] Based on the order of the expected times, the nodes to be updated that have been updated are dynamically added to the sequence of nodes to be solved, resulting in the updated queue of nodes to be solved.

[0052] As a preferred embodiment of the visual understanding-based fire safety hazard identification and labeling system of the present invention, the logic for calculating the evacuation time increment of the evacuation time field relative to the barrier-free reference scene includes:

[0053] Based on the three-dimensional semantic field, the semantic category probability distribution in the spatial sampling point coordinates is mapped to the spatial location of the wall category, which is then used as the wall obstacle area;

[0054] Set the movement speed of all spatial locations in the three-dimensional semantic field, except for the wall obstacle area, to a preset maximum movement speed to generate an accessible baseline scene.

[0055] Solve the Eikonal equations for the barrier-free baseline scenario to calculate the baseline evacuation time field;

[0056] Calculate the numerical difference between the evacuation time field and the reference evacuation time field at the same spatial location;

[0057] If the numerical difference is greater than a preset difference threshold, then the numerical difference is used as the evacuation time increment of the spatial location.

[0058] If the numerical difference is less than or equal to a preset difference threshold, the evacuation time increment of the spatial location is marked as zero.

[0059] As a preferred embodiment of the visual understanding-based fire safety hazard identification and labeling system of the present invention, the logic for outputting the hazard quantification value includes:

[0060] The spatial locations where the evacuation time increment is greater than a preset time threshold are extracted to form the obstructed area;

[0061] Calculate the product of the volume of the obstructed area and the maximum evacuation time increment within the obstructed area to obtain the product value;

[0062] The product value is used as a quantification of the severity of the hidden danger and output.

[0063] The beneficial effects of this invention are as follows: By constructing a three-dimensional semantic field containing geometric, color, and semantic information, this invention effectively overcomes the shortcomings of two-dimensional visual recognition technology in constructing a complete three-dimensional semantic expression of a scene. This enables the system to perform accurate distance measurement and spatial orientation analysis based on three-dimensional voxels. When judging potential hazards such as blocked safety exits or obstructed fire-fighting equipment, it is no longer limited by single-view visual features but utilizes the three-dimensional spatial topological relationships between entities for deep reasoning, significantly improving the spatial credibility and accuracy of the recognition results. This invention breaks through the bottleneck of existing hazard assessment methods that only remain at the qualitative judgment level. By solving the Eikonal equation to construct an evacuation time field, it achieves the physical quantification of the severity of hazards and can accurately calculate the specific incremental impact of blocked evacuation routes on personnel escape time. This innovation transforms abstract safety hazards into intuitive time cost data, providing a solid scientific basis for fire rectification decisions. Thus, in practical applications, it achieves a leap from simple visual perception to in-depth physical assessment and decision support, greatly improving the level of precision in fire management and the actual effectiveness of hazard control. Attached Figure Description

[0064] Figure 1 This is a schematic diagram of the basic process of a visual understanding-based fire safety hazard identification and labeling system provided in one embodiment of the present invention. Detailed Implementation

[0065] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0066] Reference Figure 1 As one embodiment of the present invention, a fire safety hazard identification and labeling system based on visual understanding is provided, including a construction module, a generation module, a reasoning module, and an evaluation module:

[0067] The module is used to acquire continuous images of fire scene, calculate the coordinates of spatial sampling points and the direction of target ray by combining camera pose parameters, calculate volume density value, color value and semantic category probability distribution by using multilayer perceptron network, and generate a three-dimensional semantic field through integral training;

[0068] The generation module is used to extract valid voxels and divide entity objects, calculate the distance and relative orientation angle between entities, match spatial relationship predicates, and construct scene graphs;

[0069] The reasoning module is used to input the entity category attributes and spatial topology relationships of the scene graph into the preset hidden danger judgment logic tree, traverse each node for matching, and obtain the output results;

[0070] The assessment module is used to construct and solve the Eikonal equation based on the three-dimensional semantic field when the output indicates the existence of a hazard, obtain the evacuation time field, calculate the evacuation time increment of the evacuation time field relative to the barrier-free baseline scene, and output the hazard quantification value.

[0071] This invention generates a three-dimensional semantic field through a construction module, achieving accurate mapping from two-dimensional images to three-dimensional spatial semantic features. Compared with existing methods based on two-dimensional image recognition, it solves the problems of limited perspective and lack of spatial depth information. The generation module and the reasoning module combine spatial topological relationships to perform logic tree matching, which can accurately identify fire safety hazards in complex scenarios. The evaluation module introduces the Eikonal equation to calculate the evacuation time field and the quantitative value of hazards, breaking through the limitations of existing technologies that only focus on qualitative identification, realizing quantitative assessment of the degree of hazard, and greatly improving the comprehensiveness and application value of fire safety hazard judgment.

[0072] The module is used to acquire continuous images of the fire scene as input images;

[0073] Feature extraction and matching are performed on the input image, and camera pose parameters are calculated, including the camera center position and rotation matrix.

[0074] This system preprocesses and extracts features from the input continuous images of a fire scene. Specifically, it uses feature extraction networks such as SuperPoint to obtain the key point coordinates and high-dimensional feature descriptors of each frame. It then uses feature matching networks such as SuperGlue or the nearest neighbor algorithm combined with ratio testing to establish two-dimensional pixel correspondence pairs between adjacent frames. Based on these matching point pairs, it uses the RANSAC algorithm to remove mismatched outliers and calculates the fundamental matrix using the eight-point method. It then converts the fundamental matrix into an essential matrix using a preset camera intrinsic matrix. The essential matrix is ​​then subjected to singular value decomposition (SVD) and positive depth constraints to recover the relative rotation transformation and translation vectors between adjacent frames. The system performs a series of multiplications and accumulations using the coordinate system of the first frame as the world coordinate system to accurately solve for the absolute camera pose parameters of each frame. For each pixel in the image, it is back-projected to the camera coordinate system using the inverse matrix of the camera intrinsic matrix to obtain the direction vector. Combined with the rotation matrix calculated above, the direction vector is transformed to the world coordinate system to generate the target ray direction. This establishes an accurate geometric mapping foundation for subsequent three-dimensional spatial sampling along the ray and feature calculation by the multilayer perceptron network.

[0075] Based on the pixel coordinates of the input image and the preset camera intrinsic parameter matrix, the direction vector of each pixel in the camera coordinate system is calculated, and the direction vector is converted into the target ray direction in the world coordinate system according to the camera pose parameters.

[0076] The preset camera intrinsic parameter matrix is ​​used to characterize the optical geometric properties of the camera itself. In this embodiment, it is specifically set as a 3×3 upper triangular matrix. Its internal elements are strictly assigned values ​​according to the factory calibration parameters of the actual acquisition equipment. Specifically, it includes horizontal focal length parameters, vertical focal length parameters, horizontal coordinate parameters of image principal points, vertical coordinate parameters of image principal points, and scale factor.

[0077] Spatial location points are selected at equal intervals along the direction of the target ray, and the coordinates of each spatial location point are determined as the coordinates of the spatial sampling point;

[0078] Input the spatial sampling point coordinates and the target ray direction into the multilayer perceptron network to obtain the volume density value, color value and semantic category probability distribution corresponding to the spatial sampling point coordinates;

[0079] Based on the volume density and color values, the spatial sampling points are integrated along the target ray direction to generate a predicted image;

[0080] Based on the volume density value and semantic category probability distribution, the spatial sampling points are integrated along the target ray direction to generate a predicted semantic map;

[0081] The specific processing steps for integrating spatial sampling points along the target ray direction are as follows:

[0082] Equally spaced spatial sampling points are extracted along the target ray direction. The volume density value, color value, and semantic category probability distribution corresponding to each sampling point are obtained through a multilayer perceptron network. During the integration calculation, the probability of light being blocked when passing through the sampling point is first calculated based on the spatial distance between adjacent sampling points and the volume density value of the current sampling point. The unblocked probability of all previous sampling points is accumulated along the ray direction from near to far to calculate the remaining light intensity when the light reaches the current sampling point. The cumulative transmittance of the current sampling point is multiplied by the single-point transparency as the final weight of the sampling point. The color value and semantic category probability distribution of the sampling point are weighted and summed using this weight. Finally, the weighted summation result is mapped to generate the pixel color value of the predicted image and the semantic label of the predicted semantic map, respectively. The process constructs a weighted causal chain through explicit physical occlusion relationships, realizing the accurate differentiable mapping of three-dimensional spatial voxel features to the two-dimensional image plane.

[0083] Calculate the mean square error of pixel color between the predicted image and the corresponding single frame image in the input image, and calculate the cross-entropy loss value of the semantic labels between the predicted semantic map and the corresponding single frame image in the input image.

[0084] The specific logic for calculating the mean square error of pixel color between the predicted image and the corresponding single frame image in the input image is as follows:

[0085] A pixel-by-pixel comparison is performed between the predicted image generated by volume rendering integration and the actual input image. For each pixel in the image, the difference between its predicted color value and its actual color value in each color channel is calculated and squared. Then, the average of the squared differences of each channel for the same pixel is taken as the error value of that single point. Based on this, the single-point error values ​​of all pixels in a single frame are accumulated and the arithmetic mean is calculated to obtain the overall pixel color mean square error of the frame. This error value reflects the fidelity of the rendered image under the current weights of the 3D semantic field model in a physical sense. The smaller the value, the closer the predicted color is to the real scene. The network weights are continuously updated through the backpropagation algorithm with the goal of reducing this mean square error until the error value converges to within the preset error threshold, thereby ensuring that the generated 3D semantic field has a high-precision visual feature restoration capability.

[0086] The specific logic for calculating the cross-entropy loss value is as follows:

[0087] For each pixel in the image, the predicted semantic probability distribution vector generated by the volume rendering integral weighting is compared with the pre-labeled real fire semantic category label. The predicted probability value of the category corresponding to the real label is extracted, and the negative logarithm of the probability value is taken as the classification loss value of the current pixel. The classification loss values ​​of all pixels in a single frame are accumulated and the arithmetic mean is calculated to obtain the cross-entropy loss value of the entire frame image.

[0088] Based on the mean squared error and cross-entropy loss values, the weight parameters of the multilayer perceptron network are updated using the backpropagation algorithm until the mean squared error is less than a preset error threshold, thereby generating a three-dimensional semantic field.

[0089] Considering the stringent requirements of fire scene for image color fidelity and hazard target location accuracy, the preset error threshold is specifically set to 0.005. During iterative training, the mean square error value on the validation set is monitored in real time. When the mean square error steadily decreases and falls below the preset threshold of 0.005 after several consecutive iterations, it is determined that the network has fully converged. At this point, backpropagation and weight update operations are immediately terminated. By clearly setting this quantitative convergence standard, a three-dimensional semantic field model containing high-precision scene geometry, realistic appearance color, and accurate fire semantic category labels is solidified, ensuring that it meets the accuracy requirements of subsequent three-dimensional spatial sampling and semantic feature query.

[0090] In each training iteration, the total loss function is calculated based on the predicted data output by forward propagation. Then, the gradient data of the weight matrix and bias terms of each layer are iteratively updated using the backpropagation algorithm combined with the AdamW optimizer according to the total loss function, so that the network's predicted features converge to the real scene data. The mean square error of pixel color between the rendered predicted image and the real image is monitored in real time on the validation set. When the system detects that the mean square error value decreases steadily with the number of iterations and is continuously less than the preset error threshold, it is determined that the multilayer perceptron network has reached a fully converged state. At this time, the system automatically terminates the backpropagation and parameter update operation, and solidifies the network weight parameters, bias terms and network topology at the current moment, thereby generating a three-dimensional semantic field model with high-precision spatial geometry, realistic appearance color and accurate fire-fighting semantic labels.

[0091] In this embodiment, the multilayer perceptron network adopts an 8-layer fully connected layer structure, with each layer using the ReLU activation function. The network input consists of the three-dimensional coordinates of the spatial sampling points and the direction of the target ray. Position encoding is used to improve the ability to capture high-frequency details. The output layer is divided into three branches, which output the volume density value, the three-channel color value, and the multi-channel semantic category probability distribution, respectively. In the volume rendering integration process, discrete summation is used to approximate continuous integration. The pixel color of the predicted image is obtained by multiplying the color value of each sampling point along the ray by the transmittance and accumulating it. The transmittance is calculated from the volume density value and distance interval of adjacent sampling points. The calculation method of the predicted semantic map is similar, replacing the color value with the semantic category probability distribution and accumulating it.

[0092] This invention utilizes a multilayer perceptron network to simultaneously calculate volume density values, color values, and semantic category probability distributions. It also drives network parameter updates through a combination of mean squared error and cross-entropy loss. Compared to existing single-method geometric reconstruction or semantic segmentation schemes, this invention generates a 3D semantic field that combines high-fidelity geometric structure with rich semantic information. This feature extraction and matching mechanism effectively reduces interference from illumination variations and image noise, significantly improving the accuracy and robustness of 3D spatial semantic reconstruction in complex fire-fighting scenarios, and providing highly reliable underlying data support for subsequent hazard identification.

[0093] The generation module is used to sample within the spatial range of the three-dimensional semantic field according to a preset grid spacing to obtain grid sampling points and acquire the volume density value of each grid sampling point;

[0094] The preset grid spacing is specifically set to 0.05 meters. In this embodiment, the spacing is used as the spatial step size to establish a uniformly distributed three-dimensional sampling grid within the three-dimensional bounding box of the generated three-dimensional semantic field. The coordinates of the center point of each grid voxel are extracted sequentially, and the volume density value and semantic category probability distribution output by the multilayer perceptron network are queried. Through grid sampling with a fixed spacing, the system transforms the continuous implicit neural field into a discrete set of voxel features. The setting of 0.05 meters ensures sufficient spatial resolution to identify small hidden objects such as fire hydrants and fire extinguisher handles, while avoiding computational redundancy caused by excessively dense sampling.

[0095] Grid sampling points with a volume density value greater than a preset density threshold are considered as valid voxels;

[0096] The preset density threshold is specifically set to 0.5. In this embodiment, the system reads the volume density value of the grid sampling points and compares it with the threshold. If the volume density value of a spatial point is greater than or equal to 0.5, the point is determined to be an actual physical entity and its semantic attributes are retained. If the volume density value is less than 0.5, it is determined to be an airborne noise point or an empty area and is removed. Through this hard numerical truncation operation, the system extracts a high-confidence three-dimensional point cloud of entities from the continuous field, ensuring the physical objectivity of subsequent fire hazard analysis.

[0097] Extract the probability of each category from the semantic category probability distribution corresponding to each valid voxel, take the category corresponding to the largest probability value as the target semantic category, and write the label of the target semantic category into the attribute information of the valid voxel as the entity category attribute;

[0098] Traverse each valid voxel, group together valid voxels with the same semantic category and spatially adjacent, classify and identify them as the same entity object;

[0099] Traverse the valid voxel sets contained in each of the two entity objects, calculate the Euclidean distance between the nearest points of the two sets as the minimum distance, and calculate the horizontal projection angle of the line connecting the center points of the two entity objects in the world coordinate system.

[0100] Use the horizontal projection angle as the relative azimuth angle;

[0101] If the minimum distance value is less than the preset distance threshold, the relative orientation angle and the vertical height difference between the two entity objects are input into the preset rule set to match and determine the spatial relationship predicate between the entity objects;

[0102] The preset distance threshold is specifically set to 0.50 meters. In this embodiment, after the system completes the division of entity objects, it traverses the effective voxel sets of two entity objects to calculate the Euclidean distance between the nearest points as the minimum distance. The line connecting the coordinates of the two center points is projected onto the horizontal plane of the world coordinate system, and the angle between this projection vector and the due east direction is calculated as the relative azimuth angle. Simultaneously, the height difference between the center points of the two entity objects in the vertical direction is calculated. When the minimum distance value is less than 0.50 meters, the relative azimuth angle and height difference are used as features input into a preset rule set to match and determine the spatial relationship predicate between entity objects. The specific setting of these distance and angle parameters provides a quantitative judgment standard that conforms to actual physical dimensions for the accurate construction of spatial topological relationships in the scene graph.

[0103] The preset rule set is specifically set as a spatial topological relationship knowledge base built based on IF-THEN conditional logic. In this embodiment, the system uses the calculated minimum distance value comparison results between entities and the calculated relative orientation angle results as factual input features, and sequentially matches them with the judgment clauses in the rule set. For example, when the system determines that the minimum distance value between two entity objects is less than a preset distance threshold and the relative orientation angle meets a specific interval, the rule set outputs the corresponding spatial relationship predicate label. Through this explicitly set logical rule set, the system can accurately quantify and identify the relative orientation and topological relationship between entities in three-dimensional space, realizing a precise mapping from underlying geometric coordinate data to standardized spatial relationship semantics, and providing a structured foundation for the subsequent construction of a scene graph with context-aware capabilities.

[0104] If the minimum distance value is greater than or equal to the preset distance threshold, it is determined that there is no spatial relationship between the entity objects, and the matching stops;

[0105] Each entity object is mapped to an independent node in the scene graph, and the spatial relationship predicate is used as the attribute information of the connecting edges of each entity object.

[0106] The independent nodes corresponding to the entity objects with spatial relation predicates are taken as associated node pairs, and directed connection edges are generated between each associated node pair.

[0107] By connecting the corresponding independent nodes with directed edges, a graph data structure is generated, which is then used as the scene graph.

[0108] In this embodiment, sampling is performed within a preset grid spacing in the spatial range of the three-dimensional semantic field. Grid sampling points with a volume density value greater than a preset density threshold are taken as valid voxels. The category with the highest probability of valid voxels is extracted as the target semantic category and written into the attribute information. Valid voxels are traversed, and a breadth-first search algorithm is used to group valid voxels with the same semantic category and spatially adjacent hexahedral positions, identifying them as the same entity object. The valid voxel sets of two entity objects are traversed, and the Euclidean distance between the closest point pairs between the two sets is calculated as the minimum distance. The line connecting the center points is projected onto the horizontal plane of the world coordinate system, and the angle between the projection vector and the due east direction is calculated as the relative azimuth angle. If the minimum distance is less than a preset distance threshold and the relative azimuth angle is greater than a preset angle threshold, then the spatial relationship predicate between entity objects is determined by matching from a preset rule set. Entity objects are mapped as independent nodes, and nodes with spatial relationship predicates are taken as associated node pairs. Directed connecting edges are generated, and a graph data structure is generated by combining the nodes and edges as the scene graph.

[0109] This invention, by filtering effective voxels and segmenting entity objects based on semantic connectivity, can accurately extract key items and their spatial attributes in a scene. By calculating Euclidean distance and the horizontal projection angle to match spatial relationship predicates, compared to existing technologies that rely on simple overlapping bounding boxes, it can more realistically reflect the relative orientation and distance relationships between objects in 3D space. The graph data structure constructed in this way fully preserves the topological logic of the scene, enabling the hazard assessment process to have spatial context awareness, effectively avoiding misjudgments and omissions caused by isolated identification.

[0110] The reasoning module is used to obtain the entity category attributes of entity objects in the scene graph and the spatial topological relationships between entity objects;

[0111] Spatial topology refers to the edges in a scene graph and how they are connected.

[0112] Input the entity category attributes and spatial topology relationships into the preset hidden danger judgment logic tree;

[0113] Obtain the root node of the preset hidden danger judgment logic tree as the current judgment node, and match the entity category attributes and spatial topology relationships with the judgment conditions stored in the current judgment node;

[0114] If the entity category attribute and spatial topology relationship meet the judgment conditions of the current judgment node, then jump to the child node pointed to by the current judgment node, update the child node to the current judgment node, repeat the matching until the leaf node of the preset hidden danger judgment logic tree is reached.

[0115] If the entity category attribute and spatial topology relationship do not meet the judgment conditions of the current judgment node, then jump to the negative branch node pointed to by the current judgment node, update the negative branch node to the current judgment node, repeat the matching until the leaf node of the preset hidden danger judgment logic tree is reached.

[0116] Read the conclusion identifier stored in the leaf node, generate a Boolean value indicating whether there is a fire safety hazard and the corresponding hazard type label based on the conclusion identifier, and output the Boolean value and the corresponding hazard type label.

[0117] In this embodiment, the preset hazard judgment logic tree adopts a binary tree structure. Non-leaf nodes store judgment conditions for entity category attributes and spatial topology relationships. The entity category attributes and spatial topology relationships in the scene graph are obtained as input to the logic tree. The root node is used as the current judgment node. If the judgment condition stored in the current judgment node is to determine whether the entity category is a fire extinguisher and whether the spatial topology relationship is occluded, the input data jumps to the pointed child node when the condition is met, and jumps to the pointed negative branch node when the condition is not met. This matching process is repeated until the leaf node is reached. The conclusion identifier stored in the leaf node includes the hazard existence status and hazard classification code. After reading the conclusion identifier, it is converted into a Boolean value of 0 representing safety or a Boolean value of 1 representing hazard, and the corresponding hazard type label is output.

[0118] This invention employs a pre-defined hazard identification logic tree to traverse and match entity category attributes and spatial topological relationships. Compared to existing hazard identification schemes based on black-box deep learning classifiers, this gives the system extremely strong interpretability. The layer-by-layer identification process of the logic tree fully complies with fire safety inspection standards, making the identification path of each hazard output traceable. When fire safety standards are updated, only the logic tree nodes need to be adjusted for quick adaptation, significantly reducing system maintenance costs and improving the transparency and reliability of the hazard identification process.

[0119] The evaluation module is used to receive the output results;

[0120] If the Boolean value indicates a potential risk, then the volume density value in the three-dimensional semantic field is read.

[0121] If the received boolean value is 0, it means that there is no fire safety hazard in the current scene. The evaluation module will directly terminate the calculation process and will not read the three-dimensional semantic field, thereby greatly saving the system's computing resources.

[0122] If the received Boolean value is 1, it indicates that there is a fire safety hazard in the current scene. The assessment module then triggers the quantitative assessment process and begins to read the volume density value in the three-dimensional semantic field.

[0123] A three-dimensional velocity field is constructed based on the volume density values ​​in the three-dimensional semantic field. Voxels whose semantic categories belong to the preset safety exit category are extracted from the three-dimensional semantic field, and the set of spatial coordinates of voxels belonging to the preset safety exit category is used as the safety exit location.

[0124] The preset safety exit category is specifically set as an independent high-priority label in the multi-classification task, and its corresponding category index is explicitly set as the safety exit identifier. In this embodiment, when the system performs volume rendering integral calculation on spatial sampling points along the target ray direction, the multilayer perceptron network not only outputs the color and volume density of each point, but also outputs the probability value of the point belonging to the safety exit category. The system performs volume rendering weighted accumulation based on the probability distribution of this category, and accurately separates the pixel region with the "safety exit" label from the background in the generated predictive semantic map. In the post-processing 3D spatial mesh traversal stage, the system uses the preset category label to further divide the entity point cloud retained after the volume density is truncated by a preset density threshold into safety exit point cloud clusters and other obstacle point cloud clusters. By explicitly setting this preset safety exit category, the system provides the necessary pre-semantic judgment benchmark for subsequent calls to preset distance thresholds and preset rule sets, ensuring the rigor and closed loop of the fire hazard logical reasoning chain.

[0125] The potential energy value of the safety exit location in the potential field computation space is set to zero, which serves as the zero potential energy boundary condition.

[0126] The Eikonal equation is constructed based on the three-dimensional velocity field and zero potential energy boundary conditions. The evacuation time field is obtained by solving the Eikonal equation. The evacuation time increment of the evacuation time field relative to the barrier-free baseline scene is calculated, and the hazard quantification value is output.

[0127] The calculation formula for the Eikonal equation, based on the three-dimensional velocity field and zero potential energy boundary conditions, is as follows:

[0128] ;

[0129] in, , This represents the minimum evacuation arrival time field from each point in space to the safe exit. Let be the spatial gradient magnitude of this time field. For a three-dimensional computational domain, This represents the velocity value of the person moving at spatial location x in the three-dimensional velocity field.

[0130] In this embodiment, a three-dimensional velocity field is constructed based on the volume density value in the three-dimensional semantic field. The set of voxel coordinates belonging to the semantic category of safety exit is extracted, and their potential energy value is set to zero as a zero potential energy boundary condition. The Eikonal equation is constructed, with the left side of the equation being the magnitude of the gradient of the evacuation time field and the right side being the reciprocal of the velocity value at the corresponding position in the three-dimensional velocity field. The evacuation time field is obtained by solving the equation using the fast travel method. The barrier-free benchmark scene is set as a scene that only includes wall obstacle areas and the velocity of other areas is the maximum movement speed. The benchmark evacuation time field is obtained by solving the Eikonal equation of the barrier-free benchmark scene. The difference between the evacuation time field and the benchmark evacuation time field at the same spatial position is calculated. The part of the difference that is greater than the preset difference threshold is taken as the evacuation time increment and the hazard quantification value is output.

[0131] This invention constructs the Eikonal equation based on a three-dimensional semantic field and solves the evacuation time field, directly linking hazard identification with personnel evacuation and escape. Compared with existing technologies that only mark the location of hazards, this invention can intuitively show the degree of impact of hazards on personnel evacuation paths and time. By calculating the evacuation time increment relative to an accessible baseline scenario and outputting a quantitative value of the hazard, it provides a scientific and intuitive numerical basis for the deployment of fire and rescue forces and the prioritization of hazard rectification, realizing closed-loop management from hazard discovery to hazard assessment.

[0132] The logic for constructing a three-dimensional velocity field based on the volume density values ​​in a three-dimensional semantic field includes:

[0133] The volume density values ​​in the volume density distribution data are mapped to spatial movement speed values, where the spatial movement speed values ​​in the obstacle area are set to zero.

[0134] The volume density values ​​in the three-dimensional semantic field are mapped to spatial movement speed values, and the regions in the three-dimensional semantic field with volume density values ​​greater than the preset obstacle density are designated as obstacle regions, and the spatial movement speed values ​​of the obstacle regions are set to zero.

[0135] The preset obstacle density is specifically set to 0.5. In this embodiment, the volume density value of each spatial location in the three-dimensional semantic field is read and compared with the preset obstacle density of 0.5. Since the volume density value reflects the probability and density of a spatial point belonging to a physical object in the neural radiation field, when the volume density value is greater than or equal to 0.5, the system determines that the spatial area is tightly occupied by physical objects and cannot be penetrated by personnel. Therefore, it is mapped to a spatial movement speed value of 0 and marked as an absolute obstacle area. When the volume density value is less than 0.5, the system determines that the area is a passable space or a sparse obstacle edge with low density. Through the truncation and mapping operation of this density threshold, the system accurately transforms the continuous volume density distribution in the three-dimensional semantic field into discrete velocity constraints that conform to the physical passage boundary of personnel. This not only eliminates the interference of floating noise on path planning, but also ensures the physical objectivity and conservatism of obstacle boundary determination, providing an accurate velocity field parameter basis for the subsequent solution of the Eikonal equation.

[0136] In this embodiment, a piecewise linear function is used to map the volume density values ​​in the three-dimensional semantic field to spatial movement velocity values. When the volume density value is between 0 and a preset obstacle density of 0.5, the spatial movement velocity value decreases linearly with the increase of the volume density value, simulating the deceleration behavior of people in crowded or complex environments. Regions with volume density values ​​greater than the preset obstacle density of 0.5 are designated as obstacle regions, and the spatial movement velocity value in these regions is forcibly set to zero, indicating that people cannot pass through physical obstacles. Through this mapping logic, the volume density distribution data of the three-dimensional semantic field is transformed into a three-dimensional velocity field that conforms to the physical laws of human movement.

[0137] This invention directly maps volume density values ​​to spatial movement velocity values ​​and sets the velocity of obstacle areas to zero. Compared to existing technologies that use fixed grid occupancy states, this method more delicately reflects the gradual influence of obstacle boundaries on evacuation velocity in a real environment. This velocity field construction method based on continuous volume density distribution avoids the step effect caused by the binary grid in existing technologies, making the subsequent calculation of the evacuation time field closer to the physical laws of real human movement and significantly improving the accuracy of hazard quantitative assessment.

[0138] The logic for solving the Eikonal equation to obtain the evacuation time field includes:

[0139] The three-dimensional space is discretized based on the coordinate range of the three-dimensional semantic field, and the resulting discrete grid nodes are used as initial nodes and added to the sequence of nodes to be solved.

[0140] Repeatedly propagate the time between the grid nodes until the sequence of nodes to be solved is empty. Then, map the time attribute values ​​stored in each discrete grid node back to the three-dimensional space to generate an evacuation time field.

[0141] A priority queue is used as the data structure for the sequence of nodes to be solved. The queue is automatically sorted according to the time attribute value of the nodes from smallest to largest, and the time propagation of the grid nodes is repeated: in each round of the loop, the node with the smallest time attribute value at the head of the queue is extracted as the current active node, and the time attribute values ​​of its spatially adjacent grid nodes are updated according to the finite difference scheme of the Eikonal equation; if the expected arrival time of the adjacent node is successfully optimized and updated, it is dynamically inserted into the queue according to the new time attribute value.

[0142] Due to the characteristics of the fast-moving method, once a node is extracted as the currently active node, its time attribute value is locked as the shortest evacuation arrival time at that location and no longer participates in subsequent updates. When all reachable grid nodes in the queue have been extracted and processed, i.e., when the state of the node sequence to be solved is empty, the loop propagation process terminates. The data structure of each discrete grid node has stored the minimum time attribute value for moving from that node location to the safe exit. A spatial mapping operation is performed: based on the coordinate correspondence during grid discretization, the time attribute value stored at the center point of each discrete grid node is directly assigned back to the corresponding voxel position in the 3D semantic field according to its 3D spatial coordinates. For voxels in unreachable or zero-velocity obstacle areas, their time attribute values ​​are retained as infinity. Through this mapping process, the originally discrete grid node time data is reconstructed into a 3D evacuation time field that continuously covers the entire 3D computational field, thus providing a complete and accurate data foundation for subsequent comparison with an accessible benchmark scene and extraction of evacuation time increments.

[0143] In this embodiment, based on the coordinate range of the three-dimensional semantic field, the three-dimensional space is discretized using a grid spacing of 5 mm to obtain discrete grid nodes. Each discrete grid node contains spatial coordinates and an initial time attribute value, which is set to infinity. The time attribute value of the discrete grid node corresponding to the safety exit location is set to zero and added as the initial node to the sequence of nodes to be solved. The time propagation of the grid nodes is repeated. Each time, a node is taken from the sequence for processing and adjacent nodes are updated. The propagation ends when no new nodes are added to the sequence and all nodes have been processed, i.e., when the sequence of nodes to be solved is empty. The time attribute values ​​stored in each discrete grid node are mapped back to the three-dimensional space according to the coordinates to generate the evacuation time field.

[0144] This invention employs a mesh discretization method combined with cyclic propagation of the sequence of nodes to be solved to solve the Eikonal equation. Compared to conventional full-matrix numerical solution methods, this significantly reduces computational complexity and memory consumption. The mechanism of dynamically processing the nodes to be solved and updating their temporal attribute values ​​ensures the monotonicity and convergence of the temporal field propagation. This enables the rapid processing of large-scale 3D firefighting scenarios while maintaining computational accuracy, effectively improving the system's real-time response capability for assessing potential hazards in complex environments.

[0145] The logic for time propagation of grid nodes includes:

[0146] Extract the discrete grid node with the smallest time attribute value from the sequence of nodes to be solved as the currently determined node. Obtain each spatial neighbor node of the currently determined node as the node to be updated. For each node to be updated, use the finite difference scheme of the first-order upwind scheme, based on the time attribute values ​​of the determined neighbor nodes in each direction of the node to be updated, combined with the spatial movement velocity value of the position of the node to be updated in the three-dimensional velocity field, and solve the first-order difference discrete equation corresponding to the Eikonal equation based on the fast travel method to obtain the candidate arrival time of the node to be updated.

[0147] The potential energy gradient value is the gradient magnitude of the evacuation time field in three-dimensional space. Since the three-dimensional evacuation time field has been discretized into uniformly distributed three-dimensional grid voxels, the system uses a first-order upwind scheme to construct differential discretization equations in the three spatial coordinate axes. For the discrete grid node to be updated, the system selects the node with the smaller and determined time attribute value among its positive and negative adjacent nodes in each coordinate axis direction as the upwind reference value in that direction. The upwind reference values ​​in the three directions are substituted into the quadratic discretization scheme of the Eikonal equation for simultaneous solution to obtain the candidate arrival time of the node to be updated. Through this finite difference calculation method based on discrete grid neighborhood information, the system transforms the differential operation of the continuous field into a numerical iterative operator that can be executed efficiently by a computer. While ensuring the accuracy of spatial gradient solution, it meets the efficiency requirements of real-time calculation of large-scale three-dimensional grid fields.

[0148] Based on the physical characteristics of the governing equations, the travel time between two points in space is equal to their spatial distance divided by the velocity along that path. When updating the time attribute values ​​of the currently processed discrete grid nodes using the fast travel method, the system decomposes its travel path into multiple one-dimensional sub-paths along the three-dimensional coordinate axes. Specifically, the system obtains the nearest known neighboring nodes of the current node in the positive and negative directions of the three spatial coordinate axes, and extracts the minimum evacuation time attribute value of these neighboring nodes as the reference time. The system calculates the spatial Euclidean distance between the current node and each neighboring node. In a uniform grid, this distance is one grid step size, and the system reads the value corresponding to the midpoint of the path between these two points. The system calculates the local travel time from the adjacent node to the current node by dividing the spatial distance by the corresponding midpoint travel speed. The system adds the time base value of the adjacent node to the local travel time to obtain the candidate arrival time to the current node via that direction. The system compares the candidate arrival times calculated for all coordinate axis directions and selects the minimum value as the final updated evacuation time attribute value for the current node. Through this differential calculation based on the local velocity field and the grid step size, the system accurately quantifies the time cost of personnel moving step by step along each grid path in complex three-dimensional space, ensuring the physical accuracy of the global evacuation time field numerical solution.

[0149] Obtain the spatial neighbor nodes of the currently processed discrete grid node as the nodes to be updated;

[0150] The cumulative arrival time from each node to be updated to the safe exit location is calculated based on the local travel time to obtain the estimated time, and the time attribute value of the corresponding node to be updated is updated using the estimated time.

[0151] Based on the order of the estimated time, the nodes to be updated that have been updated are dynamically added to the sequence of nodes to be solved, resulting in the updated queue of nodes to be solved.

[0152] In this embodiment, the currently processed discrete grid node is extracted from the sequence of nodes to be solved. The potential energy gradient value is calculated using a finite difference scheme with a first-order upwind scheme. Combined with the velocity value in the three-dimensional velocity field, the local travel time from the neighboring node to the current node is calculated. The six spatially adjacent nodes of the current node are selected as nodes to be updated. The local travel time is added to the time attribute value of the current node to obtain the estimated time of the node to be updated. If the estimated time is less than the original time attribute value of the node to be updated, the time attribute value of the node is updated with the estimated time. According to the order of the estimated time, the min-heap algorithm is used to dynamically add the updated nodes to be updated to the sequence of nodes to be solved, ensuring that nodes with smaller time attribute values ​​are extracted and processed first, resulting in an updated queue of nodes to be solved.

[0153] This invention utilizes a finite difference scheme to calculate the potential energy gradient and combines it with spatial movement velocity values ​​to solve for the local travel time. Subsequently, it dynamically updates the sequence of nodes to be solved according to the expected time magnitudes. Compared to existing algorithms that statically traverse the mesh, this achieves efficient time field expansion based on a fast travel method. This dynamic priority queue processing mechanism ensures that the calculation of the evacuation time field strictly adheres to the minimum time path principle, realistically simulating the pathfinding behavior of personnel in complex obstacle environments, and further improving the physical reliability of hazard quantification values.

[0154] The logic for calculating the evacuation time increment relative to the accessible baseline scenario includes:

[0155] Based on the three-dimensional semantic field, the probability distribution of semantic categories in the coordinates of spatial sampling points is mapped to the spatial location of wall categories, which is then used as the wall obstacle area.

[0156] Set the movement speed of all spatial locations in the three-dimensional semantic field, except for the wall obstacle area, to the preset maximum movement speed to generate an accessible baseline scene.

[0157] Solve the Eikonal equations for the barrier-free baseline scenario to calculate the baseline evacuation time field;

[0158] Calculate the numerical difference between the evacuation time field and the reference evacuation time field at the same spatial location;

[0159] To provide a reference benchmark for assessing the degree of obstruction in real-world scenarios, an accessible benchmark scenario is first constructed. Specifically, immovable building structures such as walls and columns in the real-world scenario are retained as absolute obstacle areas. The spatial movement velocity values ​​of all other spatial locations within the 3D computational domain, excluding these fixed building obstacles, are uniformly set to the preset maximum movement velocity of personnel. Simultaneously, the safety exit locations in the real-world scenario are used as the same zero-potential-energy boundary conditions. The Eikonal equation is reconstructed for this accessible benchmark scenario. The left side of the equation represents the magnitude of the gradient of the benchmark evacuation time field, and the right side is replaced with the reciprocal of the constant maximum movement velocity at the corresponding location in the benchmark scenario. The fast travel method is used to numerically solve this benchmark equation, starting from the safety exit boundary node. Using a preset maximum moving speed as a constant wavefront propagation rate, time propagation is performed within a grid space excluding fixed building obstacles until the time attribute values ​​of all reachable grid nodes converge and the sequence of nodes to be solved is empty. After propagation terminates, the system maps the minimum arrival time attribute values ​​stored in each discrete grid node in the barrier-free benchmark scenario back to three-dimensional space according to the same spatial coordinate mapping relationship, generating a benchmark evacuation time field. Since this scenario excludes the deceleration interference of dynamic hazards such as debris accumulation and illegal parking of equipment on personnel movement, this benchmark evacuation time field represents the shortest time required for personnel to evacuate along the optimal path in an ideal barrier-free state, providing a standard comparison benchmark for subsequent calculation of the evacuation time increment between the actual evacuation time field and the benchmark evacuation time field.

[0160] If the numerical difference is greater than the preset difference threshold, the numerical difference will be used as the evacuation time increment for the spatial location.

[0161] If the numerical difference is less than or equal to the preset difference threshold, the evacuation time increment of the spatial location will be marked as zero.

[0162] In this embodiment, based on the three-dimensional semantic field, the spatial locations in the spatial sampling point coordinates that correspond to the semantic category probability distribution of the wall category are designated as wall obstacle areas. The movement speed corresponding to all spatial locations in the three-dimensional semantic field other than the wall obstacle areas is set to a preset maximum movement speed of 1.5 meters per second to generate an obstacle-free baseline scene. The Eikonal equation of the baseline scene is solved to obtain the baseline evacuation time field. The numerical difference between the actual evacuation time field and the baseline evacuation time field at the same spatial location is calculated. The preset difference threshold is set to 2 seconds. If the numerical difference is greater than 2 seconds, the numerical difference is used as the evacuation time increment for that spatial location. If the numerical difference is less than or equal to 2 seconds, it indicates that although there is a slight obstacle in the area, it does not affect the overall evacuation efficiency, and the evacuation time increment is marked as zero.

[0163] This invention constructs an accessible benchmark scene using wall type as the absolute boundary and calculates the benchmark evacuation time field for comparison. Compared with the existing technology that directly uses absolute evacuation time as the evaluation index, this invention removes the influence of the evacuation time baseline caused by the inherent structure of the building itself. By calculating the numerical difference and filtering it with a preset difference threshold, it can accurately extract the additional evacuation delay caused by non-wall type dynamic fire hazards, effectively eliminating the interference of the inherent structure of the building, so that the final output evacuation time increment can purely reflect the true degree of harm of the fire hazard.

[0164] The logic for outputting the hazard quantification value includes:

[0165] The spatial locations where the evacuation time increment is greater than a preset time threshold are identified as obstructed areas.

[0166] Calculate the product of the volume of the obstructed area and the maximum evacuation time increment within the obstructed area to obtain the product value;

[0167] The product value is used as a quantitative value for the severity of the hidden danger and output.

[0168] In this embodiment, spatial locations with evacuation time increments greater than a preset time threshold of 5 seconds are identified as obstructed areas. The 5-second preset time threshold eliminates interference from brief delays, ensuring that the identified obstructed areas pose a significant hazard. The number of voxels within each obstructed area is counted, and the volume of the obstructed area is obtained by multiplying the number of voxels by the volume of a single voxel. The maximum evacuation time increment within this obstructed area is extracted, and the volume of the obstructed area is multiplied by the maximum evacuation time increment to obtain a product value. This product value comprehensively reflects the spatial extent and time delay of the obstruction caused by the hazard, and is output as a quantitative value of the hazard severity. A larger value indicates a more severe impact of the hazard on the obstruction of evacuation routes.

[0169] This invention extracts the obstructed area and calculates the product of the obstructed area's volume and the maximum evacuation time increment as a quantitative value for the severity of the hazard. Compared to existing hazard rating methods that rely solely on distance or area, this invention comprehensively considers two key dimensions: the extent of spatial obstruction and the degree of time delay. This dual-dimensional quantitative assessment mechanism can accurately distinguish between severe localized blockages and widespread minor blockages, providing fire safety supervisors with an intuitive and comparable rectification index, greatly enhancing the system's guiding significance in actual fire safety supervision and enforcement.

[0170] The system calculates the temporal gradient value of each spatial grid point in the evacuation time field. This gradient value reflects the rate at which evacuation arrival time changes with spatial distance. Since actual obstacles or potential hazards hinder the movement of people, they can cause drastic changes in the time field of the surrounding paths. Therefore, the system marks spatial locations where the temporal gradient value exceeds a preset gradient threshold as evacuation bottleneck nodes. The system extracts the three-dimensional spatial coordinates of these nodes, performs spatial clustering on adjacent discrete bottleneck nodes, and merges them to form connected obstructed areas. The three-dimensional spatial range of these obstructed areas corresponds to the spatial location associated with the aforementioned hazard severity quantification value. The system calls the graphics rendering interface and projects the three-dimensional spatial range of the obstructed area onto the original continuous image screen through coordinate transformation. The system overlays and displays the highlighted boundary box of the hazard area at the corresponding position, and marks the evacuation time increment value and hazard severity quantification value of the obstructed area in the area adjacent to the boundary box. Through this process, the system realizes a complete closed loop from abstract three-dimensional time field gradient calculation to intuitive two-dimensional visualization alarm, providing an intuitive visualization basis for the accurate location and rapid handling of fire hazards.

[0171] The preset gradient threshold is specifically set to 2.0. In this embodiment, because people move at a constant speed in an unobstructed open area, the evacuation time field changes linearly and smoothly with spatial distance. Theoretically, its time gradient magnitude is close to or equal to the reciprocal of speed. However, when there are hidden dangers such as fire lane obstruction or illegal stacking of objects in the space, causing the path to be blocked, people are forced to slow down or detour, which will cause a drastic change in the time field of the local area, resulting in a sharp increase in the time gradient magnitude. The preset gradient threshold is set to 2.0. When the time gradient value of a certain spatial grid point is detected to be greater than or equal to 2.0, it is determined that there is a significant passage obstruction effect at that location, and it is marked as an evacuation bottleneck node. This threshold setting of 2.0 effectively eliminates the slight field change interference at the corner of the normal building structure, ensuring that only severely congested areas caused by substantial dynamic hidden dangers are keenly captured and accurately located.

[0172] This invention effectively overcomes the limitations of two-dimensional visual recognition technology in constructing a complete three-dimensional semantic expression of a scene by building a three-dimensional semantic field containing geometric, color, and semantic information. This allows the system to perform precise distance measurement and spatial orientation analysis based on three-dimensional voxels. When judging potential hazards such as blocked safety exits or obstructed fire-fighting equipment, it is no longer limited by single-view visual features but utilizes the three-dimensional spatial topological relationships between entities for deep reasoning, significantly improving the spatial credibility and accuracy of the recognition results. This invention breaks through the bottleneck of existing hazard assessment methods that only remain at the qualitative judgment level. By solving the Eikonal equation to construct an evacuation time field, it achieves the physical quantification of the severity of hazards and can accurately calculate the specific incremental impact of blocked evacuation routes on personnel escape time. This innovation transforms abstract safety hazards into intuitive time cost data, providing a solid scientific basis for fire safety rectification decisions. In practical applications, it achieves a leap from simple visual perception to in-depth physical assessment and decision support, greatly improving the refinement of fire management and the actual effectiveness of hazard control.

[0173] The various embodiments of the techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0174] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0175] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), and erasable programmable read-only memory (EPROM).

[0176] Or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

Claims

1. A fire safety hazard identification and labeling system based on visual understanding, characterized in that, It includes a build module, a generation module, a reasoning module, and an evaluation module: The construction module is used to acquire continuous images of fire scene, calculate the coordinates of spatial sampling points and the direction of target ray by combining camera pose parameters, calculate volume density value, color value and semantic category probability distribution by using multilayer perceptron network, and generate a three-dimensional semantic field through integral training. The generation module is used to extract effective voxels and divide entity objects, calculate the distance and relative orientation angle between entities, match spatial relationship predicates, and construct a scene graph; The reasoning module is used to input the entity category attributes and spatial topology relationships of the scene graph into a preset hidden danger judgment logic tree, traverse each node for matching, and obtain the output result; The evaluation module is used to construct and solve the Eikonal equation based on the three-dimensional semantic field when the output result indicates the existence of a hidden danger, obtain the evacuation time field, calculate the evacuation time increment of the evacuation time field relative to the barrier-free benchmark scene, and output the hazard quantification value.

2. The fire safety hazard identification and labeling system based on visual understanding as described in claim 1, characterized in that, The construction module is used to acquire continuous images of the fire scene as input images; Feature extraction and matching are performed on the input image, and camera pose parameters are calculated, including the camera center position and rotation matrix; Based on the pixel coordinates of the input image and the preset camera intrinsic parameter matrix, the direction vector of each pixel in the camera coordinate system is calculated, and the direction vector is converted into the target ray direction in world coordinates according to the camera pose parameters. Spatial location points are selected at equal intervals along the direction of the target ray, and the coordinates of each spatial location point are determined as the coordinates of the spatial sampling point. The spatial sampling point coordinates and the target ray direction are input into a multilayer perceptron network to obtain the volume density value, color value and semantic category probability distribution corresponding to the spatial sampling point coordinates. Based on the volume density value and color value, the spatial sampling points are integrated along the target ray direction to generate a predicted image; Based on the volume density value and semantic category probability distribution, the spatial sampling points are integrated along the target ray direction to generate a predicted semantic map; Calculate the mean square error of pixel color between the predicted image and the corresponding single frame image in the input image, and calculate the cross-entropy loss value of the semantic labels between the predicted semantic map and the corresponding single frame image in the input image. Based on the mean squared error and cross-entropy loss value, the weight parameters of the multilayer perceptron network are updated using the backpropagation algorithm until the mean squared error is less than a preset error threshold, thereby generating a three-dimensional semantic field.

3. The fire safety hazard identification and labeling system based on visual understanding as described in claim 2, characterized in that, The generation module is used to sample within the spatial range of the three-dimensional semantic field according to a preset grid spacing to obtain grid sampling points and acquire the volume density value of each grid sampling point. Grid sampling points with a volume density value greater than a preset density threshold are considered as valid voxels; Extract the probability of each category from the semantic category probability distribution corresponding to each valid voxel, take the category corresponding to the largest probability value as the target semantic category, and write the label of the target semantic category into the attribute information of the valid voxel as the entity category attribute; Traverse each valid voxel, group together valid voxels with the same semantic category and spatially adjacent, classify and identify them as the same entity object; Traverse the valid voxel sets contained in each of the two entity objects, calculate the Euclidean distance between the nearest points of the two sets as the minimum distance, and calculate the horizontal projection angle of the line connecting the center points of the two entity objects in the world coordinate system. The horizontal projection angle is taken as the relative azimuth angle; If the minimum distance value is less than the preset distance threshold, the relative orientation angle and the vertical height difference between the two entity objects are input into the preset rule set to match and determine the spatial relationship predicate between the entity objects; If the minimum distance value is greater than or equal to the preset distance threshold, it is determined that there is no spatial relationship between the entity objects, and the matching stops; Each entity object is mapped to an independent node in the scene graph, and the spatial relationship predicate is used as the attribute information of the connecting edges of each entity object. The independent nodes corresponding to the entity objects with spatial relation predicates are taken as associated node pairs, and directed connection edges are generated between each associated node pair. The directed connecting edges are used to topologically associate the corresponding independent nodes to generate a graph data structure, which is then used as the scene graph.

4. The fire safety hazard identification and labeling system based on visual understanding as described in claim 3, characterized in that, The reasoning module is used to obtain the entity category attributes of entity objects in the scene graph and the spatial topological relationships between entity objects; Input the entity category attributes and spatial topological relationships into a preset hidden danger judgment logic tree; Obtain the root node of the preset hidden danger judgment logic tree as the current judgment node, and match the entity category attribute and spatial topology relationship with the judgment conditions stored in the current judgment node; If the entity category attribute and spatial topology relationship satisfy the judgment condition of the current judgment node, then jump to the child node pointed to by the current judgment node, update the child node to the current judgment node, and repeat the matching until the leaf node of the preset hidden danger judgment logic tree is reached. If the entity category attribute and spatial topology relationship do not meet the judgment conditions of the current judgment node, then jump to the negative branch node pointed to by the current judgment node, update the negative branch node to the current judgment node, and repeat the matching until the leaf node of the preset hidden danger judgment logic tree is reached. Read the conclusion identifier stored in the leaf node, generate a Boolean value indicating whether there is a fire safety hazard and a corresponding hazard type label based on the conclusion identifier, and output the Boolean value and the corresponding hazard type label.

5. The fire safety hazard identification and labeling system based on visual understanding as described in claim 4, characterized in that, The evaluation module is used to receive the output results; If the Boolean value indicates a potential risk, then read the volume density value in the three-dimensional semantic field; A three-dimensional velocity field is constructed based on the volume density value in the three-dimensional semantic field. Voxels whose semantic categories belong to the preset safety exit category are extracted from the three-dimensional semantic field, and the set of spatial coordinates of the voxels belonging to the preset safety exit category is used as the safety exit location. The potential energy value of the safety exit location in the potential field calculation space is set to zero, which serves as a zero potential energy boundary condition. Based on the three-dimensional velocity field and the zero potential energy boundary condition, the Eikonal equation is constructed. The evacuation time field is obtained by solving the Eikonal equation. The evacuation time increment of the evacuation time field relative to the barrier-free reference scene is calculated, and the hazard quantification value is output.

6. The fire safety hazard identification and labeling system based on visual understanding as described in claim 5, characterized in that, The logic for constructing a three-dimensional velocity field based on the volume density values ​​in the three-dimensional semantic field includes: The volume density values ​​in the volume density distribution data are mapped to spatial movement speed values, wherein the spatial movement speed values ​​in the obstacle area are set to zero. The volume density values ​​in the three-dimensional semantic field are mapped to spatial movement speed values, and the regions in the three-dimensional semantic field with volume density values ​​greater than the preset obstacle density are designated as obstacle regions, and the spatial movement speed values ​​of the obstacle regions are set to zero.

7. The fire safety hazard identification and labeling system based on visual understanding as described in claim 6, characterized in that, The logic for solving the Eikonal equation to obtain the evacuation time field includes: Based on the coordinate range of the three-dimensional semantic field, the three-dimensional space is discretized into a grid, and the discrete grid nodes are used as initial nodes and added to the sequence of nodes to be solved. Repeatedly propagate the time between the grid nodes until the sequence of nodes to be solved is empty. Then, map the time attribute values ​​stored in each discrete grid node back to the three-dimensional space to generate an evacuation time field.

8. The fire safety hazard identification and labeling system based on visual understanding as described in claim 7, characterized in that, The logic for time propagation of the grid nodes includes: The discrete grid node with the smallest time attribute value is extracted from the sequence of nodes to be solved as the currently determined node. Each spatial neighbor node of the currently determined node is obtained as the node to be updated. For each node to be updated, the finite difference scheme of the first-order upwind scheme is used. Based on the time attribute values ​​of the determined neighbor nodes in each direction of the node to be updated, combined with the spatial movement velocity value of the node to be updated in the three-dimensional velocity field, the first-order difference discrete equation corresponding to the Eikonal equation is solved based on the fast travel method to obtain the candidate arrival time of the node to be updated. Obtain the spatial neighbor nodes of the currently processed discrete grid node as the nodes to be updated; The cumulative arrival time from each node to be updated to the safety exit location is calculated based on the local travel time to obtain the estimated time, and the time attribute value of the corresponding node to be updated is updated using the estimated time. Based on the order of the expected times, the nodes to be updated that have been updated are dynamically added to the sequence of nodes to be solved, resulting in the updated queue of nodes to be solved.

9. The fire safety hazard identification and labeling system based on visual understanding as described in claim 8, characterized in that, The logic for calculating the evacuation time increment relative to the accessible baseline scenario includes: Based on the three-dimensional semantic field, the semantic category probability distribution in the spatial sampling point coordinates is mapped to the spatial location of the wall category, which is then used as the wall obstacle area; Set the movement speed of all spatial locations in the three-dimensional semantic field, except for the wall obstacle area, to a preset maximum movement speed to generate an accessible baseline scene. Solve the Eikonal equations for the barrier-free baseline scenario to calculate the baseline evacuation time field; Calculate the numerical difference between the evacuation time field and the reference evacuation time field at the same spatial location; If the numerical difference is greater than a preset difference threshold, then the numerical difference is used as the evacuation time increment of the spatial location. If the numerical difference is less than or equal to a preset difference threshold, the evacuation time increment of the spatial location is marked as zero.

10. The fire safety hazard identification and labeling system based on visual understanding as described in claim 9, characterized in that, The logic for outputting the quantified value of potential hazards includes: The spatial locations where the evacuation time increment is greater than a preset time threshold are extracted to form the obstructed area; Calculate the product of the volume of the obstructed area and the maximum evacuation time increment within the obstructed area to obtain the product value; The product value is used as a quantification of the severity of the hidden danger and output.

Images