Video Data-Driven Tunnel Fire Detection and Simulation Method and System

Abstract

The present invention discloses a video data-driven tunnel fire detection and simulation method and system, belonging to the technical fields of computer vision and machine learning, and solving the problems that the current tunnel fire target detection algorithm is prone to insufficient ability to distinguish vehicles from flames in a complex multi-target scene environment. The present invention preprocesses the obtained original data set to obtain a target data set for training an improved YOLOv8 neural network model, performs target detection on the disaster situation images of the tunnel fire scene to be detected, obtains the center coordinates of each preliminary disaster situation object target for correction, and after correction, uses a monocular vision positioning method to calibrate and distinguish the corrected disaster situation object targets, and then performs real-time scene twin modeling, that is, obtains a tunnel fire scene twin model and dynamically updates it; based on the tunnel fire scene twin model, analyzes the flame spread of the tunnel fire scene and realizes dynamic update in unity. The present invention is used for video data-driven tunnel fire detection and simulation.

Description

Technical Field

[0001] A video data-driven tunnel fire detection and simulation method and system for video data-driven tunnel fire detection and simulation, belonging to the technical fields of computer vision and machine learning. Background Art

[0002] Tunnels are important facilities for national infrastructure in transportation and play a very important role in the development of the national economy. As an important part of the infrastructure, including highway tunnels, railway tunnels, underground passages, utility tunnels, etc., they provide great convenience for our transportation, residence and travel. Due to the enclosed nature, space limitations and low visibility of the internal environment of tunnels, once a fire occurs, it often leads to serious casualties and property losses. However, due to inevitable traffic accidents and vehicle failures, tunnel fires occur frequently. Compared with fires on open sections, fires in enclosed tunnels develop rapidly, and the high-temperature and toxic smoke generated cannot be exhausted, which is the main cause of serious disasters. At the same time, due to the complex environment in the tunnel and the low visibility after a fire, it is difficult for firefighters to quickly judge the fire situation inside and take timely and effective fire extinguishing and rescue measures. Statistics show that from 2000 to 2016, a total of 161 medium and large tunnel fire accidents occurred in China, causing heavy casualties and economic losses. In order to guide fire extinguishing and rescue work, it has been clearly proposed that it is necessary to improve the intelligent management level of tunnels, emphasizing the deep integration of advanced information technology into the tunnel fire protection system to establish a more intelligent fire protection management system, which is the key direction to improve the informatization and intelligent level of tunnel management.

[0003] Digital twin technology creates a comprehensive, interconnected and highly realistic scene twin model by precisely mapping the object attributes, states and behaviors in the physical world to the digital world. This model supports interactive exploration and analysis, enabling problem diagnosis, status assessment and prediction of future trends in the digital space. In addition, the application of digital twins in disaster prevention and mitigation can not only give early warning of disasters, but also optimize resource allocation and analyze the evolution of disasters, thereby enhancing the perception, understanding and decision-making abilities of emergency responders for disaster scenarios. Therefore, constructing an environmental twin model of tunnel fires is crucial for improving the informatization management level of tunnel fire prevention and control. Monitoring videos are an important technical means for data perception, with advantages such as real-time, economy, reliability and convenience, and have become an indispensable part of the informatization construction of tunnel management. Based on tunnel monitoring videos, comprehensive perception of vehicle information can be carried out, improving the accuracy of information perception and providing higher-precision data for the twin modeling of vehicles in the tunnel scenario. However, the weak light environment in the tunnel poses a huge challenge to the comprehensive perception of vehicle information based on tunnel monitoring videos.

[0004] In view of the low-light environment in tunnels, the existing tunnel fire scene modeling methods mainly rely on manual modeling and parametric modeling. These methods mainly focus on the refined display of static scenes. However, the tunnel fire environment is dynamic and uncertain, with complex semantic knowledge and object spatial association relationships, making it difficult to perform automatic modeling and meet the rapid modeling requirements of the tunnel fire environment.

[0005] In summary, the existing technologies have the following technical problems:

[0006] 1. Due to various interference factors in the tunnel, such as reflection, shadow, noise, etc., the current tunnel fire target detection algorithms often perform poorly in complex multi-target scene environments, making it easy to have insufficient ability to distinguish vehicles from flames, resulting in low accuracy of obtaining target information of disaster objects, low model lightweight level, low computing efficiency, and not meeting the high real-time requirements of tunnel fire scene modeling;

[0007] 2. Traditional tunnel fire scene object positioning methods usually directly use the center of the bounding box of the target detection result as the center of the disaster object in the real world, resulting in a large positioning error, directly affecting the authenticity of the tunnel fire scene twin modeling, and further affecting the results of subsequent flame spread analysis;

[0008] 3. Currently, the tunnel fire scene modeling methods generally lack modular processing for flame spread analysis, usually only providing static fire models, unable to reflect the dynamic changes of flames in the tunnel in real time, and unable to effectively predict and evaluate the propagation path and speed of flames. Summary of the Invention

[0009] The purpose of the present invention is to provide a video data-driven tunnel fire detection and simulation method and system to solve the problem that the current tunnel fire target detection algorithms often perform poorly in complex multi-target scene environments, making it easy to have insufficient ability to distinguish vehicles from flames, resulting in low accuracy of obtaining target information of disaster objects, low model lightweight level, low computing efficiency, and not meeting the high real-time requirements of tunnel fire scene modeling.

[0010] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0011] A video data-driven tunnel fire detection and simulation method includes the following steps:

[0012] Step 1: Preprocess the obtained original data set to obtain a target data set;

[0013] Step 2: Train an improved YOLOv8 neural network model using the target data set, and perform target detection on the disaster situation image of the tunnel fire scene to be detected to obtain the center coordinates of each preliminary disaster object target;

[0014] Step 3: Correct the corresponding preliminary disaster situation object targets based on the central coordinates of each preliminary disaster situation object target. After correction, use the monocular vision positioning method to calibrate and distinguish the corrected disaster situation object targets, and then use the distinguished disaster situation object targets to perform real-time scene twin modeling in the unity engine, that is, obtain the tunnel fire scene twin model and dynamically update it;

[0015] Step 4: Perform flame spread analysis on the tunnel fire scene based on the tunnel fire scene twin model and achieve dynamic update in unity.

[0016] Furthermore, the specific steps of Step 1 are as follows:

[0017] Step 1.1: Obtain the original dataset and use the scene transformation detection method to extract video frames to obtain the original video frame image dataset. Among them, the original dataset includes tunnel fire monitoring video materials and the tunnel fire video data simulated in unity3d. Each image in the video frame image dataset contains vehicles and flames;

[0018] Step 1.2: Perform preliminary denoising and image enhancement processing on various tunnel fire scene images in the original video frame image dataset. Select the images that exceed the given value through the threshold comparison method, perform diversity processing on the selected images, and transfer the unselected images back to the denoising and image enhancement processing steps until the requirements are met. Add 10% negative samples to the dataset after diversity processing, and perform reference learning by calculating the contrast loss function of positive and negative samples. Among them, exceeding the given value means exceeding the given resolution, clarity, contrast, richness of details, and the recognizability of target features at the same time. The diversity processing includes brightness enhancement, contrast enhancement, rotation at any angle, and composite scaling processing;

[0019] Step 1.3: Use the LabelImg data annotation software to annotate each frame of the image obtained after reference learning in Step 1.2, and save the annotation information and the image frames extracted from the corresponding reference learning tunnel fire monitoring video materials and the tunnel fire video data simulated in unity3d as multi-modal data in the database to construct the tunnel fire disaster situation object dataset, that is, obtain the target dataset.

[0020] Furthermore, the improved YOLOv8 neural network model in Step 2 is based on the YOLOv8 neural network model, introducing the EfficientNetB0 network, the multi-semantic space attention mechanism module, and the feature extraction module that integrates the EfficientNetB0 network and the multi-semantic space attention mechanism module as the backbone network;

[0021] The feature extraction module is represented as:

[0022] F final (x) = SMSA(F(x))

[0023] Among them, F final (x) represents the feature output by the feature extraction module, F(x) represents the feature extraction output of the EfficientNetB0 network, and SMSA(F(x)) represents the feature map of F(x) after being processed by the channel attention mechanism.

[0024] Further, the specific steps of step 2 are:

[0025] The improved YOLOv8 neural network model is trained using the target data set. After training, target detection is performed on the tunnel fire scene disaster image to be detected to obtain the detection results of each preliminary disaster object target, and the center coordinate Coordinate of each preliminary disaster object target is calculated based on the detection results. The detection results include the upper left corner coordinates (u1, v1) and lower right corner coordinates (u2, v2) of each preliminary disaster object target. The preliminary disaster object targets include target vehicles and flames. The calculation formula of the center coordinate Coordinate of each preliminary disaster object target is:

[0026] Coordinate=((u1+u2) / 2,(v1+v2) / 2).

[0027] Further, the specific steps of step 3 are:

[0028] Step 3.1, using the linear regression model and the central coordinates of each preliminary disaster object target to correct each preliminary disaster object target, to obtain a corrected disaster object target;

[0029] Step 3.2, based on the monocular vision positioning method, the spatial coordinates are solved in combination with the projection matrix and pixel coordinates of the corrected disaster target. The spatial coordinates obtained after the solution are the positioning of the corrected disaster target. Then, based on the geometric information perception and attribute feature information extraction of each located disaster target, each located disaster target is distinguished to obtain a distinction result, wherein the geometric perception information includes geometric dimensions, spatial range and spatial position relationship, the geometric dimensions include the size and shape of the flame, the spatial position relationship includes the relative position relationship between the flame and each target vehicle, the attribute feature information includes the type and texture of the target vehicle, the state and timing characteristics of the flame, and the distinction result includes distinguishing the target vehicle into a small passenger car, a large truck and a large passenger car;

[0030] The geometric size L of the disaster target after positioning is:

[0031]

[0032] Among them, W is the actual width of the tunnel, W b and D are respectively the bounding box width of each located disaster object target and the distance from the center of each located disaster object target to the camera;

[0033] The spatial range boundingbox of the located disaster object target is:

[0034]

[0035] Among them, (x b , y b ) represents the center point coordinates of the located disaster object target, and z1 and z2 are the ranges of the located disaster object target in the depth direction, representing the minimum depth and the maximum depth respectively;

[0036] Step 3.3: Based on the geometric information perception results and attribute feature information of each distinguished disaster object target, determine the twin modeling parameters of the disaster object target through the "perception result + knowledge model driven" method. Subsequently, based on the twin modeling parameters, use the spatial topological semantic constraint conditions to map and instantiate the tunnel fire scene entity objects in the tunnel fire scene disaster image in the unity engine, that is, obtain the tunnel fire scene twin model. The specific steps are as follows:

[0037] Based on the geometric perception results, attribute feature information of each distinguished disaster object target, and multi-dimensional and multi-scale scene image information from local and global, use the graph database to take each distinguished disaster object target and its attribute feature information as nodes, and the geometric perception results as relationships as edges for data modeling and relationship mapping. After mapping, integrate the geometric structure of the tunnel, the fire location, and the geometric perception results and attribute feature information of each distinguished disaster object target through the spatial data fusion method to form a dynamic three-dimensional knowledge graph, so as to obtain the twin modeling parameters of each distinguished disaster object target, including spatial coordinates, temperature coefficients, and flame combustion rates. Among them, the multi-dimensional and multi-scale scene image information includes flame image information in the initial, middle, and late stages of the fire, and tunnel fire scene image information with different heights, widths, and depths;

[0038] Based on the obtained twin modeling parameters, use the spatial topological semantic constraint conditions to map and instantiate each distinguished disaster object target in the tunnel fire scene disaster image in the unity engine to establish a tunnel fire scene twin model. Among them, the spatial topological semantic constraint conditions include adjacency relationships, inclusion relationships, object type constraints, and environmental constraints;

[0039] Step 3.4: Based on the real-time obtained disaster situation images of the tunnel fire scene, re-conduct preliminary disaster situation object target detection, correction, positioning, and differentiation to update the initial twin modeling of the tunnel fire scene.

[0040] Furthermore, the specific steps of Step 4 are as follows:

[0041] Step 4.1: Extract the spread characteristics of the twin model of the tunnel fire scene from the disaster situation images of the tunnel fire scene, and construct one or more flame spread models in Unity3D. Among them, the flame spread model includes the change of the flame over time, the growth of the flame height, and the change of the flame spread shape. The change of the flame spread shape includes the generation position of the flame particles, the change of the transparency of the flame particles over time, the color change, and the speed adjustment of the wind speed particles.

[0042] Based on the linear expansion characteristic of the expansion mode of the combustion area, the formula for the change of the flame over time is:

[0043] R f (t) = R0 + v f ·t

[0044] Where, R f (t) represents the change of the flame at time t, R0 is the initial radius of the flame, and v f represents the flame spread speed, which is extracted from the twin model of the tunnel fire scene;

[0045] The formula for the growth of the flame height is:

[0046] H f (t) = H0 + v s ·t

[0047] Where, H f (t) represents the growth of the flame height at time t, H0 is the initial height of the flame, and v s is the expansion speed of the flame in the vertical direction;

[0048] The change of the flame spread shape is simulated by the generation position, transparency, color change of the flame particles, and the speed adjustment of the wind speed particles;

[0049] The formula for the generation position of the flame particles is:

[0050] x i = x0 + R·cos(θ)·u

[0051] y i = y0 + R·sin(θ)·v

[0052] z i = z0 + H·w

[0053] Among them, R is the expansion radius, representing the size of the flame combustion area, which is uniformly randomly generated by flame particles. (x i , y i , z i ) is the generation position coordinate of the i-th flame particle, (x0, y0, z0) represents the central position of the flame combustion area, θ is the angle parameter representing the flame spreading direction, H is the vertical height of the flame, and u, v, and w are random numbers used to simulate the random distribution of flame particles within the flame combustion area;

[0054] The transparency of the flame particles changes with time to simulate the blurring effect of the flame edge and the dynamic changes during the combustion process. The formula is:

[0055]

[0056] Among them, α i (t) represents the transparency of the i-th flame particle at time t, represents the initial transparency of the i-th flame particle, and T represents the life cycle of the flame;

[0057] The color change of the flame is obtained by analyzing the RGB values of the flame through a frame-by-frame analysis method. The formula is:

[0058]

[0059] Among them, Color i (t) represents the color of the i-th flame particle at time t, represents the initial color of the i-th flame particle, represents the color after the i-th flame particle burns out, and T represents the life cycle of the flame particle;

[0060] The formula for adjusting the speed based on the wind speed particles is:

[0061]

[0062] Among them, v i (t) represents the speed of the i-th flame particle at time t, represents the initial speed of the i-th flame particle under windless conditions, α represents the wind speed influence coefficient, represents the wind speed at time t;

[0063] Step 4.2: Calculate the Euclidean distance between the flame particles in the simulated flame spread model and the target vehicle. If the distance is less than the given threshold, it is considered that any flame particle has contacted the target object, and then go to Step 4.1 to generate a new flame spread simulation. Otherwise, do not simulate. Among them, the distance between the flame particle and the target vehicle is calculated by the distance formula in three-dimensional space:

[0064]

[0065] Among them, d i represents the distance between the i-th flame particle and each target vehicle. (x f , y f , z f ) is the position of the i-th flame particle in the Unity three-dimensional space, and (x c , y c , z c ) is the center position of each target vehicle. When the distance d i is less than the predetermined threshold R trigger , a new flame spread model is generated at the position of the corresponding target vehicle;

[0066] Step 4.3: Based on all the flame spread models and the twin model of the tunnel fire scene in the disaster situation image of the tunnel fire scene, conduct flame spread simulation on the tunnel fire scene and update it dynamically.

[0067] A video data-driven tunnel fire detection and simulation system, including:

[0068] Target dataset construction module: Preprocess the obtained original dataset to obtain the target dataset;

[0069] Target detection module: Use the target dataset to train the improved YOLOv8 neural network model, and conduct target detection on the disaster situation image of the tunnel fire scene to be detected to obtain the center coordinates of each preliminary disaster situation object target;

[0070] Twin model modeling module: Correct the corresponding preliminary disaster situation object target based on the center coordinates of each preliminary disaster situation object target. After correction, use the monocular vision positioning method to calibrate and distinguish the positions of the corrected disaster situation object targets, and then use the distinguished disaster situation object targets to conduct real-time scene twin modeling in the unity engine, that is, obtain the twin model of the tunnel fire scene and update it dynamically;

[0071] Flame spread module: Conduct flame spread analysis on the tunnel fire scene based on the twin model of the tunnel fire scene and achieve dynamic update in unity.

[0072] Furthermore, the specific implementation steps of the target dataset construction module are as follows:

[0073] Step 1.1: Obtain the original dataset and use the scene transformation detection method to extract video frames to obtain the original video frame image dataset. Among them, the original dataset includes tunnel fire monitoring video materials and the tunnel fire video data simulated in unity3d, and each image in the video frame image dataset contains vehicles and flames;

[0074] Step 1.2: Perform preliminary denoising and image enhancement on various tunnel fire scene images in the original video frame image dataset. Select images that exceed a given value through the threshold comparison method, perform diversity processing on the selected images, and transfer the unselected images back to the denoising and image enhancement steps until the requirements are met. Add 10% negative samples to the dataset after diversity processing, and perform reference learning by calculating the contrast loss function of positive and negative samples. Here, exceeding the given value means exceeding the given resolution, clarity, contrast, richness of details, and recognizability of target features at the same time. The diversity processing includes brightness enhancement, contrast enhancement, rotation at any angle, and composite scaling processing;

[0075] Step 1.3: Use the LabelImg data annotation software to annotate each frame of the image obtained after reference learning in Step 1.2. Save the annotation information, the corresponding tunnel fire monitoring video data after reference learning, and the image frames extracted from the tunnel fire video data simulated in unity3d as multi-modal data in the database to construct a tunnel fire disaster object dataset, that is, obtain the target dataset.

[0076] Furthermore, the improved YOLOv8 neural network model in the target detection module is based on the YOLOv8 neural network model, introducing the EfficientNetB0 network, the multi-semantic space attention mechanism module, and the feature extraction module that integrates the EfficientNetB0 network and the multi-semantic space attention mechanism module as the backbone network;

[0077] The representation of the feature extraction module is:

[0078] F final (x) = SMSA(F(x))

[0079] where F final (x) represents the features output by the feature extraction module, F(x) represents the feature extraction output of the EfficientNetB0 network, and SMSA(F(x)) represents the feature map after F(x) is processed by the channel attention mechanism;

[0080] The specific implementation steps of the target detection module are as follows:

[0081] The improved YOLOv8 neural network model is trained using the target dataset. After training, object detection is performed on the disaster situation images of the tunnel fire scene to be detected, and the detection results of each preliminary disaster situation object target are obtained. Based on the detection results, the center coordinates Coordinate of each preliminary disaster situation object target are calculated. Among them, the detection results include the upper left coordinates (u1, v1) and the lower right coordinates (u2, v2) of each preliminary disaster situation object target. The preliminary disaster situation object targets include target vehicles and flames. The calculation formula for the center coordinates Coordinate of each preliminary disaster situation object target is:

[0082] Coordinate = ((u1 + u2) / 2, (v1 + v2) / 2);

[0083] The specific steps of the twin model modeling module are as follows:

[0084] Step 3.1: Use the linear regression model and the center coordinates of each preliminary disaster situation object target to correct each preliminary disaster situation object target, and obtain the corrected disaster situation object target;

[0085] Step 3.2: Based on the monocular vision positioning method, combine the projection matrix and pixel coordinates of the corrected disaster situation object target to calculate the spatial coordinates. The calculated spatial coordinates are the positioning of the corrected disaster situation object target. Then, based on the positioned disaster situation object targets, geometric information perception and attribute feature information extraction are performed to distinguish the positioned disaster situation object targets, and the discrimination result is obtained. Among them, the geometric perception information includes geometric size, spatial range, and spatial position relationship. The geometric size includes the size and shape of the flame. The spatial position relationship includes the relative position relationship between the flame and each target vehicle. The attribute feature information includes the type and texture of the target vehicle, the state and temporal characteristics of the flame. The discrimination result includes classifying the target vehicle into small passenger cars, large trucks, and large buses;

[0086] The geometric size L of the positioned disaster situation object target is:

[0087]

[0088] Among them, W is the actual width of the tunnel, W b and D are the bounding box width of each positioned disaster situation object target and the distance from the center of each positioned disaster situation object target to the camera, respectively;

[0089] The spatial range boundingbox of the positioned disaster situation object target is:

[0090]

[0091] Among them, (x b , y b) represents the center point coordinates of the disaster object target after positioning. z1 and z2 are the ranges of the disaster object target in the depth direction after positioning, representing the minimum depth and the maximum depth respectively;

[0092] Step 3.3: Based on the geometric information perception results and attribute feature information of each distinguished disaster object target, determine the twin modeling parameters of the disaster object target through the "perception result + knowledge model-driven" method. Subsequently, based on the twin modeling parameters, use the spatial topological semantic constraint conditions to map and instantiate the tunnel fire scene entity objects in the tunnel fire scene disaster image in the unity engine, that is, obtain the tunnel fire scene twin model. The specific steps are as follows:

[0093] Based on the geometric perception results, attribute feature information of each distinguished disaster object target, and multi-dimensional and multi-scale scene image information from local and global, use the graph database to take each distinguished disaster object target and its attribute feature information as nodes, and the geometric perception result as the relationship as the edge to perform data modeling and relationship mapping. After mapping, use the spatial data fusion method to integrate the geometric structure of the tunnel, the fire location, and the geometric perception results and attribute feature information of each distinguished disaster object target to form a dynamic three-dimensional knowledge graph, so as to obtain the twin modeling parameters of each distinguished disaster object target, including spatial coordinates, temperature coefficient, and flame combustion rate. Among them, the multi-dimensional and multi-scale scene image information includes flame image information in the initial, middle, and late stages of the fire, and tunnel fire scene image information with different heights, widths, and depths;

[0094] Based on the obtained twin modeling parameters, use the spatial topological semantic constraint conditions to map and instantiate each distinguished disaster object target in the tunnel fire scene disaster image in the unity engine to establish a tunnel fire scene twin model. Among them, the spatial topological semantic constraint conditions include adjacency relationship, inclusion relationship, object type constraint, and environmental constraint;

[0095] Step 3.4: Re-perform preliminary disaster object target detection, correction, positioning, and distinction according to the real-time obtained tunnel fire scene disaster image to update the initial twin modeling of the tunnel fire scene.

[0096] Furthermore, the specific implementation steps of the flame spread module are as follows:

[0097] Step 4.1: Extract the spread characteristics of the tunnel fire scene twin model from the tunnel fire scene disaster image and construct one or more flame spread models in Unity3D. Among them, the flame spread model includes the change of the flame over time, the growth of the flame height, and the change of the flame spread shape. The change of the flame spread shape includes the generation position of the flame particles, the change of the transparency of the flame particles over time, the color change, and the speed adjustment of the wind speed particles;

[0098] Based on the linear expansion characteristic of the combustion area expansion pattern, the formula for the change of the flame over time is:

[0099] R f (t) = R0 + v f ·t

[0100] Among them, R f (t) represents the change of the flame at time t, R0 is the initial radius of the flame, and v f represents the flame spread speed, which is extracted from the twin model of the tunnel fire scene;

[0101] The formula for the growth of the flame height is:

[0102] H f (t) = H0 + v s ·t

[0103] Among them, H f (t) represents the growth of the flame height at time t, H0 is the initial height of the flame, and v s is the expansion speed of the flame in the vertical direction;

[0104] The change of the flame spread shape is simulated by adjusting the generation position, transparency, color change of the flame particles and the speed of the wind speed particles;

[0105] The formula for the generation position of the flame particles is:

[0106] x i = x0 + R·cos(θ)·u

[0107] y i = y0 + R·sin(θ)·v

[0108] z i = z0 + H·w

[0109] Among them, R is the expansion radius, representing the size of the flame combustion area, which is uniformly randomly generated by the flame particles. (x i , y i , z i ) is the generation position coordinate of the i-th flame particle, (x0, y0, z0) represents the center position of the flame combustion area, θ is the angle parameter used to represent the flame spread direction, H is the vertical height of the flame, and u, v, and w are random numbers used to simulate the random distribution of the flame particles within the flame combustion area;

[0110] The transparency of the flame particles changes over time to simulate the blur effect of the flame edge and the dynamic change of the combustion process. The formula is:

[0111]

[0112] where α i (t) represents the transparency of the i-th flame particle at time t, represents the initial transparency of the i-th flame particle, and T represents the life cycle of the flame;

[0113] The color change of the flame is obtained by analyzing the RGB values of the flame through a frame-by-frame analysis method. The formula is:

[0114]

[0115] where Color i (t) represents the color of the i-th flame particle at time t, represents the initial color of the i-th flame particle, represents the color after the i-th flame particle burns out, and T represents the life cycle of the flame particle;

[0116] The formula for adjusting the speed based on the wind speed particle is:

[0117]

[0118] where v i (t) represents the speed of the i-th flame particle at time t, represents the initial speed of the i-th flame particle under windless conditions, α represents the wind speed influence coefficient, represents the wind speed at time t;

[0119] Step 4.2: Calculate the Euclidean distance between the flame particles in the simulated flame spread model and the target vehicle. If the distance is less than the given threshold, it is considered that any flame particle touches the target object, and then go to Step 4.1 to generate a new flame spread simulation. Otherwise, do not simulate. Among them, the distance between the flame particle and the target vehicle is calculated by the distance formula in three-dimensional space:

[0120]

[0121] where d i represents the distance between the i-th flame particle and each target vehicle, (x f , y f , z f ) is the position of the i-th flame particle in the Unity three-dimensional space, (x c , y c , z c ) is the center position of each target vehicle. When the distance d i is less than the predetermined threshold R trigger , a new flame spread model is generated at the position of the corresponding target vehicle;

[0122] Step 4.3: Based on all the flame spread models and the twin models of the tunnel fire scene for the disaster situation images of the tunnel fire scene, conduct flame spread simulation on the tunnel fire scene and update it dynamically.

[0123] Compared with the prior art, the advantages of the present invention are as follows:

[0124] First, the present invention establishes a dataset of tunnel fire disaster situation objects based on monitoring data and simulation data. By establishing a dataset of tunnel fire scene disaster situation objects based on monitoring data (tunnel fire monitoring video materials) and simulation data, effective detection and recognition of the fire scene are realized. First, through frame extraction of the tunnel monitoring video, rich original image data are obtained, and combined with Unity simulation and Internet image data, diverse fire images are generated, greatly improving the diversity and representativeness of the dataset. Subsequently, denoising, enhancement, and background mixing processing are carried out for different fire scenes, ensuring high image quality and significant target effects in the dataset. In addition, by adding negative samples for reference learning, the false detection rate is effectively reduced, and the accuracy of fire detection is improved. Finally, the images are systematically labeled using LabelImg software, and the labeled information is combined with the image frames and stored in the database to form a complete and structured fire disaster situation object dataset. This process not only improves the efficiency of data collection and processing for tunnel fire scenes but also lays a solid data foundation for subsequent automated fire detection and alarm systems.

[0125] Second, the present invention proposes an improved YOLOv8 neural network model. By introducing the EfficientNetB0 network, it can achieve efficient and lightweight feature extraction for features with smaller early targets such as flames, significantly improving the detection performance of the model in complex scenarios. The application of the SMSA attention mechanism (multi-semantic space attention mechanism module) enhances the sensitivity of the model to the features of disaster situation objects, enabling it to more accurately identify the features of flames, smoke, and target vehicles, thereby improving the accuracy and timeliness of fire monitoring. At the same time, the feature extraction module effectively integrates the EfficientNetB0 network and the SMSA attention mechanism, making the model more adaptable when extracting complex scene information. By fine-tuning the model using the target dataset and introducing various hyperparameter settings under the YOLOv8 architecture, the model achieves rapid convergence and effective learning, solving the problem of insufficient recognition ability for small targets and complex backgrounds in previous object detection. Overall, this technology has significant innovation points, not only improving the object detection accuracy and robustness of tunnel fire scenes but also providing more accurate technical support for quickly responding to and handling fire accidents, which helps to improve the intelligent level of the tunnel safety monitoring system.

[0126] III. The present invention conducts rapid scene twin modeling based on a monocular vision positioning method, significantly improving the real-time performance and accuracy of fire disaster monitoring. That is, the results obtained by detecting the real-time video stream of a monitoring camera using an improved YOLOv8 neural network model are used to correct the target center coordinates through a linear regression model, and the spatial coordinates are calculated by combining the projection matrix and pixel coordinates, ensuring the precise perception of the geometric information, spatial range, and type of the disaster object target in the subsequent process. After correction, the monocular vision positioning method can effectively identify and locate the disaster object target in the two-dimensional disaster image of the tunnel fire scene to be detected, and realize the reverse association between the two-dimensional object and the three-dimensional geographical entity, thus solving the deviation problem between the two-dimensional image and the three-dimensional spatial data in the traditional method. In addition, this technology extracts multi-modal and multi-scale information, such as flame size and vehicle type, and uses the method of "perception result + knowledge model drive" to obtain the modeling parameters, optimizing the twin modeling process of the tunnel fire scene. Finally, through the automatic processing of change perception and modeling parameters, the system can achieve dynamic update and real-time adjustment in the initially constructed initial twin model of the tunnel fire scene, improving the intelligence and response speed of the monitoring system. Overall, this technology has the advantages of high efficiency, real-time performance, and precision, innovatively combines visual recognition and spatial modeling, and promotes the development of the disaster monitoring field.

[0127] IV. The Unity3D tunnel fire spread simulation based on video perception of the present invention realizes the efficient and accurate simulation of tunnel fires through advanced physical models, flame particles, and flame modeling. Compared with traditional fire simulation methods, this technology has the advantages of improving simulation accuracy, providing real-time feedback on dynamic fire behavior, and enhancing flexibility and scalability. It is suitable for the study of tunnel fire spread and the formulation of emergency plans. That is, this simulation introduces real physical environment parameters, dynamically adjusts the spread speed and direction of the flame, and ensures a highly realistic flame spread process. The flame spread model is updated in real time through the overall flame and flame particles. The changes of the flame over time, the growth of the flame height, and the changes in the flame spread shape are all simulated according to the actual fire spread characteristics, making the spread trajectory of the flame conform to the laws of real physics. At the same time, through the action of the physical engine on the overall flame and flame particles, the spread trajectory of the flame changes, and the size and speed of the overall flame and flame particles are dynamically adjusted over time, accurately simulating the combustion behavior of the fire at different stages. A collision detection model based on Euclidean distance is also introduced. When the flame particles come into contact with objects in the tunnel, the system will automatically trigger the generation of new flame particles to simulate the secondary spread of the flame. This modeling method supports flexible adjustment of the length, width, and internal environmental characteristics of the tunnel, enabling the system to adapt to different types of tunnel scenarios. By introducing different physical scene elements, such as obstacles, vehicles, ventilation equipment, etc., researchers can quickly generate diverse fire scenarios and evaluate the spread path and speed of the flame under different conditions. This flexibility provides a solid technical support for the formulation of fire emergency plans. BRIEF DESCRIPTION OF THE DRAWINGS

[0128] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0129] Figure 1 is the technical roadmap of the present invention;

[0130] Figure 2 is the structural schematic diagram of the improved YOLOv8 neural network model of the present invention. Among them, EfficientnetB0 is the EfficientnetB0 network, SMSA represents the multi-semantic space attention mechanism module, C2f (Context to Feature Module) represents the module for context feature extraction and feature fusion, Conca represents the connection module, Upsample represents the upsampling module, Conv represents the convolutional kernel, and Head represents the output module;

[0131] Figure 3 This is a schematic diagram of the structure of the multi-semantic space attention mechanism module in the present invention. Among them, AvgPool represents average pooling, MS-DWConvld represents depthwise separable convolution, Group Norm-4 represents group normalization, Concat represents a connection module, and Sigmoid represents the Sigmoid activation function;

[0132] Figure 4 This is a schematic diagram of monocular vision positioning in the present invention. Specific implementation manners

[0133] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0134] First, an original dataset is formed by collecting tunnel fire monitoring video materials and tunnel fire video data simulated in unity3d. Video frames are extracted using image processing software, and operations such as denoising and image enhancement are performed on them. Finally, they are labeled to construct a target dataset. Then, the EfficientNetB0 network, the multi-semantic space attention mechanism (SMSA), and the feature extraction module are introduced as the backbone network of the YOLOv8 neural network model to obtain an improved YOLOv8 neural network model, which realizes the detection of targets (including target vehicles and flames, etc.) in the disaster situation images of the tunnel fire scene to be detected. Considering the usage cost and distribution of tunnel monitoring cameras, the center of the initially detected disaster situation object target is corrected, and then the position calibration and differentiation of the tunnel fire scene object are carried out through the monocular vision method to complete the real-time scene twin modeling in the unity engine. Finally, the flame spread analysis of the existing tunnel fire scene is carried out and dynamically updated in unity.

[0135] The present invention focuses on the research of accurate perception modeling of disaster situation objects and simulation prediction analysis of smoke-fire spread in tunnel fire scenes. In terms of accurate perception modeling of disaster situation objects, the improved YOLOv8 neural network model is introduced to study the acquisition of accurate perception information of disaster situation objects. Secondly, the monocular vision positioning principle is used to study the method of quickly modeling disaster situation objects. In terms of simulation prediction analysis of smoke-fire spread, a flame spread algorithm is designed and combined with its physical characteristics to realize the simulation and dynamic update of tunnel fire smoke spread.

[0136] The light environment in tunnel fire scenarios is complex and there are many interference factors, resulting in a low accuracy of identifying disaster situation objects in tunnel fires. Moreover, the three-dimensional information that video images can express is limited, and the ability to perceive disaster situation information is weak. In addition, the current scene modeling methods mainly focus on manual modeling and parametric modeling. Such methods mainly focus on the refined display of static scenes. However, the tunnel fire environment is dynamic and uncertain, and the semantic knowledge and object spatial association relationships involved are complex, making it difficult to automatically model and difficult to meet the rapid modeling requirements of the tunnel fire environment.

[0137] To solve the problems of insufficient computing power, latency, low accuracy, and high cost of traditional perception technologies in the abnormal disaster situation scenarios of tunnel fires, a precise perception technology for tunnel fire disaster situation objects driven by video data that can be deployed on low-cost monitoring devices is developed. Extract the salient features of disaster situation objects in video data, and realize the rapid twin modeling of disaster situation objects from two-dimensional data to the three-dimensional engine space. On this basis, design the simulation analysis of the spread of flames and smoke in tunnel fire scenarios. The results can directly serve the informatization management in the operation process of highway tunnels, promote the development of tunnel intelligent fire protection theories and methods, and have broad application value and great practical significance.

[0138] The present invention provides a video data-driven tunnel fire detection and simulation method, and the specific steps are as follows:

[0139] Preprocess the obtained original data set to obtain a target data set, and the specific steps are as follows:

[0140] Obtain the original data set, and use the scene change detection method to extract video frames to obtain the original video frame image data set. Among them, the original data set includes tunnel fire monitoring video materials and the tunnel fire video data simulated in unity3d, and each image in the video frame image data set contains vehicles and flames;

[0141] Perform preliminary denoising and image enhancement processing on various tunnel fire scene images in the original video frame image data set. Select the images that exceed the given value through the threshold comparison method, perform diversity processing on the selected images, and transfer the unselected images back to the denoising and image enhancement processing steps until the requirements are met. Add 10% negative samples to the obtained data set after diversity processing, and perform reference learning by calculating the contrast loss function of positive and negative samples. Among them, exceeding the given value means exceeding the given resolution, clarity, contrast, richness of details, and the recognizable degree of target features at the same time. The diversity processing includes brightness enhancement, contrast enhancement, rotation at any angle, and composite scaling processing;

[0142] Use the LabelImg data annotation software to annotate each frame of the image after reference learning. The annotation information, the corresponding tunnel fire monitoring video data after reference learning, and the image frames extracted from the tunnel fire video data simulated in unity3d are combined to form multi-modal data and saved in the database, thus constructing a tunnel fire disaster object dataset, that is, obtaining the target dataset.

[0143] Considering that the early target of the flame is small, EfficientNetB0 is introduced as the backbone network to improve the model's efficient feature extraction ability and achieve lightweight processing. Then, a lightweight multi-semantic space attention mechanism (SMSA) module is introduced to fully utilize the collaborative potential of multi-semantic information for feature guidance and alleviating semantic differences, dynamically adjusting the weights of features according to the context of the input image, enabling the network to see both global and local information simultaneously, thereby enhancing the inductive bias. This enhances the model's sensitivity to the characteristics of disaster objects and improves the detection performance. Subsequently, a new feature extraction module is constructed, integrating the EfficientNetB0 network and the SMSA module and incorporating it into the backbone network, enabling the model to better extract the features of smoke-fire and vehicles in complex scenarios. During the training process, based on the YOLOv8 architecture, the modified feature extraction module is used to fine-tune the network. According to the actual requirements and tasks, the parameters of the EfficientNetB0 network, such as the number of channels and the size of the convolutional kernel, are adjusted to adapt to the new dataset and task. Also, the loss function and optimizer of YOLOv8 are modified to match the new EfficientNetB0 network. Finally, to improve the robustness of the model in complex scenarios, different hyperparameter settings, such as the learning rate, batch size, and number of iterations, are introduced to ensure that the model can converge quickly and effectively learn the features of flames and vehicles during training, realizing a refined tunnel fire scene object perception method.

[0144] Specifically:

[0145] Use the target dataset to train the improved YOLOv8 neural network model and perform object detection on the disaster situation images of the tunnel fire scene to be detected to obtain the center coordinates of each preliminary disaster object target. The specific steps are as follows:

[0146] Use the target dataset to train the improved YOLOv8 neural network model. After training, perform object detection on the disaster situation images of the tunnel fire scene to be detected to obtain the detection results of each preliminary disaster object target, and calculate the center coordinates Coordinate of each preliminary disaster object target based on the detection results. Among them, the detection results include the upper left coordinates (u1, v1) and the lower right coordinates (u2, v2) of each preliminary disaster object target. The preliminary disaster object targets include target vehicles and flames. The calculation formula for the center coordinates Coordinate of each preliminary disaster object target is:

[0147] Coordinate = ((u1 + u2) / 2, (v1 + v2) / 2).

[0148] Among them, the improved YOLOv8 neural network model is based on the YOLOv8 neural network model, introducing the EfficientNetB0 network, a multi-semantic space attention mechanism module, and a feature extraction module that integrates the EfficientNetB0 network and the multi-semantic space attention mechanism module as the backbone network;

[0149] The representation of the feature extraction module is:

[0150] F final (x) = SMSA(F(x))

[0151] Among them, F final (x) represents the features output by the feature extraction module, F(x) represents the feature extraction output of the EfficientNetB0 network, SMSA(F(x)) represents the feature map after F(x) is processed by the channel attention mechanism, and x is the disaster situation image of the tunnel fire scene to be detected. The twin model of the tunnel fire scene takes the real-time video stream of the monitoring camera as the input, constructs a spatial topological semantic association according to the two-dimensional tunnel fire disaster situation objects in the video data (that is, the geometric information perception result and attribute feature information of the distinguished disaster situation object target), and realizes the reverse association from the two-dimensional object to the three-dimensional geographical entity. Since there are some deviations between the target center obtained by directly solving the rectangular box through the two-dimensional image and the target center in the three-dimensional world, first, use a linear regression model to correct the center coordinates of the target rectangular box obtained by the preliminary target detection. Secondly, based on the principle of monocular vision positioning, combined with the projection matrix and pixel coordinates, perform the solution of spatial coordinates to complete the positioning of the disaster situation object target. After positioning, the geometric information of the disaster situation object target is accurately perceived and the attribute feature information is extracted to distinguish the disaster situation object target after positioning.

[0152] Next, based on the multi-modal and multi-scale information of the disaster situation objects obtained from the real-time video stream, such as the size and temporal characteristics of the flame, the type and texture of the vehicle, etc. Determine the relevant parameters for the twin modeling of the disaster situation object through the "perception result + knowledge model-driven" method, use the spatial topological semantic constraint conditions to map and instantiate the entity objects of the tunnel fire scene, and establish a twin model of the tunnel scene; finally, combine the real-time perception results of the dynamic and static disaster situation objects in the video image for change perception, and through constructing a series of rules and parameters, realize the program automation three-dimensional scene modeling in the initially constructed tunnel model.

[0153] Specifically:

[0154] Based on the center coordinates of each preliminary disaster situation object target, the corresponding preliminary disaster situation object target is corrected. After correction, the monocular vision positioning method is used to calibrate and distinguish the corrected disaster situation object target. Then, the distinguished disaster situation object target is used to perform real-time scene twin modeling in the unity engine, that is, the tunnel fire scene twin model is obtained and dynamically updated. The specific steps are as follows:

[0155] Using the linear regression model and the center coordinates of each preliminary disaster situation object target to correct each preliminary disaster situation object target, and obtaining the corrected disaster situation object target;

[0156] Based on the monocular vision positioning method, combined with the projection matrix and pixel coordinates of the corrected disaster situation object target, the spatial coordinates are calculated. The obtained spatial coordinates after calculation are the positioning of the corrected disaster situation object target. Then, based on the geometric information perception and attribute feature information extraction of each positioned disaster situation object target, each positioned disaster situation object target is distinguished to obtain the distinction result. Among them, the geometric perception information includes geometric dimensions, spatial range, and spatial position relationship. The geometric dimensions include the size and shape of the flame, and the spatial position relationship includes the relative position relationship between the flame and each target vehicle. The attribute feature information includes the type and texture of the target vehicle, the state and temporal characteristics of the flame. The distinction result includes distinguishing the target vehicle into small passenger cars, large trucks, and large buses;

[0157] The geometric dimension L of the positioned disaster situation object target is:

[0158]

[0159] Among them, W is the actual width of the tunnel, W b and D are the bounding box width of each positioned disaster situation object target and the distance from the center of each positioned disaster situation object target to the camera respectively;

[0160] The spatial range boundingbox of the positioned disaster situation object target is:

[0161]

[0162] Among them, (x b , y b ) represents the center point coordinates of the positioned disaster situation object target, and z1 and z2 are the ranges of the positioned disaster situation object target in the depth direction, representing the minimum depth value and the maximum depth value respectively;

[0163] Based on the geometric information perception results and attribute feature information of each differentiated disaster situation object target, determine the twin modeling parameters of the disaster situation object target through the "perception result + knowledge model driven" method. Subsequently, based on the twin modeling parameters, use the spatial topological semantic constraint conditions to map and instantiate the tunnel fire scene entity objects in the tunnel fire scene image in the unity engine, that is, obtain the tunnel fire scene twin model. The specific steps are as follows:

[0164] Based on the geometric perception results, attribute feature information of each differentiated disaster situation object target, and multi-dimensional and multi-scale scene image information from local and global, use the graph database to take each differentiated disaster situation object target and its attribute feature information as nodes, and the geometric perception result as the relationship as the edge to perform data modeling and relationship mapping. After mapping, use the spatial data fusion method to integrate the geometric structure of the tunnel, the fire location, and the geometric perception results and attribute feature information of each differentiated disaster situation object target to form a dynamic three-dimensional knowledge graph, so as to obtain the twin modeling parameters of each differentiated disaster situation object target, including spatial coordinates, temperature coefficient, and flame combustion rate. Among them, the multi-dimensional and multi-scale scene image information includes flame image information in the initial, middle, and late stages of the fire, and tunnel fire scene image information with different heights, widths, and depths.

[0165] Based on the obtained twin modeling parameters, use the spatial topological semantic constraint conditions to map and instantiate each differentiated disaster situation object target in the tunnel fire scene image in the unity engine to establish a tunnel fire scene twin model. Among them, the spatial topological semantic constraint conditions include adjacency relationship, inclusion relationship, object type constraint, and environmental constraint.

[0166] According to the tunnel fire scene image obtained in real time, re-perform preliminary disaster situation object target detection, correction, positioning, and differentiation to update the initial twin modeling of the tunnel fire scene.

[0167] Most traditional tunnel fire spread simulation methods are based on predefined models and historical data, lacking real-time performance and dynamic adjustment capabilities, and it is difficult to accurately reflect the complex behavior and influencing factors of the fire. The present invention proposes a tunnel fire spread simulation method based on video perception and Unity3D, which uses the flame characteristic parameters extracted from the video to dynamically simulate the fire spread process through the entire flame and flame particles.

[0168] Specifically:

[0169] Based on the tunnel fire scene twin model, perform flame spread analysis on the tunnel fire scene and achieve dynamic update in unity. The specific steps are as follows:

[0170] Extract the spread characteristics of the twin model of the tunnel fire scene from the disaster situation images of the tunnel fire scene. Construct one or more flame spread models in Unity3D. Among them, the flame spread model includes the change of the flame over time, the growth of the flame height, and the change of the flame spread shape. The change of the flame spread shape includes the generation position of the flame particles, the change of the transparency of the flame particles over time, the color change, and the speed adjustment of the wind speed particles;

[0171] Based on the linear expansion characteristic of the expansion mode of the combustion area, the formula for the change of the flame over time is:

[0172] R f (t) = R0 + v f ·t

[0173] Among them, R f (t) represents the change of the flame at time t, R0 is the initial radius of the flame, and v f represents the flame spread speed, which is extracted from the twin model of the tunnel fire scene;

[0174] The formula for the growth of the flame height is:

[0175] H f (t) = H0 + v s ·t

[0176] Among them, H f (t) represents the growth of the flame height at time t, H0 is the initial height of the flame, and v s is the expansion speed of the flame in the vertical direction;

[0177] The change of the flame spread shape is to simulate the change of the flame spread shape through the generation position, transparency, color change of the flame particles and the speed adjustment of the wind speed particles;

[0178] The formula for the generation position of the flame particles is:

[0179] x i = x0 + R·cos(θ)·u

[0180] y i = y0 + R·sin(θ)·v

[0181] z i = z0 + H·w

[0182] Among them, R is the expansion radius, which represents the size of the flame combustion area and is uniformly randomly generated by the flame particles. (x i , y i , z i) is the generation position coordinate of the i-th flame particle, (x0, y0, z0) represents the central position of the flame combustion area, θ is the angular parameter representing the flame spreading direction, H is the vertical height of the flame, and u, v, and w are random numbers used to simulate the random distribution of flame particles within the flame combustion area;

[0183] The transparency of the flame particles changes with time to simulate the blurring effect at the flame edge and the dynamic changes during the combustion process. The formula is:

[0184]

[0185] where α i (t) represents the transparency of the i-th flame particle at time t, represents the initial transparency of the i-th flame particle, and T represents the life cycle of the flame;

[0186] The color change of the flame is obtained by analyzing the RGB values of the flame through a frame-by-frame analysis method. The formula is:

[0187]

[0188] where Color i (t) represents the color of the i-th flame particle at time t, represents the initial color of the i-th flame particle, represents the color after the i-th flame particle burns out, and T represents the life cycle of the flame particle;

[0189] The formula for adjusting the speed based on the wind speed particles is:

[0190]

[0191] where v i (t) represents the speed of the i-th flame particle at time t, represents the initial speed of the i-th flame particle under windless conditions, α represents the wind speed influence coefficient, represents the wind speed at time t;

[0192] Calculate the Euclidean distance between the flame particles in the simulated flame spread model and the target vehicle. If the distance is less than the given threshold, it is considered that any flame particle has contacted the target object, and then go to step 4.1 to generate a new flame spread simulation. Otherwise, do not simulate. Among them, the distance between the flame particle and the target vehicle is calculated by the distance formula in three-dimensional space:

[0193]

[0194] where d i represents the distance between the i-th flame particle and each target vehicle, (xf , y f , z f ) is the position of the i-th flame particle in the Unity three-dimensional space, (x c , y c , z c ) is the center position of each target vehicle. When the distance d i is less than the predetermined threshold R trigger , a new flame spread model is generated at the position of the corresponding target vehicle;

[0195] Based on all the flame spread models and the twin model of the tunnel fire scene of the tunnel fire scene disaster image, the flame spread of the tunnel fire scene is simulated and dynamically updated.

Claims

1. A video data driven tunnel fire detection and simulation method, characterized in that: The steps include: Step 1: preprocess the acquired original data set to obtain the target data set; Step 2: Use the target data set to train the improved YOLOv8 neural network model, and perform target detection on the tunnel fire scene disaster image to be detected to obtain the center coordinates of each preliminary disaster object target; Step 3: Based on the central coordinates of each preliminary disaster target, the corresponding preliminary disaster target is corrected. After correction, the corrected disaster target is calibrated and distinguished using a monocular vision positioning method. The distinguished disaster target is then used to perform real-time scene twin modeling in the Unity engine, thus obtaining a tunnel fire scene twin model and dynamically updating it. Step 4: Perform flame spread analysis on the tunnel fire scene based on the tunnel fire scene twin model and implement dynamic update in Unity; The specific steps of step 4 are: Step 4.1, extracting the spread characteristics of the twin model of the tunnel fire scene from the tunnel fire scene disaster image, constructing one or more flame spread models in Unity3D, wherein the flame spread model includes the change of flame over time, the growth of flame height and the change of flame spread shape, and the change of flame spread shape includes the generation position of flame particles, the change of transparency of flame particles over time, the color change and the speed adjustment of wind speed particles; Based on the linear expansion characteristics of the combustion area, the formula for the change of flame over time is: in, express The flame changes from moment to moment, is the initial radius of the flame, represents the flame spread speed, which is extracted from the twin model of the tunnel fire scene; The flame height growth formula is: in, express The flames grow in height from moment to moment, is the initial height of the flame, is the flame expansion speed in the vertical direction; The flame spreading shape change is simulated by adjusting the generation position, transparency, color change of flame particles and the speed of wind particles; The formula for the flame particle generation position is: in, is the expansion radius, which indicates the size of the flame combustion area and is generated uniformly and randomly by flame particles. ( , , ) is the The generated position coordinates of the flame particles, ( ) indicates the center position of the flame burning area, Angle parameter used to indicate the direction of flame spread. is the vertical height of the flame, , and is a random number used to simulate the random distribution of flame particles in the flame combustion area; The transparency of the flame particles changes over time to simulate the blurring effect of the flame edge and the dynamic changes of the combustion process. The formula is: in, express Moment The transparency of each flame particle. Indicates The initial transparency of the flame particles, Represents the life cycle of a flame; The color change of the flame is obtained by analyzing the RGB value of the flame frame by frame, and the formula is: in, express Moment The color of the flame particles, Indicates The initial color of the flame particles, Indicates The color of the flame particles after they burn out. Represents the life cycle of flame particles; The formula for adjusting the speed of particles based on wind speed is: in, express Moment The speed of the flame particles, Indicates The initial velocity of a flame particle in windless conditions is represents the wind speed influence coefficient, express Wind speed at the moment; Step 4.2: Calculate the Euclidean distance between the flame particles and the target vehicle in the simulated flame spread model. If the distance is less than a given threshold, it is considered that any flame particle has touched the target object, and then go to step 4.1 to generate a new flame spread simulation. Otherwise, do not simulate. The distance between the flame particles and the target vehicle is calculated using the distance formula in three-dimensional space: in, Indicates The distance between each flame particle and each target vehicle, It is The position of the flame particles in Unity's 3D space, is the center position of each target vehicle. Less than a predetermined threshold When , a new flame spread model is generated at the location of the corresponding target vehicle; Step 4.3: Based on all flame spread models of the tunnel fire scene disaster image and the tunnel fire scene twin model, the flame spread of the tunnel fire scene is simulated and dynamically updated.

2. A video data driven tunnel fire detection and simulation method according to claim 1, characterized in that: The specific steps of step 1 are: Step 1.1, obtaining an original data set, and using a scene change detection method to extract video frames to obtain an original video frame image data set, wherein the original data set includes tunnel fire monitoring video data and tunnel fire video data simulated in unity3d, and each image in the video frame image data set contains a vehicle and a flame; Step 1.2: Perform preliminary denoising and image enhancement processing on various tunnel fire scene images in the original video frame image data set, select images that exceed a given value by using a threshold comparison method, perform diversity processing on the selected images, and transfer the unselected images to the denoising and image enhancement processing steps again until the requirements are met. Add 10% of negative samples to the data set after diversity processing, and perform reference learning by calculating the contrast loss function of positive and negative samples, where exceeding a given value means simultaneously exceeding a given resolution, clarity, contrast, detail richness, and recognizability of target features. Diversity processing includes brightness enhancement, contrast enhancement, rotation at any angle, and compound scaling processing; Step 1.3, use LabelImg data annotation software to annotate each frame of the image after reference learning obtained in step 1.2, and store the annotation information together with the corresponding tunnel fire monitoring video data after reference learning and the image frames extracted from the tunnel fire video data simulated in unity3d to form multimodal data in the database, thus constructing a tunnel fire disaster object dataset, that is, obtaining the target dataset.

3. A video data driven tunnel fire detection and simulation method according to claim 2, characterized in that: The improved YOLOv8 neural network model in step 2 is based on the YOLOv8 neural network model, introducing the Effici-entNetB0 network, the multi-semantic space attention mechanism module, and the feature extraction module integrating the Effici-entNetB0 network and the multi-semantic space attention mechanism module as the backbone network; The feature extraction module is represented as: in, represents the features output by the feature extraction module, represents the feature extraction output of the EfficientNetB0 network, express Feature map after channel attention mechanism processing.

4. A video data driven tunnel fire detection and simulation method according to claim 3, characterized in that: The specific steps of step 2 are: The improved YOLOv8 neural network model is trained using the target data set. After training, target detection is performed on the tunnel fire scene disaster image to be detected, and the detection results of each preliminary disaster object target are obtained. The center coordinates of each preliminary disaster object target are calculated based on the detection results. , where the detection results include the upper left corner coordinates of each preliminary disaster target and the lower right corner coordinates The initial disaster targets include target vehicles and flames. The center coordinates of each initial disaster target are The calculation formula is: 。 5. A video data driven tunnel fire detection and simulation method according to claim 4, characterized in that: The specific steps of step 3 are: Step 3.1, using the linear regression model and the central coordinates of each preliminary disaster object target to correct each preliminary disaster object target, to obtain a corrected disaster object target; Step 3.2, based on the monocular vision positioning method, the spatial coordinates are solved in combination with the projection matrix and pixel coordinates of the corrected disaster target. The spatial coordinates obtained after the solution are the positioning of the corrected disaster target. Then, based on the geometric information perception and attribute feature information extraction of each located disaster target, each located disaster target is distinguished to obtain a distinction result, wherein the geometric perception information includes geometric dimensions, spatial range and spatial position relationship, the geometric dimensions include the size and shape of the flame, the spatial position relationship includes the relative position relationship between the flame and each target vehicle, the attribute feature information includes the type and texture of the target vehicle, the state and timing characteristics of the flame, and the distinction result includes distinguishing the target vehicle into a small passenger car, a large truck and a large passenger car; The geometric dimensions of the disaster target after positioning for: in, is the actual width of the tunnel, and are respectively the width of the bounding box of each located disaster object and the distance from the center of each located disaster object to the camera; The spatial range of the disaster target after positioning for: in, Indicates the coordinates of the center point of the disaster target after positioning, and is the range of the disaster target in the depth direction after positioning, representing the minimum depth and the maximum depth respectively; Step 3.3: Based on the geometric information perception results and attribute feature information of each differentiated disaster object target, the twin modeling parameters of the disaster object target are determined by the "perception result + knowledge model driven" method. Then, based on the twin modeling parameters, the tunnel fire scene entity objects in the tunnel fire scene disaster image are mapped and instantiated in the unity engine using the spatial topological semantic constraints to obtain the tunnel fire scene twin model. The specific steps are as follows: Based on the geometric perception results, attribute feature information and multi-dimensional and multi-scale scene image information from local and global aspects of each disaster object target after differentiation, the graph database is used to take each differentiated disaster object target and its attribute feature information as nodes, and the geometric perception results as relationships as edges, to perform data modeling and relationship mapping. After mapping, the geometric structure of the tunnel, the fire location and the geometric perception results and attribute feature information of each differentiated disaster object target are integrated through the spatial data fusion method to form a dynamic three-dimensional knowledge graph, so as to obtain the twin modeling parameters of each differentiated disaster object target, including spatial coordinates, temperature coefficient and flame burning rate. Among them, the multi-dimensional and multi-scale scene image information includes flame image information in the early, middle and late stages of the fire, and tunnel fire scene image information of different heights, widths and depths; Based on the twin modeling parameters obtained, the spatial topological semantic constraints are used to map and instantiate the different disaster objects in the tunnel fire scene disaster image in the Unity engine to establish a twin model of the tunnel fire scene. The spatial topological semantic constraints include adjacency, inclusion, object type constraints, and environmental constraints. Step 3.4: Based on the real-time acquired tunnel fire scene disaster image, preliminary disaster object detection, correction, positioning and differentiation are re-performed to update the initial twin modeling of the tunnel fire scene.

6. A video data driven tunnel fire detection and simulation system, characterized in that: include: Target data set construction module: preprocess the acquired original data set to obtain the target data set; Target detection module: The target data set is used to train the improved YOLOv8 neural network model, and target detection is performed on the tunnel fire scene disaster image to be detected to obtain the center coordinates of each preliminary disaster object target; Twin modeling module: Based on the central coordinates of each preliminary disaster object, the corresponding preliminary disaster object is corrected. After correction, the corrected disaster object is calibrated and distinguished using a monocular vision positioning method. The distinguished disaster object is then used to perform real-time scene twin modeling in the Unity engine, thus obtaining a tunnel fire scene twin model and dynamically updating it. Flame spread module: analyzes the flame spread of tunnel fire scenes based on the twin model of tunnel fire scenes and implements dynamic updates in Unity; The specific implementation steps of the flame spread module are: Step 4.1, extracting the spread characteristics of the twin model of the tunnel fire scene from the tunnel fire scene disaster image, constructing one or more flame spread models in Unity3D, wherein the flame spread model includes the change of flame over time, the growth of flame height and the change of flame spread shape, and the change of flame spread shape includes the generation position of flame particles, the change of transparency of flame particles over time, the color change and the speed adjustment of wind speed particles; Based on the linear expansion characteristics of the combustion area, the formula for the change of flame over time is: in, express The flame changes from moment to moment, is the initial radius of the flame, represents the flame spread speed, which is extracted from the twin model of the tunnel fire scene; The flame height growth formula is: in, express The flames grow in height from moment to moment, is the initial height of the flame, is the flame expansion speed in the vertical direction; The flame spreading shape change is simulated by adjusting the generation position, transparency, color change of flame particles and the speed of wind particles; The formula for the flame particle generation position is: in, is the expansion radius, which indicates the size of the flame combustion area and is generated uniformly and randomly by flame particles. ( , , ) is the The generated position coordinates of the flame particles, ( ) indicates the center position of the flame burning area, Angle parameter used to indicate the direction of flame spread. is the vertical height of the flame, , and is a random number used to simulate the random distribution of flame particles in the flame combustion area; The transparency of the flame particles changes over time to simulate the blurring effect of the flame edge and the dynamic changes of the combustion process. The formula is: in, express Moment The transparency of each flame particle. Indicates The initial transparency of the flame particles, Represents the life cycle of a flame; The color change of the flame is obtained by analyzing the RGB value of the flame frame by frame, and the formula is: in, express Moment The color of the flame particles, Indicates The initial color of the flame particles, Indicates The color of the flame particles after they burn out. Represents the life cycle of flame particles; The formula for adjusting the speed of particles based on wind speed is: in, express Moment The speed of the flame particles, Indicates The initial velocity of a flame particle in windless conditions is represents the wind speed influence coefficient, express Wind speed at the moment; Step 4.2: Calculate the Euclidean distance between the flame particles and the target vehicle in the simulated flame spread model. If the distance is less than a given threshold, it is considered that any flame particle has touched the target object, and then go to step 4.1 to generate a new flame spread simulation. Otherwise, do not simulate. The distance between the flame particles and the target vehicle is calculated using the distance formula in three-dimensional space: in, Indicates The distance between each flame particle and each target vehicle, It is The position of the flame particles in Unity's 3D space, is the center position of each target vehicle. Less than a predetermined threshold When , a new flame spread model is generated at the location of the corresponding target vehicle; Step 4.3: Based on all flame spread models of the tunnel fire scene disaster image and the tunnel fire scene twin model, the flame spread of the tunnel fire scene is simulated and dynamically updated.

7. A video data driven tunnel fire detection and simulation system according to claim 6, characterized in that: The specific implementation steps of the target data set construction module are: Step 1.1, obtaining an original data set, and using a scene change detection method to extract video frames to obtain an original video frame image data set, wherein the original data set includes tunnel fire monitoring video data and tunnel fire video data simulated in unity3d, and each image in the video frame image data set contains a vehicle and a flame; Step 1.2: Perform preliminary denoising and image enhancement processing on various tunnel fire scene images in the original video frame image data set, select images that exceed a given value by using a threshold comparison method, perform diversity processing on the selected images, and transfer the unselected images to the denoising and image enhancement processing steps again until the requirements are met. Add 10% of negative samples to the data set after diversity processing, and perform reference learning by calculating the contrast loss function of positive and negative samples, where exceeding a given value means simultaneously exceeding a given resolution, clarity, contrast, detail richness, and recognizability of target features. Diversity processing includes brightness enhancement, contrast enhancement, rotation at any angle, and compound scaling processing; Step 1.3, use LabelImg data annotation software to annotate each frame of the image after reference learning obtained in step 1.2, and store the annotation information together with the corresponding tunnel fire monitoring video data after reference learning and the image frames extracted from the tunnel fire video data simulated in unity3d to form multimodal data in the database, thus constructing a tunnel fire disaster object dataset, that is, obtaining the target dataset.

8. A video data driven tunnel fire detection and simulation system according to claim 7, characterized in that: The improved YOLOv8 neural network model in the target detection module is based on the YOLOv8 neural network model, which introduces the Effici-entNetB0 network, the multi-semantic space attention mechanism module, and the feature extraction module integrating the Effici-entNetB0 network and the multi-semantic space attention mechanism module as the backbone network; The feature extraction module is represented as: in, represents the features output by the feature extraction module, represents the feature extraction output of the EfficientNetB0 network, express Feature map after channel attention mechanism processing; The specific implementation steps of the target detection module are: The improved YOLOv8 neural network model is trained using the target data set. After training, target detection is performed on the tunnel fire scene disaster image to be detected, and the detection results of each preliminary disaster object target are obtained. The center coordinates of each preliminary disaster object target are calculated based on the detection results. , where the detection results include the upper left corner coordinates of each preliminary disaster target and the lower right corner coordinates The initial disaster targets include target vehicles and flames. The center coordinates of each initial disaster target are The calculation formula is: ； The specific steps of the twin model modeling module are: Step 3.1, using the linear regression model and the central coordinates of each preliminary disaster object target to correct each preliminary disaster object target, to obtain a corrected disaster object target; Step 3.2, based on the monocular vision positioning method, the spatial coordinates are solved in combination with the projection matrix and pixel coordinates of the corrected disaster target. The spatial coordinates obtained after the solution are the positioning of the corrected disaster target. Then, based on the geometric information perception and attribute feature information extraction of each located disaster target, each located disaster target is distinguished to obtain a distinction result, wherein the geometric perception information includes geometric dimensions, spatial range and spatial position relationship, the geometric dimensions include the size and shape of the flame, the spatial position relationship includes the relative position relationship between the flame and each target vehicle, the attribute feature information includes the type and texture of the target vehicle, the state and timing characteristics of the flame, and the distinction result includes distinguishing the target vehicle into a small passenger car, a large truck and a large passenger car; The geometric dimensions of the disaster target after positioning for: in, is the actual width of the tunnel, and are respectively the width of the bounding box of each located disaster object and the distance from the center of each located disaster object to the camera; The spatial range of the disaster target after positioning for: in, Indicates the coordinates of the center point of the disaster target after positioning, and is the range of the disaster target in the depth direction after positioning, representing the minimum depth and the maximum depth respectively; Step 3.3: Based on the geometric information perception results and attribute feature information of each differentiated disaster object target, the twin modeling parameters of the disaster object target are determined by the "perception result + knowledge model driven" method. Then, based on the twin modeling parameters, the tunnel fire scene entity objects in the tunnel fire scene disaster image are mapped and instantiated in the unity engine using the spatial topological semantic constraints to obtain the tunnel fire scene twin model. The specific steps are as follows: Based on the geometric perception results, attribute feature information and multi-dimensional and multi-scale scene image information from local and global aspects of each disaster object target after differentiation, the graph database is used to take each differentiated disaster object target and its attribute feature information as nodes, and the geometric perception results as relationships as edges, to perform data modeling and relationship mapping. After mapping, the geometric structure of the tunnel, the fire location and the geometric perception results and attribute feature information of each differentiated disaster object target are integrated through the spatial data fusion method to form a dynamic three-dimensional knowledge graph, so as to obtain the twin modeling parameters of each differentiated disaster object target, including spatial coordinates, temperature coefficient and flame burning rate. Among them, the multi-dimensional and multi-scale scene image information includes flame image information in the early, middle and late stages of the fire, and tunnel fire scene image information of different heights, widths and depths; Based on the twin modeling parameters obtained, the spatial topological semantic constraints are used to map and instantiate the different disaster objects in the tunnel fire scene disaster image in the Unity engine to establish a twin model of the tunnel fire scene. The spatial topological semantic constraints include adjacency, inclusion, object type constraints, and environmental constraints. Step 3.4: Based on the real-time acquired tunnel fire scene disaster image, preliminary disaster object detection, correction, positioning and differentiation are re-performed to update the initial twin modeling of the tunnel fire scene.

Images