Video data driven tunnel fire detection and simulation method and system

Abstract

The invention discloses a tunnel fire detection and simulation method and system driven by video data, belongs to the technical field of computer vision and machine learning, and solves the problem that the current tunnel fire target detection algorithm is easy to cause insufficient distinguishing capability of vehicles and flames in a complex multi-target scene environment. The method comprises the following steps: preprocessing an obtained original data set to obtain a target data set to train an improved YOLOv8 neural network model, and carrying out target detection on a to-be-detected tunnel fire scene disaster image to obtain a central coordinate of each preliminary disaster object target for correction; after correction, a monocular vision positioning method is adopted to carry out position calibration and distinguishing on the corrected disaster situation object target, and then real-time scene twinborn modeling is carried out, so that a tunnel fire scene twinborn model is obtained and dynamically updated; and performing flame spread analysis on the tunnel fire scene based on the tunnel fire scene twinborn model, and realizing dynamic updating in un ity. The method is used for tunnel fire detection and simulation driven by video data.

Description

Technical Field

[0001] A video data-driven tunnel fire detection and simulation method and system are used for video data-driven tunnel fire detection and simulation, belonging to the technical field of computer vision and machine learning. Background Art

[0002] Tunnels are important facilities for national infrastructure construction and occupy a very important position in the development of the national economy. As an important part of infrastructure, including highway tunnels, railway tunnels, underground passages, integrated pipe corridors, etc., they provide great convenience for our transportation, living and travel. Due to its closed nature, space limitations and low visibility, the internal environment of the tunnel often leads to serious casualties and property losses once a fire occurs. However, due to the unavoidable traffic accidents and car failures, tunnel fires occur from time to time. Compared with fires in open sections, fires in closed tunnels will develop rapidly, and the high-temperature toxic smoke produced cannot be eliminated, which is the main cause of serious disasters. At the same time, the complex environment in the tunnel and the low visibility after the fire make it difficult for firefighters to quickly judge the fire situation inside and take timely and effective firefighting and rescue measures. Statistics show that from 2000 to 2016, there were 161 medium and large tunnel fire accidents in my country, causing significant casualties and economic losses. In order to guide fire-fighting and rescue work, it has been clearly proposed to improve the intelligent management level of tunnels, and emphasized that advanced information technology should be deeply integrated into the tunnel fire protection system to establish a more intelligent fire protection management system. This is the key direction to improve the level of informationization and intelligence of tunnel management.

[0003] Digital twin technology creates a comprehensive, interconnected and highly realistic scene twin model by accurately mapping the properties, states and behaviors of objects in the physical world to the digital world. This model supports interactive exploration and analysis, enabling digital space to diagnose problems, evaluate states and predict future trends. In addition, the application of digital twins in disaster prevention and mitigation can not only provide early warning of disasters, but also optimize resource allocation and analyze disaster evolution, thereby enhancing the perception, understanding and decision-making capabilities of emergency responders on disaster scenes. Therefore, building an environmental twin model of tunnel fires is crucial to improving the level of information management of tunnel fire prevention and control. Surveillance video is an important technical means for data perception. It has the advantages of real-time, economical, reliable and convenient, and has become an indispensable part of tunnel management information construction. Based on tunnel surveillance video, vehicle information is fully perceived, the accuracy of information perception is improved, and higher-precision data can be provided for twin modeling of vehicles in tunnel scenes. However, the low-light environment of the tunnel poses a huge challenge to the comprehensive perception of vehicle information based on tunnel surveillance video.

[0004] In view of the weak light environment of the tunnel, the tunnel fire scene modeling methods in the existing technology are mainly based on manual modeling and parametric modeling. Such methods mainly focus on the refined display of static scenes. However, the tunnel fire environment is dynamic and uncertain. The semantic knowledge and object space association relationships involved are complex, and automatic modeling is difficult to meet the needs of rapid modeling of the tunnel fire environment.

[0005] In summary, the prior art has the following technical problems:

[0006] 1. Due to the presence of various interference factors in tunnels, such as reflection, shadow, noise, etc., the current tunnel fire target detection algorithm often performs poorly in complex multi-target scene environments, which can easily lead to insufficient ability to distinguish between vehicles and flames, resulting in low accuracy in obtaining target information of disaster objects, low model lightweight, low computational efficiency, and failure to meet the high real-time requirements of tunnel fire scene modeling;

[0007] 2. Traditional tunnel fire scene object positioning methods usually directly use the center of the rectangular frame of the target detection result as the center of the disaster object in the real world, which causes a large positioning error, directly affecting the authenticity of the tunnel fire scene twin modeling, and further affecting the results of the subsequent flame spread analysis;

[0008] 3. The current tunnel fire scene modeling methods generally lack modular processing of flame spread analysis, usually only provide static fire models, cannot reflect the dynamic changes of flames in the tunnel in real time, and cannot effectively predict and evaluate the flame propagation path and speed. Summary of the invention

[0009] The purpose of the present invention is to provide a video data driven tunnel fire detection and simulation method and system to solve the problem that the current tunnel fire target detection algorithm often performs less than ideally in complex multi-target scene environments, which easily leads to insufficient ability to distinguish between vehicles and flames, resulting in low accuracy in obtaining target information of disaster objects, low model lightweightness, low computational efficiency and failure to meet the high real-time requirements of tunnel fire scene modeling.

[0010] In order to achieve the above object, the technical solution adopted by the present invention is:

[0011] A video data driven tunnel fire detection and simulation method comprises the following steps:

[0012] Step 1: preprocess the acquired original data set to obtain the target data set;

[0013] Step 2: Use the target data set to train the improved YOLOv8 neural network model, and perform target detection on the tunnel fire scene disaster image to be detected to obtain the center coordinates of each preliminary disaster object target;

[0014] Step 3: Based on the central coordinates of each preliminary disaster target, the corresponding preliminary disaster target is corrected. After correction, the corrected disaster target is calibrated and distinguished using a monocular vision positioning method. The distinguished disaster target is then used to perform real-time scene twin modeling in the Unity engine, thus obtaining a tunnel fire scene twin model and dynamically updating it.

[0015] Step 4: Perform flame spread analysis on the tunnel fire scene based on the tunnel fire scene twin model and implement dynamic updates in Unity.

[0016] Further, the specific steps of step 1 are:

[0017] Step 1.1, obtaining an original data set, and using a scene change detection method to extract video frames to obtain an original video frame image data set, wherein the original data set includes tunnel fire monitoring video data and tunnel fire video data simulated in unity3d, and each image in the video frame image data set contains a vehicle and a flame;

[0018] Step 1.2: Perform preliminary denoising and image enhancement processing on various tunnel fire scene images in the original video frame image data set, select images that exceed a given value by using a threshold comparison method, perform diversity processing on the selected images, and transfer the unselected images to the denoising and image enhancement processing steps again until the requirements are met. Add 10% of negative samples to the data set after diversity processing, and perform reference learning by calculating the contrast loss function of positive and negative samples, where exceeding a given value means simultaneously exceeding a given resolution, clarity, contrast, detail richness, and recognizability of target features. Diversity processing includes brightness enhancement, contrast enhancement, rotation at any angle, and compound scaling processing;

[0019] Step 1.3, use LabelImg data annotation software to annotate each frame of the image after reference learning obtained in step 1.2, and store the annotation information together with the corresponding tunnel fire monitoring video data after reference learning and the image frames extracted from the tunnel fire video data simulated in unity3d to form multimodal data in the database, thus constructing a tunnel fire disaster object dataset, that is, obtaining the target dataset.

[0020] Furthermore, the improved YOLOv8 neural network model in step 2 is based on the YOLOv8 neural network model, introducing the Effici-entNetB0 network, the multi-semantic space attention mechanism module, and the feature extraction module integrating the Effici-entNetB0 network and the multi-semantic space attention mechanism module as the backbone network;

[0021] The feature extraction module is represented as:

[0022] F final (x) = SMSA(F(x))

[0023] Among them, F final (x) represents the feature output by the feature extraction module, F(x) represents the feature extraction output of the EfficientNetB0 network, and SMSA(F(x)) represents the feature map of F(x) after being processed by the channel attention mechanism.

[0024] Further, the specific steps of step 2 are:

[0025] The improved YOLOv8 neural network model is trained using the target data set. After training, target detection is performed on the tunnel fire scene disaster image to be detected to obtain the detection results of each preliminary disaster object target, and the center coordinate Coordinate of each preliminary disaster object target is calculated based on the detection results. The detection results include the upper left corner coordinates (u1, v1) and lower right corner coordinates (u2, v2) of each preliminary disaster object target. The preliminary disaster object targets include target vehicles and flames. The calculation formula of the center coordinate Coordinate of each preliminary disaster object target is:

[0026] Coordinate=((u1+u2) / 2,(v1+v2) / 2).

[0027] Further, the specific steps of step 3 are:

[0028] Step 3.1, using the linear regression model and the central coordinates of each preliminary disaster object target to correct each preliminary disaster object target, to obtain a corrected disaster object target;

[0029] Step 3.2, based on the monocular vision positioning method, the spatial coordinates are solved in combination with the projection matrix and pixel coordinates of the corrected disaster target. The spatial coordinates obtained after the solution are the positioning of the corrected disaster target. Then, based on the geometric information perception and attribute feature information extraction of each located disaster target, each located disaster target is distinguished to obtain a distinction result, wherein the geometric perception information includes geometric dimensions, spatial range and spatial position relationship, the geometric dimensions include the size and shape of the flame, the spatial position relationship includes the relative position relationship between the flame and each target vehicle, the attribute feature information includes the type and texture of the target vehicle, the state and timing characteristics of the flame, and the distinction result includes distinguishing the target vehicle into a small passenger car, a large truck and a large passenger car;

[0030] The geometric size L of the disaster target after positioning is:

[0031]

[0032] Where W is the actual width of the tunnel, W b and D are respectively the width of the bounding box of each located disaster object and the distance from the center of each located disaster object to the camera;

[0033] The spatial range boundingbox of the located disaster target is:

[0034]

[0035] Among them, (x b ,y b ) represents the coordinates of the center point of the disaster target after positioning, z1 and z2 are the ranges of the disaster target in the depth direction after positioning, representing the minimum depth and the maximum depth respectively;

[0036] Step 3.3: Based on the geometric information perception results and attribute feature information of each differentiated disaster object target, the twin modeling parameters of the disaster object target are determined by the "perception result + knowledge model driven" method. Then, based on the twin modeling parameters, the tunnel fire scene entity objects in the tunnel fire scene disaster image are mapped and instantiated in the unity engine using the spatial topological semantic constraints to obtain the tunnel fire scene twin model. The specific steps are as follows:

[0037] Based on the geometric perception results, attribute feature information and multi-dimensional and multi-scale scene image information from local and global aspects of each disaster object target after differentiation, the graph database is used to take each differentiated disaster object target and its attribute feature information as nodes, and the geometric perception results as relationships as edges, to perform data modeling and relationship mapping. After mapping, the geometric structure of the tunnel, the fire location and the geometric perception results and attribute feature information of each differentiated disaster object target are integrated through the spatial data fusion method to form a dynamic three-dimensional knowledge graph, so as to obtain the twin modeling parameters of each differentiated disaster object target, including spatial coordinates, temperature coefficient and flame burning rate. Among them, the multi-dimensional and multi-scale scene image information includes flame image information in the early, middle and late stages of the fire, and tunnel fire scene image information of different heights, widths and depths;

[0038] Based on the twin modeling parameters obtained, the spatial topological semantic constraints are used to map and instantiate the different disaster objects in the tunnel fire scene disaster image in the Unity engine to establish a twin model of the tunnel fire scene. The spatial topological semantic constraints include adjacency, inclusion, object type constraints, and environmental constraints.

[0039] Step 3.4: Based on the real-time acquired tunnel fire scene disaster image, preliminary disaster object detection, correction, positioning and differentiation are re-performed to update the initial twin modeling of the tunnel fire scene.

[0040] Further, the specific steps of step 4 are:

[0041] Step 4.1, extracting the spread characteristics of the twin model of the tunnel fire scene from the tunnel fire scene disaster image, constructing one or more flame spread models in Unity3D, wherein the flame spread model includes the change of flame over time, the growth of flame height and the change of flame spread shape, and the change of flame spread shape includes the generation position of flame particles, the change of transparency of flame particles over time, the color change and the speed adjustment of wind speed particles;

[0042] Based on the linear expansion characteristics of the combustion area, the formula for the change of flame over time is:

[0043] R f (t) = R0 + v f ·t

[0044] Among them, R f (t) represents the change of the flame at time t, R0 is the initial radius of the flame, v f represents the flame spread speed, which is extracted from the twin model of the tunnel fire scene;

[0045] The flame height growth formula is:

[0046] H f (t)=H0+v s ·t

[0047] Among them, H f (t) represents the growth of flame height at time t, H0 is the initial height of the flame, v s is the flame expansion speed in the vertical direction;

[0048] The flame spreading shape change is simulated by adjusting the generation position, transparency, color change of flame particles and the speed of wind particles;

[0049] The formula for the flame particle generation position is:

[0050] x i =x0+R·cos(θ)·u

[0051] y i =y0+R·sin(θ)·v

[0052] z i =z0+H·w

[0053] Among them, R is the expansion radius, which indicates the size of the flame combustion area and is generated uniformly and randomly by flame particles. (x i ,y i ,z i ) is the generated position coordinate of the i-th flame particle, (x0, y0, z0) represents the center position of the flame combustion area, θ is used to represent the angle parameter of the flame spreading direction, H is the vertical height of the flame, u, v and w are random numbers used to simulate the random distribution of flame particles in the flame combustion area;

[0054] The transparency of the flame particles changes over time to simulate the blurring effect of the flame edge and the dynamic changes of the combustion process. The formula is:

[0055]

[0056] Among them, α i (t) represents the transparency of the i-th flame particle at time t, represents the initial transparency of the i-th flame particle, and T represents the life cycle of the flame;

[0057] The color change of the flame is obtained by analyzing the RGB value of the flame frame by frame, and the formula is:

[0058]

[0059] Among them, Color i (t) represents the color of the i-th flame particle at time t, represents the initial color of the i-th flame particle, represents the color of the i-th flame particle after it burns out, and T represents the life cycle of the flame particle;

[0060] The formula for adjusting the speed of particles based on wind speed is:

[0061]

[0062] Among them, v i (t) represents the velocity of the i-th flame particle at time t, represents the initial velocity of the ith flame particle under windless conditions, α represents the wind speed influence coefficient, represents the wind speed at time t;

[0063] Step 4.2: Calculate the Euclidean distance between the flame particles and the target vehicle in the simulated flame spread model. If the distance is less than a given threshold, it is considered that any flame particle has touched the target object, and then go to step 4.1 to generate a new flame spread simulation. Otherwise, do not simulate. The distance between the flame particles and the target vehicle is calculated using the distance formula in three-dimensional space:

[0064]

[0065] Among them, d i represents the distance between the ith flame particle and each target vehicle, (x f ,y f ,z f ) is the position of the ith flame particle in Unity's three-dimensional space, (x c ,y c ,z c ) is the center position of each target vehicle. When the distance d i Less than the predetermined threshold R trigger When , a new flame spread model is generated at the location of the corresponding target vehicle;

[0066] Step 4.3: Based on all flame spread models of the tunnel fire scene disaster image and the tunnel fire scene twin model, the flame spread of the tunnel fire scene is simulated and dynamically updated.

[0067] A video data driven tunnel fire detection and simulation system, comprising:

[0068] Target data set construction module: preprocess the acquired original data set to obtain the target data set;

[0069] Target detection module: The target data set is used to train the improved YOLOv8 neural network model, and target detection is performed on the tunnel fire scene disaster image to be detected to obtain the center coordinates of each preliminary disaster object target;

[0070] Twin modeling module: Based on the central coordinates of each preliminary disaster object, the corresponding preliminary disaster object is corrected. After correction, the corrected disaster object is calibrated and distinguished using a monocular vision positioning method. The distinguished disaster object is then used to perform real-time scene twin modeling in the Unity engine, thus obtaining a tunnel fire scene twin model and dynamically updating it.

[0071] Flame spread module: Based on the tunnel fire scene twin model, the flame spread analysis of the tunnel fire scene is carried out and dynamic updates are realized in Unity.

[0072] Furthermore, the specific implementation steps of the target data set construction module are:

[0073] Step 1.1, obtaining an original data set, and using a scene change detection method to extract video frames to obtain an original video frame image data set, wherein the original data set includes tunnel fire monitoring video data and tunnel fire video data simulated in unity3d, and each image in the video frame image data set contains a vehicle and a flame;

[0074] Step 1.2: Perform preliminary denoising and image enhancement processing on various tunnel fire scene images in the original video frame image data set, select images that exceed a given value by using a threshold comparison method, perform diversity processing on the selected images, and transfer the unselected images to the denoising and image enhancement processing steps again until the requirements are met. Add 10% of negative samples to the data set after diversity processing, and perform reference learning by calculating the contrast loss function of positive and negative samples, where exceeding a given value means simultaneously exceeding a given resolution, clarity, contrast, detail richness, and recognizability of target features. Diversity processing includes brightness enhancement, contrast enhancement, rotation at any angle, and compound scaling processing;

[0075] Step 1.3, use LabelImg data annotation software to annotate each frame of the image after reference learning obtained in step 1.2, and store the annotation information together with the corresponding tunnel fire monitoring video data after reference learning and the image frames extracted from the tunnel fire video data simulated in unity3d to form multimodal data in the database, thus constructing a tunnel fire disaster object dataset, that is, obtaining the target dataset.

[0076] Furthermore, the improved YOLOv8 neural network model in the target detection module is based on the YOLOv8 neural network model, which introduces the Effici-entNetB0 network, the multi-semantic space attention mechanism module, and the feature extraction module integrating the Effici-entNetB0 network and the multi-semantic space attention mechanism module as the backbone network;

[0077] The feature extraction module is represented as:

[0078] F final (x) = SMSA(F(x))

[0079] Among them, F final (x) represents the feature output by the feature extraction module, F(x) represents the feature extraction output of the EfficientNetB0 network, and SMSA(F(x)) represents the feature map of F(x) after being processed by the channel attention mechanism;

[0080] The specific implementation steps of the target detection module are:

[0081] The improved YOLOv8 neural network model is trained using the target data set. After training, target detection is performed on the tunnel fire scene disaster image to be detected to obtain the detection results of each preliminary disaster object target, and the center coordinate Coordinate of each preliminary disaster object target is calculated based on the detection results. The detection results include the upper left corner coordinates (u1, v1) and lower right corner coordinates (u2, v2) of each preliminary disaster object target. The preliminary disaster object targets include target vehicles and flames. The calculation formula of the center coordinate Coordinate of each preliminary disaster object target is:

[0082] Coordinate=((u1+u2) / 2,(v1+v2) / 2);

[0083] The specific steps of the twin model modeling module are:

[0084] Step 3.1, using the linear regression model and the central coordinates of each preliminary disaster object target to correct each preliminary disaster object target, to obtain a corrected disaster object target;

[0085] Step 3.2, based on the monocular vision positioning method, the spatial coordinates are solved in combination with the projection matrix and pixel coordinates of the corrected disaster target. The spatial coordinates obtained after the solution are the positioning of the corrected disaster target. Then, based on the geometric information perception and attribute feature information extraction of each located disaster target, each located disaster target is distinguished to obtain a distinction result, wherein the geometric perception information includes geometric dimensions, spatial range and spatial position relationship, the geometric dimensions include the size and shape of the flame, the spatial position relationship includes the relative position relationship between the flame and each target vehicle, the attribute feature information includes the type and texture of the target vehicle, the state and timing characteristics of the flame, and the distinction result includes distinguishing the target vehicle into a small passenger car, a large truck and a large passenger car;

[0086] The geometric size L of the disaster target after positioning is:

[0087]

[0088] Where W is the actual width of the tunnel, W b and D are respectively the width of the bounding box of each located disaster object and the distance from the center of each located disaster object to the camera;

[0089] The spatial range boundingbox of the located disaster target is:

[0090]

[0091] Among them, (x b ,y b) represents the coordinates of the center point of the disaster target after positioning, z1 and z2 are the ranges of the disaster target in the depth direction after positioning, representing the minimum depth and the maximum depth respectively;

[0092] Step 3.3: Based on the geometric information perception results and attribute feature information of each differentiated disaster object target, the twin modeling parameters of the disaster object target are determined by the "perception result + knowledge model driven" method. Then, based on the twin modeling parameters, the tunnel fire scene entity objects in the tunnel fire scene disaster image are mapped and instantiated in the unity engine using the spatial topological semantic constraints to obtain the tunnel fire scene twin model. The specific steps are as follows:

[0093] Based on the geometric perception results, attribute feature information and multi-dimensional and multi-scale scene image information from local and global aspects of each disaster object target after differentiation, the graph database is used to take each differentiated disaster object target and its attribute feature information as nodes, and the geometric perception results as relationships as edges, to perform data modeling and relationship mapping. After mapping, the geometric structure of the tunnel, the fire location and the geometric perception results and attribute feature information of each differentiated disaster object target are integrated through the spatial data fusion method to form a dynamic three-dimensional knowledge graph, so as to obtain the twin modeling parameters of each differentiated disaster object target, including spatial coordinates, temperature coefficient and flame burning rate. Among them, the multi-dimensional and multi-scale scene image information includes flame image information in the early, middle and late stages of the fire, and tunnel fire scene image information of different heights, widths and depths;

[0094] Based on the twin modeling parameters obtained, the spatial topological semantic constraints are used to map and instantiate the different disaster objects in the tunnel fire scene disaster image in the Unity engine to establish a twin model of the tunnel fire scene. The spatial topological semantic constraints include adjacency, inclusion, object type constraints, and environmental constraints.

[0095] Step 3.4: Based on the real-time acquired tunnel fire scene disaster image, preliminary disaster object detection, correction, positioning and differentiation are re-performed to update the initial twin modeling of the tunnel fire scene.

[0096] Furthermore, the specific implementation steps of the flame spread module are:

[0097] Step 4.1, extracting the spread characteristics of the twin model of the tunnel fire scene from the tunnel fire scene disaster image, constructing one or more flame spread models in Unity3D, wherein the flame spread model includes the change of flame over time, the growth of flame height and the change of flame spread shape, and the change of flame spread shape includes the generation position of flame particles, the change of transparency of flame particles over time, the color change and the speed adjustment of wind speed particles;

[0098] Based on the linear expansion characteristics of the combustion area, the formula for the change of flame over time is:

[0099] R f (t) = R0 + v f ·t

[0100] Among them, R f (t) represents the change of the flame at time t, R0 is the initial radius of the flame, v f represents the flame spread speed, which is extracted from the twin model of the tunnel fire scene;

[0101] The flame height growth formula is:

[0102] H f (t)=H0+v s ·t

[0103] Among them, H f (t) represents the growth of flame height at time t, H0 is the initial height of the flame, v s is the flame expansion speed in the vertical direction;

[0104] The flame spreading shape change is simulated by adjusting the generation position, transparency, color change of flame particles and the speed of wind particles;

[0105] The formula for the flame particle generation position is:

[0106] x i =x0+R·cos(θ)·u

[0107] y i =y0+R·sin(θ)·v

[0108] z i =z0+H·w

[0109] Among them, R is the expansion radius, which indicates the size of the flame combustion area and is generated uniformly and randomly by flame particles. (x i ,y i ,z i ) is the generated position coordinate of the i-th flame particle, (x0, y0, z0) represents the center position of the flame combustion area, θ is used to represent the angle parameter of the flame spreading direction, H is the vertical height of the flame, u, v and w are random numbers used to simulate the random distribution of flame particles in the flame combustion area;

[0110] The transparency of the flame particles changes over time to simulate the blurring effect of the flame edge and the dynamic changes of the combustion process. The formula is:

[0111]

[0112] Among them, α i (t) represents the transparency of the i-th flame particle at time t, represents the initial transparency of the i-th flame particle, and T represents the life cycle of the flame;

[0113] The color change of the flame is obtained by analyzing the RGB value of the flame frame by frame, and the formula is:

[0114]

[0115] Among them, Color i (t) represents the color of the i-th flame particle at time t, represents the initial color of the i-th flame particle, represents the color of the i-th flame particle after it burns out, and T represents the life cycle of the flame particle;

[0116] The formula for adjusting the speed of particles based on wind speed is:

[0117]

[0118] Among them, v i (t) represents the velocity of the i-th flame particle at time t, represents the initial velocity of the ith flame particle under windless conditions, α represents the wind speed influence coefficient, represents the wind speed at time t;

[0119] Step 4.2: Calculate the Euclidean distance between the flame particles and the target vehicle in the simulated flame spread model. If the distance is less than a given threshold, it is considered that any flame particle has touched the target object, and then go to step 4.1 to generate a new flame spread simulation. Otherwise, do not simulate. The distance between the flame particles and the target vehicle is calculated using the distance formula in three-dimensional space:

[0120]

[0121] Among them, d i represents the distance between the ith flame particle and each target vehicle, (x f ,y f ,z f ) is the position of the ith flame particle in Unity's three-dimensional space, (x c ,y c ,z c ) is the center position of each target vehicle. When the distance d i Less than the predetermined threshold R trigger When , a new flame spread model is generated at the location of the corresponding target vehicle;

[0122] Step 4.3: Based on all flame spread models of the tunnel fire scene disaster image and the tunnel fire scene twin model, the flame spread of the tunnel fire scene is simulated and dynamically updated.

[0123] Compared with the prior art, the advantages of the present invention are:

[0124] First, the present invention establishes a tunnel fire disaster object data set based on monitoring data and simulation data. By establishing a tunnel fire scene disaster object data set based on monitoring data (tunnel fire monitoring video data) and simulation data, effective detection and identification of fire scenes are achieved. First, by extracting frames from tunnel monitoring videos, rich original image data is obtained, and combined with Unity simulation and Internet image data, diversified fire images are generated, which greatly improves the diversity and representativeness of the data set. Subsequently, denoising, enhancement and background mixing processing are performed for different fire scenes to ensure that the image quality in the data set is high and the target effect is significant. In addition, by adding negative samples for reference learning, the false detection rate is effectively reduced and the accuracy of fire detection is improved. Finally, the images are systematically labeled using LabelImg software, and the labeled information is combined with the image frames and stored in the database to form a complete and structured fire disaster object data set. This process not only improves the efficiency of collecting and processing tunnel fire scene data, but also lays a solid data foundation for the subsequent automated fire detection and alarm system.

[0125] Second, the present invention proposes to improve the YOLOv8 neural network model. By introducing the EfficientNetB0 network, it can achieve efficient and lightweight feature extraction for the features of early targets such as flames, significantly improving the detection performance of the model in complex scenes. The application of the SMSA attention mechanism (multi-semantic spatial attention mechanism module) enhances the sensitivity of the model to the characteristics of disaster objects, enabling it to more accurately identify the characteristics of flames, smoke and target vehicles, thereby improving the accuracy and timeliness of fire monitoring. At the same time, the feature extraction module effectively integrates the EfficientNetB0 network and the SMSA attention mechanism, making the model more adaptable when extracting complex scene information. By fine-tuning the target data set and introducing a variety of hyperparameter settings under the YOLOv8 architecture, the model achieves rapid convergence and effective learning, solving the problem of insufficient recognition of small targets and complex backgrounds in previous target detection. Overall, the technology has significant innovations, not only improving the target detection accuracy and robustness of tunnel fire scenes, but also providing more accurate technical support for rapid response and handling of fire accidents, which is helpful to improve the intelligence level of tunnel safety monitoring systems.

[0126] 3. The present invention uses a monocular vision positioning method to perform rapid twin modeling of scenes, which significantly improves the real-time and accuracy of fire disaster monitoring. That is, the improved YOLOv8 neural network model is used to detect the real-time video stream of the surveillance camera, and the target center coordinates are corrected by a linear regression model, and the spatial coordinates are solved by combining the projection matrix and the pixel coordinates, thereby ensuring the subsequent accurate perception of the geometric information, spatial range and type of the disaster object. After correction, the monocular vision positioning method can effectively identify and locate the disaster object target in the two-dimensional tunnel fire scene disaster image to be detected, and realize the reverse association between the two-dimensional object and the three-dimensional geographic entity, thereby It solves the deviation problem between two-dimensional images and three-dimensional spatial data in traditional methods; in addition, the technology extracts multi-modal and multi-scale information, such as flame size and vehicle type, and adopts the "perception results + knowledge model driven" method to obtain modeling parameters, thereby optimizing the twin modeling process of tunnel fire scenes. Finally, through the automatic processing of change perception and modeling parameters, the system can realize dynamic update and real-time adjustment in the initial twin model of the tunnel fire scene initially constructed, thereby improving the intelligence and response speed of the monitoring system. Overall, the technology has the advantages of high efficiency, real-time and precision, and innovatively combines visual recognition and spatial modeling to promote the development of disaster monitoring.

[0127] 4. The Unity3D tunnel fire spread simulation based on video perception of the present invention realizes efficient and accurate simulation of tunnel fire through advanced physical models, flame particles and flame modeling. Compared with traditional fire simulation methods, this technology has the advantages of improving simulation accuracy, real-time feedback of dynamic fire behavior, and enhanced flexibility and scalability. It is suitable for tunnel fire spread research and emergency plan formulation. That is, the simulation introduces real physical environment parameters, dynamically adjusts the spread speed and direction of the flame, and ensures the high realism of the flame spread process. The flame spread model is updated in real time through the overall flame and flame particles. The change of flame over time, the growth of flame height and the change of flame spread shape are simulated according to the actual fire spread characteristics, so that the flame spread trajectory conforms to the real physical laws. At the same time, through physical The engine acts on the overall flame and flame particles, causing the flame spread trajectory to change. The size and speed of the overall flame and flame particles are dynamically adjusted over time, accurately simulating the burning behavior of the fire at different stages. It also introduces a collision detection model based on Euclidean distance. When flame particles come into contact with objects in the tunnel, the system automatically triggers the generation of new flame particles to simulate the secondary spread of the flame. This modeling method supports flexible adjustment of the length, width, and internal environmental characteristics of the tunnel, allowing the system to adapt to different types of tunnel scenes. By introducing different physical scene elements, such as obstacles, vehicles, ventilation equipment, etc., researchers can quickly generate a variety of fire scenarios and evaluate the spread path and spread speed of the flame under different conditions. This flexibility provides solid technical support for the formulation of fire emergency plans. BRIEF DESCRIPTION OF THE DRAWINGS

[0128] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0129] Figure 1 It is the technical roadmap of the present invention;

[0130] Figure 2 Schematic diagram of the structure of the improved YOLOv8 neural network model in the present invention, wherein EfficientnetB0: EfficientnetB0 network, SMSA represents a multi-semantic spatial attention mechanism module, C2f (Context to Feature Module) represents context feature extraction and feature fusion, Conca represents a connection module, Upsample represents an upsampling module, Conv represents a convolution kernel, and Head represents an output module;

[0131] Figure 3 Schematic diagram of the structure of the multi-semantic spatial attention mechanism module in the present invention, wherein AvgPool represents average pooling, MS-DWConvld represents depth-separable convolution, Group Norm-4 represents group normalization, Concat represents connection module, and Sigmoid represents Sigmoid activation function;

[0132] Figure 4 It is a schematic diagram of monocular vision positioning in the present invention. DETAILED DESCRIPTION

[0133] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0134] Firstly, the original data set is formed by collecting tunnel fire monitoring video data and tunnel fire video data simulated in unity3d. The image processing software is used to extract video frames, perform denoising and image enhancement, and finally annotate them to construct the target data set. Then, the EfficientNetB0 network, multi-semantic spatial attention mechanism (SMSA) and feature extraction module are introduced as the backbone network of the YOLOv8 neural network model to obtain an improved YOLOv8 neural network model to realize the detection of targets (including target vehicles and flames, etc.) in the disaster image of the tunnel fire scene to be detected. Considering the use cost and distribution of tunnel monitoring cameras, the target center of the preliminary disaster object detected is corrected, and the position of the tunnel fire scene objects is calibrated and distinguished by monocular vision method to complete the real-time scene twin modeling in the unity engine. Finally, the flame spread analysis of the existing tunnel fire scene is carried out and dynamic update is realized in unity.

[0135] This paper focuses on the research of accurate perception modeling of disaster objects and smoke-fire spread simulation prediction and analysis in tunnel fire scenes. In terms of accurate perception modeling of disaster objects, the improved YOLOv8 neural network model is introduced to study the acquisition of accurate perception information of disaster objects, and then the monocular vision positioning principle is used to study the method of rapid modeling of disaster objects; in terms of smoke-fire spread simulation prediction and analysis, the flame spread algorithm is designed and combined with its physical characteristics to realize the simulation and dynamic update of tunnel fire smoke spread.

[0136] The light environment of tunnel fire scenes is complex and there are many interference factors, which makes the recognition accuracy of tunnel fire disaster objects low, and the three-dimensional information that video images can express is limited, and the information perception ability of the disaster is weak. In addition, the current scene modeling methods are mainly based on manual modeling and parametric modeling. Such methods mainly focus on the refined display of static scenes, while the tunnel fire environment is dynamic and uncertain. The semantic knowledge and object space association involved are complex, and automatic modeling is difficult, which makes it difficult to meet the needs of rapid modeling of tunnel fire environments.

[0137] In order to solve the problems of insufficient computing power, delay, low precision and high cost of traditional perception technology in abnormal disaster scenarios of tunnel fire, this paper develops a precise perception technology for tunnel fire disaster objects driven by video data which can be deployed on low-cost monitoring equipment. The technology extracts the significant features of disaster objects in video data, and realizes rapid twin modeling of disaster objects from two-dimensional data to three-dimensional engine space. On this basis, a simulation analysis of the spread of flame and smoke in tunnel fire scenarios is designed. The results can directly serve the information management of highway tunnel operation process and promote the development of tunnel intelligent fire protection theory and methods. It has broad application value and great practical significance.

[0138] The present invention provides a video data driven tunnel fire detection and simulation method, the specific steps are:

[0139] The obtained original data set is preprocessed to obtain the target data set. The specific steps are as follows:

[0140] Obtain an original data set, and use a scene change detection method to extract video frames to obtain an original video frame image data set, wherein the original data set includes tunnel fire monitoring video data and tunnel fire video data simulated in unity3d, and each image in the video frame image data set contains a vehicle and a flame;

[0141] The various tunnel fire scene images in the original video frame image data set are preliminarily denoised and enhanced. The images exceeding the given value are selected by the threshold comparison method, and the selected images are subjected to diversity processing. The unselected images are transferred to the denoising and image enhancement processing steps again until the requirements are met. 10% of negative samples are added to the obtained diversity processed data set, and reference learning is performed by calculating the contrast loss function of positive and negative samples. Exceeding the given value means exceeding the given resolution, clarity, contrast, detail richness and recognizability of the target features at the same time. Diversity processing includes brightness enhancement, contrast enhancement, rotation of any angle and compound scaling processing;

[0142] LabelImg data labeling software is used to label each frame of the image after reference learning. The labeling information and the corresponding image frames extracted from the tunnel fire monitoring video data after reference learning and the tunnel fire video data simulated in unity3d are stored in the database to form multimodal data. The tunnel fire disaster object dataset, that is, the target dataset, is constructed.

[0143] Considering that the early targets of fire are small, EfficientNetB0 is introduced as the backbone network to improve the model's efficient feature extraction capabilities and achieve lightweight processing. Then, a lightweight multi-semantic spatial attention mechanism (SMSA) module is introduced to fully utilize the collaborative potential of multi-semantic information for feature guidance and alleviate semantic differences. The weight of the feature is dynamically adjusted according to the context of the input image, so that the network can see global and local information at the same time, thereby enhancing the inductive bias. Enhance the model's sensitivity to the characteristics of disaster objects and improve detection performance. Subsequently, a new feature extraction module is constructed, integrating the EfficientNetB0 network and the SMSA module, and incorporating it into the backbone network, so that the model can better extract smoke-fire and vehicle features in complex scenes. During the training process, based on the YOLOv8 architecture, the modified feature extraction module was used to fine-tune the network. According to actual needs and tasks, the parameters of the Ef-ficientNetB0 network, such as the number of channels and convolution kernel size, were adjusted to adapt to the new data sets and tasks. The loss function and optimizer of YOLOv8 were modified to match the new Efficien-tNetB0 network. Finally, in order to improve the robustness of the model in complex scenarios, different hyperparameter settings, such as learning rate, batch size, and number of iterations, were introduced to ensure that the model can converge quickly during training and effectively learn the flame and vehicle features, thereby realizing a refined tunnel fire scene object perception method.

[0144] Specifically:

[0145] The target data set is used to train the improved YOLOv8 neural network model, and target detection is performed on the tunnel fire scene disaster image to be detected to obtain the center coordinates of each preliminary disaster object. The specific steps are as follows:

[0146] The improved YOLOv8 neural network model is trained using the target data set. After training, target detection is performed on the tunnel fire scene disaster image to be detected to obtain the detection results of each preliminary disaster object target, and the center coordinate Coordinate of each preliminary disaster object target is calculated based on the detection results. The detection results include the upper left corner coordinates (u1, v1) and lower right corner coordinates (u2, v2) of each preliminary disaster object target. The preliminary disaster object targets include target vehicles and flames. The calculation formula of the center coordinate Coordinate of each preliminary disaster object target is:

[0147] Coordinate=((u1+u2) / 2,(v1+v2) / 2).

[0148] Among them, the improved YOLOv8 neural network model is based on the YOLOv8 neural network model, which introduces the Effici-entNetB0 network, the multi-semantic spatial attention mechanism module, and the feature extraction module integrating the Effici-entNetB0 network and the multi-semantic spatial attention mechanism module as the backbone network;

[0149] The feature extraction module is represented as:

[0150] F final (x) = SMSA(F(x))

[0151] Among them, F final (x) represents the feature output by the feature extraction module, F(x) represents the feature extraction output of the EfficientNetB0 network, SMSA(F(x)) represents the feature map of F(x) after being processed by the channel attention mechanism, and x represents the tunnel fire scene disaster image to be detected. The twin model of the tunnel fire scene takes the real-time video stream of the monitoring camera as input, and constructs spatial topological semantic associations based on the two-dimensional tunnel fire disaster objects in the video data (i.e., the geometric information perception results and attribute feature information of the disaster object targets after differentiation), and realizes the reverse association from the two-dimensional object to the three-dimensional geographic entity. Since there is a partial deviation between the target center obtained by directly obtaining the rectangular box from the two-dimensional image and the target center of the three-dimensional world, firstly, the linear regression model is used to correct the target rectangular box center coordinates obtained by the preliminary target detection. Secondly, based on the principle of monocular vision positioning, the spatial coordinates are solved in combination with the projection matrix and the pixel coordinates to complete the positioning of the disaster object target. After positioning, the geometric information of the disaster object target is accurately perceived and the attribute feature information is extracted to distinguish the disaster object target after positioning.

[0152] Next, based on the acquisition of multi-modal and multi-scale information of disaster objects in real-time video streams, such as the size and timing characteristics of flames, the type and texture of vehicles, etc., the relevant parameters of twin modeling of disaster objects are determined through the "perception results + knowledge model driven" method, and the entity objects of the tunnel fire scene are mapped and instantiated using spatial topological semantic constraints to establish a twin model of the tunnel scene; finally, combined with the real-time perception results of dynamic and static disaster objects in video images, change perception is performed, and by constructing a series of rules and parameters, program-automated three-dimensional scene modeling is realized in the initially constructed tunnel model.

[0153] Specifically:

[0154] Based on the central coordinates of each preliminary disaster target, the corresponding preliminary disaster target is corrected. After correction, the monocular vision positioning method is used to calibrate and distinguish the corrected disaster target. Then, the distinguished disaster target is used to perform real-time scene twin modeling in the Unity engine, that is, the tunnel fire scene twin model is obtained and dynamically updated. The specific steps are as follows:

[0155] Using the linear regression model and the central coordinates of each preliminary disaster situation object target, each preliminary disaster situation object target is corrected to obtain a corrected disaster situation object target;

[0156] Based on the monocular vision positioning method, the spatial coordinates are solved in combination with the projection matrix and pixel coordinates of the corrected disaster object target. The spatial coordinates obtained after the solution are the positioning of the corrected disaster object target. Then, based on the geometric information perception and attribute feature information extraction of each located disaster object target, each located disaster object target is distinguished to obtain a distinction result, wherein the geometric perception information includes geometric dimensions, spatial range and spatial position relationship, the geometric dimensions include the size and shape of the flame, the spatial position relationship includes the relative position relationship between the flame and each target vehicle, the attribute feature information includes the type and texture of the target vehicle, the state and timing characteristics of the flame, and the distinction result includes distinguishing the target vehicles into small passenger cars, large trucks and large passenger cars;

[0157] The geometric size L of the disaster target after positioning is:

[0158]

[0159] Where W is the actual width of the tunnel, W b and D are respectively the width of the bounding box of each located disaster object and the distance from the center of each located disaster object to the camera;

[0160] The spatial range boundingbox of the located disaster target is:

[0161]

[0162] Among them, (x b ,y b ) represents the coordinates of the center point of the disaster target after positioning, z1 and z2 are the ranges of the disaster target in the depth direction after positioning, representing the minimum depth and the maximum depth respectively;

[0163] Based on the geometric information perception results and attribute feature information of each differentiated disaster object target, the twin modeling parameters of the disaster object target are determined through the "perception result + knowledge model driven" method. Then, based on the twin modeling parameters, the tunnel fire scene entity object mapping and instantiation in the tunnel fire scene disaster image are carried out in the unity engine using the spatial topological semantic constraints, that is, the tunnel fire scene twin model is obtained. The specific steps are as follows:

[0164] Based on the geometric perception results, attribute feature information and multi-dimensional and multi-scale scene image information from local and global aspects of each disaster object target after differentiation, the graph database is used to take each differentiated disaster object target and its attribute feature information as nodes, and the geometric perception results as relationships as edges, to perform data modeling and relationship mapping. After mapping, the geometric structure of the tunnel, the fire location and the geometric perception results and attribute feature information of each differentiated disaster object target are integrated through the spatial data fusion method to form a dynamic three-dimensional knowledge graph, so as to obtain the twin modeling parameters of each differentiated disaster object target, including spatial coordinates, temperature coefficient and flame burning rate. Among them, the multi-dimensional and multi-scale scene image information includes flame image information in the early, middle and late stages of the fire, and tunnel fire scene image information of different heights, widths and depths;

[0165] Based on the obtained twin modeling parameters, the spatial topological semantic constraints are used to map and instantiate the targets of the disaster objects distinguished in the tunnel fire scene disaster image in the Unity engine to establish a twin model of the tunnel fire scene. The spatial topological semantic constraints include adjacency relationships, inclusion relationships, object type constraints and environmental constraints.

[0166] According to the real-time acquired tunnel fire scene disaster images, preliminary disaster object detection, correction, positioning and differentiation are re-performed to update the initial twin modeling of the tunnel fire scene.

[0167] Traditional tunnel fire spread simulation methods are mostly based on predefined models and historical data, lacking real-time and dynamic adjustment capabilities, and are difficult to accurately reflect the complex behavior and influencing factors of fire. This paper proposes a tunnel fire spread simulation method based on video perception and Unity3D, which uses flame characteristic parameters extracted from video to dynamically simulate the fire spread process of the entire flame and flame particles.

[0168] Specifically:

[0169] Based on the twin model of the tunnel fire scene, the flame spread analysis of the tunnel fire scene is carried out and dynamically updated in Unity. The specific steps are as follows:

[0170] Extract the spread characteristics of the twin model of the tunnel fire scene from the tunnel fire scene disaster image. Construct one or more flame spread models in Unity3D, where the flame spread model includes the change of flame over time, the growth of flame height and the change of flame spread shape. The change of flame spread shape includes the generation position of flame particles, the change of transparency of flame particles over time, the color change and the speed adjustment of wind speed particles.

[0171] Based on the linear expansion characteristics of the combustion area, the formula for the change of flame over time is:

[0172] R f (t) = R0 + v f ·t

[0173] Among them, R f (t) represents the change of the flame at time t, R0 is the initial radius of the flame, v f represents the flame spread speed, which is extracted from the twin model of the tunnel fire scene;

[0174] The flame height growth formula is:

[0175] H f (t)=H0+v s ·t

[0176] Among them, H f (t) represents the growth of flame height at time t, H0 is the initial height of the flame, v s is the flame expansion speed in the vertical direction;

[0177] The flame spreading shape change is simulated by adjusting the generation position, transparency, color change of flame particles and the speed of wind particles;

[0178] The formula for the flame particle generation position is:

[0179] x i =x0+R·cos(θ)·u

[0180] y i =y0+R·sin(θ)·v

[0181] z i =z0+H·w

[0182] Among them, R is the expansion radius, which indicates the size of the flame combustion area and is generated uniformly and randomly by flame particles. (x i ,y i ,z i) is the generated position coordinate of the i-th flame particle, (x0, y0, z0) represents the center position of the flame combustion area, θ is used to represent the angle parameter of the flame spreading direction, H is the vertical height of the flame, u, v and w are random numbers used to simulate the random distribution of flame particles in the flame combustion area;

[0183] The transparency of the flame particles changes over time to simulate the blurring effect of the flame edge and the dynamic changes of the combustion process. The formula is:

[0184]

[0185] Among them, α i (t) represents the transparency of the i-th flame particle at time t, represents the initial transparency of the i-th flame particle, and T represents the life cycle of the flame;

[0186] The color change of the flame is obtained by analyzing the RGB value of the flame frame by frame, and the formula is:

[0187]

[0188] Among them, Color i (t) represents the color of the i-th flame particle at time t, represents the initial color of the i-th flame particle, represents the color of the i-th flame particle after it burns out, and T represents the life cycle of the flame particle;

[0189] The formula for adjusting the speed of particles based on wind speed is:

[0190]

[0191] Among them, v i (t) represents the velocity of the i-th flame particle at time t, represents the initial velocity of the ith flame particle under windless conditions, α represents the wind speed influence coefficient, represents the wind speed at time t;

[0192] The Euclidean distance between the flame particles and the target vehicle in the simulated flame spread model is calculated. If the distance is less than a given threshold, it is considered that any flame particle has touched the target object, and then go to step 4.1 to generate a new flame spread simulation. Otherwise, no simulation is performed. The distance between the flame particles and the target vehicle is calculated using the distance formula in three-dimensional space:

[0193]

[0194] Among them, d i represents the distance between the ith flame particle and each target vehicle, (xf ,y f ,z f ) is the position of the ith flame particle in Unity's three-dimensional space, (x c ,y c ,z c ) is the center position of each target vehicle. When the distance d i Less than the predetermined threshold R trigger When , a new flame spread model is generated at the location of the corresponding target vehicle;

[0195] Based on all flame spread models of the tunnel fire scene disaster image and the tunnel fire scene twin model, the flame spread of the tunnel fire scene is simulated and dynamically updated.

Claims

1. A video data driven tunnel fire detection and simulation method, characterized in that: The steps include: Step 1: preprocess the acquired original data set to obtain the target data set; Step 2: Use the target data set to train the improved YOLOv8 neural network model, and perform target detection on the tunnel fire scene disaster image to be detected to obtain the center coordinates of each preliminary disaster object target; Step 3: Based on the central coordinates of each preliminary disaster target, the corresponding preliminary disaster target is corrected. After correction, the corrected disaster target is calibrated and distinguished using a monocular vision positioning method. The distinguished disaster target is then used to perform real-time scene twin modeling in the Unity engine, thus obtaining a tunnel fire scene twin model and dynamically updating it. Step 4: Perform flame spread analysis on the tunnel fire scene based on the tunnel fire scene twin model and implement dynamic updates in Unity.

2. A video data driven tunnel fire detection and simulation method according to claim 1, characterized in that: The specific steps of step 1 are: Step 1.1, obtaining an original data set, and using a scene change detection method to extract video frames to obtain an original video frame image data set, wherein the original data set includes tunnel fire monitoring video data and tunnel fire video data simulated in unity3d, and each image in the video frame image data set contains a vehicle and a flame; Step 1.2: Perform preliminary denoising and image enhancement processing on various tunnel fire scene images in the original video frame image data set, select images that exceed a given value by using a threshold comparison method, perform diversity processing on the selected images, and transfer the unselected images to the denoising and image enhancement processing steps again until the requirements are met. Add 10% of negative samples to the data set after diversity processing, and perform reference learning by calculating the contrast loss function of positive and negative samples, where exceeding a given value means simultaneously exceeding a given resolution, clarity, contrast, detail richness, and recognizability of target features. Diversity processing includes brightness enhancement, contrast enhancement, rotation at any angle, and compound scaling processing; Step 1.3, use Label Img data annotation software to annotate each frame of the image after reference learning obtained in step 1.2, and store the annotation information together with the corresponding tunnel fire monitoring video data after reference learning and the image frames extracted from the tunnel fire video data simulated in unity3d to form multimodal data in the database, thus constructing a tunnel fire disaster object dataset, that is, obtaining the target dataset.

3. A video data driven tunnel fire detection and simulation method according to claim 2, characterized in that: The improved YOLOv8 neural network model in step 2 is based on the YOLOv8 neural network model, introducing the Effici-entNetB0 network, the multi-semantic space attention mechanism module, and the feature extraction module integrating the Effici-entNetB0 network and the multi-semantic space attention mechanism module as the backbone network; The feature extraction module is represented as: F final (x)=SMSA(F(x)) Among them, F final (x) represents the feature output by the feature extraction module, F(x) represents the feature extraction output of the EfficientNetB0 network, and SMSA(F(x)) represents the feature map of F(x) after being processed by the channel attention mechanism.

4. A video data driven tunnel fire detection and simulation method according to claim 3, characterized in that: The specific steps of step 2 are: The improved YOLOv8 neural network model is trained using the target data set. After training, target detection is performed on the tunnel fire scene disaster image to be detected to obtain the detection results of each preliminary disaster object target, and the center coordinate Coordinate of each preliminary disaster object target is calculated based on the detection results. The detection results include the upper left corner coordinates (u1, v1) and lower right corner coordinates (u2, v2) of each preliminary disaster object target. The preliminary disaster object targets include target vehicles and flames. The calculation formula of the center coordinate Coordinate of each preliminary disaster object target is: Coordinate=((u1+u2) / 2,(v1+v2) / 2).

5. A video data driven tunnel fire detection and simulation method according to claim 4, characterized in that: The specific steps of step 3 are: Step 3.1, using the linear regression model and the central coordinates of each preliminary disaster object target to correct each preliminary disaster object target, to obtain a corrected disaster object target; Step 3.2, based on the monocular vision positioning method, the spatial coordinates are solved in combination with the projection matrix and pixel coordinates of the corrected disaster target. The spatial coordinates obtained after the solution are the positioning of the corrected disaster target. Then, based on the geometric information perception and attribute feature information extraction of each located disaster target, each located disaster target is distinguished to obtain a distinction result, wherein the geometric perception information includes geometric dimensions, spatial range and spatial position relationship, the geometric dimensions include the size and shape of the flame, the spatial position relationship includes the relative position relationship between the flame and each target vehicle, the attribute feature information includes the type and texture of the target vehicle, the state and timing characteristics of the flame, and the distinction result includes distinguishing the target vehicle into a small passenger car, a large truck and a large passenger car; The geometric size L of the disaster target after positioning is: Where W is the actual width of the tunnel, W b and D are respectively the width of the bounding box of each located disaster object and the distance from the center of each located disaster object to the camera; The spatial range boundingbox of the located disaster target is: Among them, (x b ,y b ) represents the coordinates of the center point of the disaster target after positioning, z1 and z2 are the ranges of the disaster target in the depth direction after positioning, representing the minimum depth and the maximum depth respectively; Step 3.3: Based on the geometric information perception results and attribute feature information of each differentiated disaster object target, the twin modeling parameters of the disaster object target are determined by the "perception result + knowledge model driven" method. Then, based on the twin modeling parameters, the tunnel fire scene entity objects in the tunnel fire scene disaster image are mapped and instantiated in the unity engine using the spatial topological semantic constraints to obtain the tunnel fire scene twin model. The specific steps are as follows: Based on the geometric perception results, attribute feature information and multi-dimensional and multi-scale scene image information from local and global aspects of each disaster object target after differentiation, the graph database is used to take each differentiated disaster object target and its attribute feature information as nodes, and the geometric perception results as relationships as edges, to perform data modeling and relationship mapping. After mapping, the geometric structure of the tunnel, the fire location and the geometric perception results and attribute feature information of each differentiated disaster object target are integrated through the spatial data fusion method to form a dynamic three-dimensional knowledge graph, so as to obtain the twin modeling parameters of each differentiated disaster object target, including spatial coordinates, temperature coefficient and flame burning rate. Among them, the multi-dimensional and multi-scale scene image information includes flame image information in the early, middle and late stages of the fire, and tunnel fire scene image information of different heights, widths and depths; Based on the twin modeling parameters obtained, the spatial topological semantic constraints are used to map and instantiate the different disaster objects in the tunnel fire scene disaster image in the Unity engine to establish a twin model of the tunnel fire scene. The spatial topological semantic constraints include adjacency, inclusion, object type constraints, and environmental constraints. Step 3.4: Based on the real-time acquired tunnel fire scene disaster image, preliminary disaster object detection, correction, positioning and differentiation are re-performed to update the initial twin modeling of the tunnel fire scene.

6. A video data driven tunnel fire detection and simulation method according to claim 5, characterized in that: The specific steps of step 4 are: Step 4.1, extracting the spread characteristics of the twin model of the tunnel fire scene from the tunnel fire scene disaster image, constructing one or more flame spread models in Unity3D, wherein the flame spread model includes the change of flame over time, the growth of flame height and the change of flame spread shape, and the change of flame spread shape includes the generation position of flame particles, the change of transparency of flame particles over time, the color change and the speed adjustment of wind speed particles; Based on the linear expansion characteristics of the combustion area, the formula for the change of flame over time is: R f (t)=R0+v f ·t Among them, R f (t) represents the change of the flame at time t, R0 is the initial radius of the flame, v f represents the flame spread speed, which is extracted from the twin model of the tunnel fire scene; The flame height growth formula is: H f (t)=H0+v s ·t Among them, H f (t) represents the growth of flame height at time t, H0 is the initial height of the flame, v s is the flame expansion speed in the vertical direction; The flame spreading shape change is simulated by adjusting the generation position, transparency, color change of flame particles and the speed of wind particles; The formula for the flame particle generation position is: x i =x0+R·cos(θ)·u and i =y0+R·sin(θ)·v With i =z0+H w Among them, R is the expansion radius, which indicates the size of the flame combustion area and is generated uniformly and randomly by flame particles. (x i ,y i ,z i ) is the generated position coordinate of the i-th flame particle, (x0, y0, z0) represents the center position of the flame combustion area, θ is used to represent the angle parameter of the flame spreading direction, H is the vertical height of the flame, u, v and w are random numbers used to simulate the random distribution of flame particles in the flame combustion area; The transparency of the flame particles changes over time to simulate the blurring effect of the flame edge and the dynamic changes of the combustion process. The formula is: Among them, α i (t) represents the transparency of the i-th flame particle at time t, represents the initial transparency of the i-th flame particle, and T represents the life cycle of the flame; The color change of the flame is obtained by analyzing the RGB value of the flame frame by frame, and the formula is: Among them, Color i (t) represents the color of the i-th flame particle at time t, represents the initial color of the i-th flame particle, represents the color of the i-th flame particle after it burns out, and T represents the life cycle of the flame particle; The formula for adjusting the speed of particles based on wind speed is: Among them, v i (t) represents the velocity of the i-th flame particle at time t, represents the initial velocity of the ith flame particle under windless conditions, α represents the wind speed influence coefficient, represents the wind speed at time t; Step 4.2: Calculate the Euclidean distance between the flame particles and the target vehicle in the simulated flame spread model. If the distance is less than a given threshold, it is considered that any flame particle has touched the target object, and then go to step 4.1 to generate a new flame spread simulation. Otherwise, do not simulate. The distance between the flame particles and the target vehicle is calculated using the distance formula in three-dimensional space: Among them, d i represents the distance between the ith flame particle and each target vehicle, (x f ,y f ,z f ) is the position of the ith flame particle in Unity's three-dimensional space, (x c ,y c ,z c ) is the center position of each target vehicle. When the distance d i Less than the predetermined threshold R trigger When , a new flame spread model is generated at the location of the corresponding target vehicle; Step 4.3: Based on all flame spread models of the tunnel fire scene disaster image and the tunnel fire scene twin model, the flame spread of the tunnel fire scene is simulated and dynamically updated.

7. A video data driven tunnel fire detection and simulation system, characterized in that: include: Target data set construction module: preprocess the acquired original data set to obtain the target data set; Target detection module: The target data set is used to train the improved YOLOv8 neural network model, and target detection is performed on the tunnel fire scene disaster image to be detected to obtain the center coordinates of each preliminary disaster object target; Twin modeling module: Based on the central coordinates of each preliminary disaster object, the corresponding preliminary disaster object is corrected. After correction, the corrected disaster object is calibrated and distinguished using a monocular vision positioning method. The distinguished disaster object is then used to perform real-time scene twin modeling in the Unity engine, thus obtaining a tunnel fire scene twin model and dynamically updating it. Flame spread module: Based on the tunnel fire scene twin model, the flame spread analysis of the tunnel fire scene is carried out and dynamic updates are realized in Unity.

8. A video data driven tunnel fire detection and simulation system according to claim 7, characterized in that: The specific implementation steps of the target data set construction module are: Step 1.1, obtaining an original data set, and using a scene change detection method to extract video frames to obtain an original video frame image data set, wherein the original data set includes tunnel fire monitoring video data and tunnel fire video data simulated in unity3d, and each image in the video frame image data set contains a vehicle and a flame; Step 1.2: Perform preliminary denoising and image enhancement processing on various tunnel fire scene images in the original video frame image data set, select images that exceed a given value by using a threshold comparison method, perform diversity processing on the selected images, and transfer the unselected images to the denoising and image enhancement processing steps again until the requirements are met. Add 10% of negative samples to the data set after diversity processing, and perform reference learning by calculating the contrast loss function of positive and negative samples, where exceeding a given value means simultaneously exceeding a given resolution, clarity, contrast, detail richness, and recognizability of target features. Diversity processing includes brightness enhancement, contrast enhancement, rotation at any angle, and compound scaling processing; Step 1.3, use Label Img data annotation software to annotate each frame of the image after reference learning obtained in step 1.2, and store the annotation information together with the corresponding tunnel fire monitoring video data after reference learning and the image frames extracted from the tunnel fire video data simulated in unity3d to form multimodal data in the database, thus constructing a tunnel fire disaster object dataset, that is, obtaining the target dataset.

9. A video data driven tunnel fire detection and simulation system according to claim 8, characterized in that: The improved YOLOv8 neural network model in the target detection module is based on the YOLOv8 neural network model, which introduces the Effici-entNetB0 network, the multi-semantic space attention mechanism module, and the feature extraction module integrating the Effici-entNetB0 network and the multi-semantic space attention mechanism module as the backbone network; The feature extraction module is represented as: F final (x)=SMSA(F(x)) Among them, F final (x) represents the feature output by the feature extraction module, F(x) represents the feature extraction output of the EfficientNetB0 network, and SMSA(F(x)) represents the feature map of F(x) after being processed by the channel attention mechanism; The specific implementation steps of the target detection module are: The improved YOLOv8 neural network model is trained using the target data set. After training, target detection is performed on the tunnel fire scene disaster image to be detected to obtain the detection results of each preliminary disaster object target, and the center coordinate Coordinate of each preliminary disaster object target is calculated based on the detection results. The detection results include the upper left corner coordinates (u1, v1) and lower right corner coordinates (u2, v2) of each preliminary disaster object target. The preliminary disaster object targets include target vehicles and flames. The calculation formula of the center coordinate Coordinate of each preliminary disaster object target is: Coordinate=((u1+u2) / 2,(v1+v2) / 2); The specific steps of the twin model modeling module are: Step 3.1, using the linear regression model and the central coordinates of each preliminary disaster object target to correct each preliminary disaster object target, to obtain a corrected disaster object target; Step 3.2, based on the monocular vision positioning method, the spatial coordinates are solved in combination with the projection matrix and pixel coordinates of the corrected disaster target. The spatial coordinates obtained after the solution are the positioning of the corrected disaster target. Then, based on the geometric information perception and attribute feature information extraction of each located disaster target, each located disaster target is distinguished to obtain a distinction result, wherein the geometric perception information includes geometric dimensions, spatial range and spatial position relationship, the geometric dimensions include the size and shape of the flame, the spatial position relationship includes the relative position relationship between the flame and each target vehicle, the attribute feature information includes the type and texture of the target vehicle, the state and timing characteristics of the flame, and the distinction result includes distinguishing the target vehicle into a small passenger car, a large truck and a large passenger car; The geometric size L of the disaster target after positioning is: Where W is the actual width of the tunnel, W b and D are respectively the width of the bounding box of each located disaster object and the distance from the center of each located disaster object to the camera; The spatial range boundingbox of the located disaster target is: Among them, (x b ,y b ) represents the coordinates of the center point of the disaster target after positioning, z1 and z2 are the ranges of the disaster target in the depth direction after positioning, representing the minimum depth and the maximum depth respectively; Step 3.3: Based on the geometric information perception results and attribute feature information of each differentiated disaster object target, the twin modeling parameters of the disaster object target are determined by the "perception result + knowledge model driven" method. Then, based on the twin modeling parameters, the tunnel fire scene entity objects in the tunnel fire scene disaster image are mapped and instantiated in the unity engine using the spatial topological semantic constraints to obtain the tunnel fire scene twin model. The specific steps are as follows: Based on the geometric perception results, attribute feature information and multi-dimensional and multi-scale scene image information from local and global aspects of each disaster object target after differentiation, the graph database is used to take each differentiated disaster object target and its attribute feature information as nodes, and the geometric perception results as relationships as edges, to perform data modeling and relationship mapping. After mapping, the geometric structure of the tunnel, the fire location and the geometric perception results and attribute feature information of each differentiated disaster object target are integrated through the spatial data fusion method to form a dynamic three-dimensional knowledge graph, so as to obtain the twin modeling parameters of each differentiated disaster object target, including spatial coordinates, temperature coefficient and flame burning rate. Among them, the multi-dimensional and multi-scale scene image information includes flame image information in the early, middle and late stages of the fire, and tunnel fire scene image information of different heights, widths and depths; Based on the twin modeling parameters obtained, the spatial topological semantic constraints are used to map and instantiate the different disaster objects in the tunnel fire scene disaster image in the Unity engine to establish a twin model of the tunnel fire scene. The spatial topological semantic constraints include adjacency, inclusion, object type constraints, and environmental constraints. Step 3.4: Based on the real-time acquired tunnel fire scene disaster image, preliminary disaster object detection, correction, positioning and differentiation are re-performed to update the initial twin modeling of the tunnel fire scene.

10. A video data driven tunnel fire detection and simulation system according to claim 9, characterized in that: The specific implementation steps of the flame spread module are: Step 4.1, extracting the spread characteristics of the twin model of the tunnel fire scene from the tunnel fire scene disaster image, constructing one or more flame spread models in Unity3D, wherein the flame spread model includes the change of flame over time, the growth of flame height and the change of flame spread shape, and the change of flame spread shape includes the generation position of flame particles, the change of transparency of flame particles over time, the color change and the speed adjustment of wind speed particles; Based on the linear expansion characteristics of the combustion area, the formula for the change of flame over time is: R f (t)=R0+v f ·t Among them, R f (t) represents the change of the flame at time t, R0 is the initial radius of the flame, v f represents the flame spread speed, which is extracted from the twin model of the tunnel fire scene; The flame height growth formula is: H f (t)=H0+v s ·t Among them, H f (t) represents the growth of flame height at time t, H0 is the initial height of the flame, v s is the flame expansion speed in the vertical direction; The flame spreading shape change is simulated by adjusting the generation position, transparency, color change of flame particles and the speed of wind particles; The formula for the flame particle generation position is: x i =x0+R·cos(θ)·u and i =y0+R·sin(θ)·v With i =z0+H w Among them, R is the expansion radius, which indicates the size of the flame combustion area and is generated uniformly and randomly by flame particles. (x i ,y i ,z i ) is the generated position coordinate of the i-th flame particle, (x0, y0, z0) represents the center position of the flame combustion area, θ is used to represent the angle parameter of the flame spreading direction, H is the vertical height of the flame, u, v and w are random numbers used to simulate the random distribution of flame particles in the flame combustion area; The transparency of the flame particles changes over time to simulate the blurring effect of the flame edge and the dynamic changes of the combustion process. The formula is: Among them, α i (t) represents the transparency of the i-th flame particle at time t, represents the initial transparency of the i-th flame particle, and T represents the life cycle of the flame; The color change of the flame is obtained by analyzing the RGB value of the flame frame by frame, and the formula is: Among them, Color i (t) represents the color of the i-th flame particle at time t, represents the initial color of the i-th flame particle, represents the color of the i-th flame particle after it burns out, and T represents the life cycle of the flame particle; The formula for adjusting the speed of particles based on wind speed is: Among them, v i (t) represents the velocity of the i-th flame particle at time t, represents the initial velocity of the ith flame particle under windless conditions, α represents the wind speed influence coefficient, represents the wind speed at time t; Step 4.2: Calculate the Euclidean distance between the flame particles and the target vehicle in the simulated flame spread model. If the distance is less than a given threshold, it is considered that any flame particle has touched the target object, and then go to step 4.1 to generate a new flame spread simulation. Otherwise, do not simulate. The distance between the flame particles and the target vehicle is calculated using the distance formula in three-dimensional space: Among them, d i represents the distance between the ith flame particle and each target vehicle, (x f ,y f ,z f ) is the position of the ith flame particle in Unity's three-dimensional space, (x c ,y c ,z c ) is the center position of each target vehicle. When the distance d i Less than the predetermined threshold R trigger When , a new flame spread model is generated at the location of the corresponding target vehicle; Step 4.3: Based on all flame spread models of the tunnel fire scene disaster image and the tunnel fire scene twin model, the flame spread of the tunnel fire scene is simulated and dynamically updated.

Images