A tunnel new energy vehicle fire early warning method and system

Abstract

The present application relates to the field of tunnel fire prediction, in particular to a tunnel new energy vehicle fire warning method and system. The method comprises: using a YOLOv8 model to identify new energy vehicles entering the tunnel, and obtaining visible light images, thermal imaging images and millimeter wave radar point cloud data of the new energy vehicles; introducing a multi-target tracking algorithm to track the new energy vehicles, and calculating the battery temperature of each new energy vehicle in combination with tunnel environment data, and synchronously calibrating the collected data in time and space, and then calculating the motion trajectory of the new energy vehicles; time-synchronously calibrating the motion trajectory and the battery temperature, and calculating the battery temperature gradient of the new energy vehicles using the time-synchronously calibrated results; and predicting and warning tunnel fires according to the battery temperature gradient. The present application solves the problem of lack of tunnel new energy vehicle fire warning in the prior art, and improves the safety of tunnel operation.

Description

Technical Field

[0001] This invention relates to the field of tunnel fire prediction, specifically a method and system for early warning of fires involving new energy vehicles in tunnels. Background Technology

[0002] With the increasing popularity of new energy vehicles, their safety issues, especially battery fires, are receiving growing attention. New energy vehicles typically use lithium-ion batteries, which are prone to thermal runaway under conditions such as high temperatures, impacts, and overcharging, potentially leading to fires.

[0003] Among various scenarios, fires involving new energy vehicles in tunnels are particularly dangerous. The narrow, confined space of tunnels accelerates the spread of fire, increases smoke accumulation and diffusion, and makes rescue efforts more difficult. Once a new energy vehicle catches fire in a tunnel, the fire can quickly spread throughout the entire tunnel, causing traffic paralysis and even endangering lives. Currently, prevention and response measures for new energy vehicle fires mainly focus on improving battery materials, designing protective battery packs, and maintaining the vehicle. However, these measures cannot completely eliminate the risk of fire. Therefore, predicting and issuing early warnings for new energy vehicle fires in tunnels is of paramount importance.

[0004] In the current technology, there is relatively little systematic research on fires involving new energy vehicles within the confined space of tunnels, especially regarding battery combustion characteristics, fire plume characteristics, and the spread and control of toxic fumes. Therefore, developing an effective tunnel fire prediction system for new energy vehicles is not only of significant theoretical importance, but also of great practical value in ensuring tunnel traffic safety and the safety of people's lives and property. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method and system for early warning of fires involving new energy vehicles in tunnels, thus solving the problem of the lack of early warning for fires involving new energy vehicles in tunnels in existing technologies.

[0006] To achieve the above objectives, one aspect of the present invention provides a method for early warning of fires involving new energy vehicles in tunnels. The method includes: introducing a YOLOv8 model to identify new energy vehicles entering the tunnel and acquiring visible light images, thermal imaging images, and millimeter-wave radar point cloud data of the new energy vehicles; introducing a multi-target tracking algorithm to track the new energy vehicles based on the visible light images, thermal imaging images, and millimeter-wave radar point cloud data; calculating the battery temperature of each new energy vehicle based on the target tracking result using the visible light images, thermal imaging images, and pre-acquired tunnel environment data; performing spatiotemporal synchronization calibration on the visible light images, thermal imaging images, and millimeter-wave radar point cloud data based on the target tracking result, and obtaining the motion trajectory of each new energy vehicle according to the spatiotemporal synchronization calibration result; performing time synchronization calibration on the motion trajectory and the battery temperature, and calculating the battery temperature gradient of the new energy vehicle using the time synchronization calibration result; and predicting and issuing an alarm for tunnel fires based on the battery temperature gradient.

[0007] This invention, by introducing the mature YOLOv8 model, can accurately identify new energy vehicles entering tunnels and acquire visible light images, thermal imaging images, and millimeter-wave radar point cloud data of these vehicles, providing a rich data foundation for subsequent analysis. By integrating visible light images, thermal imaging images, and real-time environmental data, the battery temperature of new energy vehicles can be calculated more accurately, improving data reliability. The application of a multi-target tracking algorithm can accurately track new energy vehicles and obtain their movement trajectories, facilitating the understanding of the vehicle's dynamic situation within the tunnel. Calculating the battery temperature gradient based on the movement trajectory and battery temperature allows for in-depth analysis of battery state change trends. Predicting and issuing warnings for tunnel fires based on the battery temperature gradient enables early warning of potential fire risks within tunnels, effectively reducing the probability of fires and providing strong protection for the safety of personnel and vehicles within the tunnel, reducing loss of life and property caused by fires, and also contributing to improving the safety and stability of tunnel operations.

[0008] Optionally, calculating the battery temperature of each of the new energy vehicles using the visible light image, the thermal imaging image, and pre-acquired tunnel environment data includes: performing semantic segmentation on the visible light image to obtain a battery area mask; extracting feature points from the visible light image and the thermal imaging image respectively to obtain visible light feature points and thermal imaging feature points; matching the visible light feature points and the thermal imaging feature points, and constructing a homography matrix from the visible light feature points to the thermal imaging feature points based on the matching result; mapping the battery area mask onto the thermal imaging image based on the homography matrix to obtain a thermal imaging battery area; calculating the average temperature of the thermal imaging battery area to obtain the battery measurement temperature of the new energy vehicle; and correcting the battery measurement temperature using the tunnel environment data to obtain the battery temperature of each of the new energy vehicles.

[0009] This invention enables precise localization of the battery region through semantic segmentation of visible light images, providing a clear target area for subsequent analysis and eliminating the influence of heat generation in other areas of the new energy vehicle, thus improving the focus and accuracy of the analysis. Feature points are extracted from both the visible light and thermal imaging images, and a homography matrix is ​​constructed through matching. This achieves effective fusion and correlation of the two types of image information, allowing the battery region in the visible light image to be accurately mapped onto the thermal imaging image, thereby obtaining the thermally imaged battery region. The battery measurement temperature is obtained by calculating the average temperature of this region, providing fundamental data for battery temperature acquisition. Finally, real-time environmental data is used to correct the measured temperature, fully considering the impact of environmental factors on battery temperature, further improving the accuracy and reliability of battery temperature calculation.

[0010] Optionally, the battery temperature satisfies the following formula:

[0011]

[0012] in, The value of the battery temperature. The temperature value of the battery is measured. This is the ambient temperature correction factor. This represents the real-time ambient temperature inside the tunnel. For reference temperature value, The airflow velocity influencing factor, This represents the airflow velocity inside the tunnel. The relative humidity factor This refers to the relative humidity of the air inside the tunnel. For reference relative humidity, This is the compensation value for the impact of thermal radiation from the tunnel environment on battery temperature.

[0013] The battery temperature calculation formula of this invention incorporates multiple factors to correct the measured battery temperature. It includes the effects of ambient temperature, airflow speed, relative humidity, and thermal radiation, thus more accurately reflecting the actual battery temperature.

[0014] Optionally, performing spatiotemporal synchronous calibration on the visible light image, the thermal imaging image, and the millimeter-wave radar point cloud data, and obtaining the motion trajectory of each of the new energy vehicles based on the spatiotemporal synchronous calibration results includes: performing spatiotemporal synchronous calibration on the visible light image, the thermal imaging image, and the millimeter-wave radar point cloud data to generate multimodal spatiotemporal sequence data; generating historical motion trajectories based on the multimodal spatiotemporal sequence data; constructing an occlusion compensation model; using the occlusion compensation model to predict the vehicle's predicted trajectory and confidence level based on the historical motion trajectory; and stitching the predicted vehicle trajectory and the historical motion trajectory together based on the confidence level to obtain the motion trajectory of each of the new energy vehicles.

[0015] This invention unifies different data in time and space by performing spatiotemporal synchronization calibration on the collected data of new energy vehicles, which facilitates subsequent analysis. It generates historical motion trajectories based on the multimodal spatiotemporal sequence data, which improves the accuracy of motion state estimation. It also uses an occlusion compensation model to predict the driving trajectory during occlusion, which improves the accuracy and robustness of motion trajectory calculation for new energy vehicles.

[0016] Optionally, the construction of the occlusion compensation model includes: constructing a time series feature extraction module using a long short-term memory network; constructing a confidence calculation module based on the differences in the input sample data; constructing a loss function based on the time series feature extraction module and the confidence calculation module; and constructing an occlusion compensation model based on the time series feature extraction module, the confidence calculation module, and the loss function.

[0017] This invention utilizes a time-series feature extraction module built with a Long Short-Term Memory (LSTM) network to effectively capture the temporal dependencies of vehicle motion data and accurately extract key features, providing strong support for prediction. A confidence calculation module is constructed based on the differences in the input sample data, enabling a more reasonable assessment of the reliability of different sample data and making the model more adaptable to the data. The loss function built based on these two modules guides continuous model optimization. The occlusion compensation model constructed by these three modules can accurately predict the trajectory of a vehicle when it is occluded, based on features and confidence levels, enhancing the accuracy and robustness of multi-target tracking and improving the prediction accuracy of the occlusion compensation model.

[0018] Optionally, the motion trajectory of the new energy vehicle satisfies the following formula:

[0019]

[0020] in, For the movement trajectory of new energy vehicles, This represents the historical trajectory of new energy vehicles. For confidence level, The occlusion compensation model is used to predict the trajectory of new energy vehicles.

[0021] The motion trajectory formula for new energy vehicles in this invention determines the motion trajectory of new energy vehicles by flexibly integrating measured and predicted trajectories based on confidence levels. At high confidence levels, measured trajectories are used to ensure accuracy; at moderate confidence levels, a weighted fusion method is used to balance actual and predicted data; and at low confidence levels, predicted trajectories are used to avoid data gaps, thereby improving the reliability and adaptability of motion trajectory acquisition.

[0022] Optionally, the step of performing time synchronization calibration on the motion trajectory and the battery temperature, and calculating the battery temperature gradient of the new energy vehicle using the results of the time synchronization calibration, includes: performing time synchronization calibration on the motion trajectory and the battery temperature to obtain trajectory-temperature fusion data; extracting timestamp data and speed data from the trajectory-temperature fusion data, and calculating the peak acceleration of the new energy vehicle based on the timestamp data and the speed data; extracting temperature data from the trajectory-temperature fusion data, and calculating the measured battery temperature gradient of the new energy vehicle based on the timestamp data and the temperature data; calculating the aggressive driving level of the new energy vehicle using the peak acceleration to obtain a violent driving index; calculating the impact coefficient of violent driving on battery temperature based on the violent driving index to obtain a driving interference coefficient; and correcting the measured battery temperature gradient using the peak acceleration and the driving interference coefficient to obtain the battery temperature gradient of the new energy vehicle.

[0023] This invention aligns the timestamps of the motion trajectory and battery temperature through time synchronization calibration, directly linking the battery temperature and the motion trajectory. It calculates a violent driving index using peak acceleration, quantifying the driver's aggressiveness, and further calculates a driving interference coefficient, fully considering the impact of human driving factors on battery temperature. The violent driving index and driving interference coefficient are used to correct the measured battery temperature gradient, making the final battery temperature gradient more accurately reflect the true state of the vehicle battery, avoiding misjudgments caused by driving behavior, and improving the accuracy of battery temperature gradient calculation.

[0024] Optionally, the battery temperature gradient satisfies the following formula:

[0025] The battery temperature gradient satisfies the following formula:

[0026]

[0027] in, For the battery temperature gradient, To measure the battery temperature gradient, To correct the temperature weighting, The index is for violent driving. This represents the peak acceleration.

[0028] The battery temperature gradient formula of this invention integrates factors such as measured gradient, aggressive driving index, and peak acceleration, fully considering the impact of aggressive driving and environmental factors on battery temperature, thus improving the accuracy of calculating the battery temperature gradient.

[0029] Optionally, the method of using the battery temperature gradient to predict and alarm tunnel fires includes: setting a safety threshold, calculating the risk level of the tunnel fire based on the safety threshold and the battery temperature gradient; assessing the risk level and issuing an alarm based on the level.

[0030] This invention, by setting safety thresholds and calculating the risk level of tunnel fires accordingly, transforms the abstract data of battery temperature gradients into concrete, quantifiable risk indicators, facilitating a direct assessment of the likelihood of a fire. The risk level classification enables tiered risk management, with different levels corresponding to different degrees of danger. Alarms based on these levels allow managers and relevant personnel to quickly understand the risk situation and take timely measures, further enhancing the practical application capabilities of this invention.

[0031] Another aspect of the present invention provides a fire early warning system for new energy vehicles in tunnels, comprising: a processor, an input device, an output device, and a memory, wherein the processor, the input device, the output device, and the memory are interconnected, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is configured to invoke the program instructions to execute a fire early warning method for new energy vehicles in tunnels as described in any of the preceding aspects of the present invention.

[0032] The present invention provides a tunnel fire early warning system for new energy vehicles, which is compact in structure, stable in performance, highly integrated and simple in composition. It can stably execute the tunnel fire early warning method for new energy vehicles provided in the preceding aspect of the present invention, further improving the overall applicability and practical application capability of the present invention. Attached Figure Description

[0033] Figure 1 This is a flowchart of a method for early warning of fires involving new energy vehicles in tunnels, according to an embodiment of the present invention.

[0034] Figure 2 This is a schematic diagram of a fire early warning system for new energy vehicles in a tunnel, according to an embodiment of the present invention. Detailed Implementation

[0035] Specific embodiments of the present invention will now be described in detail. It should be noted that the embodiments described herein are for illustrative purposes only and are not intended to limit the invention. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that these specific details are not necessary to practice the invention. In other instances, well-known circuits, software, or methods have not been specifically described to avoid obscuring the invention.

[0036] Throughout this specification, references to "an embodiment," "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in connection with that embodiment or example is included in at least one embodiment of the invention. Therefore, the phrases "in an embodiment," "in an embodiment," "an example," or "an example" appearing in various places throughout the specification do not necessarily refer to the same embodiment or example. Furthermore, specific features, structures, or characteristics can be combined in one or more embodiments or examples in any suitable combination and / or sub-combination. Moreover, those skilled in the art will understand that the illustrations provided herein are for illustrative purposes and are not necessarily drawn to scale.

[0037] Please see Figure 1 ,like Figure 1 This is a flowchart of a method for early warning of fires involving new energy vehicles in tunnels according to an embodiment of the present invention. To address the lack of early warning systems for fires involving new energy vehicles in tunnels in existing technologies, this method includes the following steps:

[0038] Step S1: Introduce the YOLOv8 model, identify new energy vehicles entering the tunnel based on the YOLOv8 model, and acquire visible light images, thermal imaging images, and millimeter-wave radar point cloud data of the new energy vehicles.

[0039] In this embodiment, the YOLOv8 model, developed by Ultralytics, is an advanced and efficient target detection and image segmentation model. YOLOv8 itself is a pre-trained model with good recognition capabilities for many common targets. First, image data of new energy vehicles and other vehicles in a tunnel environment are collected, covering different lighting conditions, angles, and positions, with precise annotations of vehicle positions and categories. Next, the YOLOv8 model is fine-tuned using the labeled data. The convolutional kernel parameters of the backbone network, the feature fusion parameters of the neck network, and the anchor box parameters of the detection head can be adjusted appropriately to improve performance. During training, the model is evaluated using a validation set, and parameters are further optimized based on metrics such as accuracy and recall, as well as error analysis results. Finally, the trained model is deployed to the tunnel detection system to achieve accurate recognition of new energy vehicles.

[0040] For data acquisition, high-definition visible light cameras, thermal imaging equipment, and millimeter-wave radar need to be deployed at tunnel entrances and key locations. When a new energy vehicle enters the tunnel, the YOLOv8 model, with its powerful target detection capabilities, can quickly and accurately identify the vehicle and, based on the model's output information, precisely capture the vehicle's visible light image, thermal imaging image, and millimeter-wave radar point cloud data.

[0041] The acquired visible light images clearly display vehicle information such as body color, model, and license plate, aiding in vehicle identification. Thermal imaging images focus on the vehicle's temperature distribution, particularly showing the heating status of key components like the battery, motor, and tires. Millimeter-wave radar point cloud data provides precise information on the vehicle's position, speed, and attitude in three-dimensional space. Analysis of this data allows for a more accurate understanding of the vehicle's trajectory and motion. For example, when a vehicle is traveling in a tunnel, millimeter-wave radar point cloud data can provide real-time feedback on speed changes and distances to surrounding obstacles. Combined with visible light and thermal imaging images, this enables comprehensive, multi-angle monitoring of the vehicle.

[0042] Step S2: Introduce a multi-target tracking algorithm to track the new energy vehicle based on the visible light image, the thermal imaging image, and the millimeter-wave radar point cloud data.

[0043] In this embodiment, the multi-target tracking algorithm refers to simultaneously detecting and tracking multiple target objects (new energy vehicles) in a video sequence, while maintaining the continuity of the identity (ID) of each new energy vehicle between different frames.

[0044] The multi-target tracking algorithm tracks new energy vehicles based on visible light images, thermal images, and millimeter-wave radar point cloud data acquired by visible light cameras, thermal imaging cameras, and millimeter-wave radar installed inside the tunnel. First, a target detection model detects the vehicle's position and category from each frame of the image based on visual information such as the vehicle's appearance, outline, and color, generating detection boxes. Then, the multi-target tracking algorithm tracks the detected new energy vehicles, assigning a unique ID to each vehicle to create a unique identifier, thus distinguishing each vehicle from its visible light images, thermal images, and millimeter-wave radar point cloud data.

[0045] Step S3: Based on the target tracking results, calculate the battery temperature of each of the new energy vehicles using the visible light image, the thermal imaging image, and the pre-obtained tunnel environment data.

[0046] The calculation of the battery temperature of each of the new energy vehicles using the visible light image, the thermal imaging image, and the pre-obtained tunnel environment data specifically includes the following sub-steps:

[0047] Step S301: Perform semantic segmentation on the visible light image to obtain the battery area mask.

[0048] In this embodiment, based on the target tracking results, it is shown that the visible light images of each new energy vehicle have been correctly distinguished. The acquired visible light images of the new energy vehicles are input into a trained semantic segmentation model, such as U-Net. This model has been trained on a large number of vehicle image samples containing battery regions and can accurately learn the features of the battery regions. During model operation, it determines whether each pixel belongs to the battery region based on pixel features and context information. After segmentation, a binary image is generated as a battery region mask, where the pixel value of the battery region is 1 and the pixel value of other regions is 0.

[0049] Step S302: Extract feature points from the visible light image and the thermal imaging image respectively to obtain visible light feature points and thermal imaging feature points.

[0050] In this embodiment, a scale-invariant feature transform algorithm can be used to extract feature points from both visible light and thermal imaging images. This algorithm can effectively identify unique points in visible light images, such as corner points of vehicle outlines and obvious feature points of components, thereby obtaining visible light feature points. For thermal imaging images, since they reflect temperature distribution information, a feature point extraction method based on temperature gradient changes can be used. In areas where the temperature of heat-generating components such as batteries changes significantly, the extreme points of the temperature gradient can be detected as thermal imaging feature points. By extracting feature points from both types of images separately, important basic data is provided for subsequent image matching and analysis, which helps to more accurately correlate vehicle information in visible light and thermal imaging images.

[0051] Step S303: Match the visible light feature points and the thermal imaging feature points, and construct the homography matrix from the visible light feature points to the thermal imaging feature points based on the matching result.

[0052] In this embodiment, after extracting visible light feature points and thermal imaging feature points, the Fast Nearest Neighbor (FLANN) function library can be used to calculate the similarity metric between feature points based on their descriptors, and to match visible light and thermal imaging feature points with high similarity. After feature point matching is completed, the Random Sampling Consensus (RANSAC) algorithm is used. It selects a small number of feature point pairs through random sampling to estimate the homography matrix, and then uses this estimated matrix to test all other feature point pairs. Point pairs that conform to the matrix transformation relationship are considered inliers, and those that do not are considered outliers. Through multiple sampling and calculations, the estimated matrix with the most inliers is selected as the final homography matrix.

[0053] The homography matrix describes the projection transformation relationship of a point on a plane from one image to another. When the scene points corresponding to the two image planes are located on the same plane (the vehicle battery area is approximated as a plane), there is a two-dimensional projection transformation between the two images. This transformation can be represented by a 3×3 matrix, namely the homography matrix.

[0054] By constructing a homography matrix from visible light feature points to thermal imaging feature points, the battery area mask segmented in the visible light image can be accurately mapped onto the thermal imaging image. Since thermal imaging images primarily reflect temperature information, it is inherently difficult to directly and accurately locate the battery area. In contrast, visible light images provide clear vehicle appearance information, facilitating battery area segmentation. The homography matrix is ​​used to fuse the two types of image information, thereby accurately locating the battery area in the thermal imaging image.

[0055] Step S304: Map the battery area mask onto the thermal imaging image according to the homography matrix to obtain the thermal imaging battery area.

[0056] In this embodiment, for each pixel in the battery area mask, its coordinates are converted to homogeneous coordinates, and then the homography matrix is ​​used to perform transformation calculations on that point. The spatial transformation relationship implied by the homography matrix can accurately map the battery area location information on the visible light image to the corresponding location in the thermal imaging image.

[0057] It is important to note that during the mapping process, some boundary conditions may be encountered, such as the mapped point falling outside the boundary of the thermal imaging image. In such cases, appropriate handling is required, such as discarding the point or correcting it according to boundary rules. After mapping all pixels in the battery area mask, a new region is obtained on the thermal imaging image, namely the thermal imaging battery region.

[0058] Step S305: Calculate the average temperature of the thermal imaging battery area to obtain the battery measurement temperature of the new energy vehicle.

[0059] In this embodiment, each pixel in the thermal imaging image corresponds to a temperature value. Therefore, by traversing all pixels within the thermal imaging battery area, the temperature value of each pixel is obtained. Then, these temperature values ​​are summed and divided by the total number of pixels within the thermal imaging battery area to obtain the average temperature value of the area, i.e., the battery measurement temperature. This measurement temperature reflects the overall thermal state of the battery area, providing important basic data for subsequent evaluation of battery performance, prediction of battery-related risks, and early warning of tunnel fires.

[0060] Step S306: Correct the measured battery temperature using the tunnel environment data to obtain the battery temperature of each of the new energy vehicles.

[0061] In this embodiment, the actual operating temperature of a new energy vehicle battery may be affected by various factors, including ambient temperature, airflow speed, humidity, and thermal radiation. Directly measured battery temperature may not accurately reflect its true operating state; through correction, the actual battery temperature can be obtained more accurately.

[0062] The battery temperature satisfies the following formula:

[0063]

[0064] in, The value of the battery temperature. The temperature value of the battery is measured. This is the ambient temperature correction factor. This represents the real-time ambient temperature inside the tunnel. For reference temperature value, The airflow velocity influencing factor, This represents the airflow velocity inside the tunnel. The relative humidity factor This refers to the relative humidity of the air inside the tunnel. For reference relative humidity, This is the compensation value for the impact of thermal radiation from the tunnel environment on battery temperature.

[0065] The compensation value for the effect of tunnel environmental thermal radiation on battery temperature satisfies the following formula:

[0066]

[0067] in, This is the compensation value for the impact of tunnel environmental thermal radiation on battery temperature. The absorption coefficient of the battery surface to the thermal radiation from the tunnel is given. This represents the thermal radiation intensity value of the tunnel environment. This represents the effective area of ​​the battery for receiving thermal radiation. This represents the duration during which the battery is exposed to a specific intensity of thermal radiation inside the tunnel. This refers to the thermal capacity of a lithium battery.

[0068] In this embodiment, This indicates that when the real-time ambient temperature inside the tunnel is higher than the reference temperature, the battery has difficulty dissipating heat, and the battery temperature will rise. Conversely, when the real-time ambient temperature inside the tunnel is lower than the reference temperature, the battery dissipates heat more quickly, and the battery temperature will drop. The difference between the two values ​​is based on the reference temperature; the larger the difference, the greater the rise or fall in battery temperature. According to research, the reference temperature is generally taken as 25 degrees Celsius.

[0069] The principle behind the relative humidity of the air inside the tunnel is the same as that of the real-time ambient temperature inside the tunnel. The value is typically 50%.

[0070] Step S4: Based on the target tracking results, the visible light image, the thermal imaging image, and the millimeter-wave radar point cloud data are spatiotemporally synchronized and calibrated, and the motion trajectory of each new energy vehicle is obtained according to the spatiotemporal synchronization calibration results.

[0071] The process of performing spatiotemporal synchronization calibration on the visible light image, the thermal imaging image, and the millimeter-wave radar point cloud data, and obtaining the motion trajectory of each new energy vehicle based on the spatiotemporal synchronization calibration results, specifically includes the following sub-steps:

[0072] Step S401: Perform spatiotemporal synchronization calibration on the visible light image, the thermal imaging image, and the millimeter-wave radar point cloud data to generate multimodal spatiotemporal sequence data.

[0073] In this embodiment, the multimodal spatiotemporal sequence data includes the unique identifier of the new energy vehicle, the global location of the new energy vehicle, the global speed of the new energy vehicle, the timestamp of the new energy vehicle, and the raw data of each sensor of the new energy vehicle.

[0074] Based on the target tracking results, it is shown that the visible light image, thermal imaging image, and millimeter-wave radar point cloud data of each new energy vehicle have been distinguished. These data are then processed separately for each new energy vehicle. For geometric alignment, tools such as calibration boards are used to calibrate the intrinsic parameters (e.g., focal length, distortion coefficients) and extrinsic parameters (rotation matrix, translation vector) of the visible light and thermal imaging images. Simultaneously, by synchronizing the millimeter-wave radar point cloud data, transformation parameters from the radar coordinate system to the visible light image coordinate system are obtained, transforming the data from each sensor to the global coordinate system. For temporal alignment, since different sensors have different frequencies (e.g., millimeter-wave radar typically operates at 10-20Hz, while visible light cameras operate at 30-60Hz), software interpolation (e.g., linear interpolation) can be used to ensure consistent timestamps across all sensor data.

[0075] For calculating velocity in visible light images, techniques such as optical flow can be employed. Optical flow analyzes the pixel displacements of the target in adjacent frames, combining intrinsic and extrinsic parameters from both visible light and thermal imaging images to convert these pixel displacements into actual physical displacements, thereby calculating the target's velocity in the global coordinate system. For millimeter-wave radar, target velocity information can be directly obtained from its point cloud data. Thermal imaging images can assist visible light images in target identification and localization.

[0076] The calculated visible light velocity, radar velocity, and other data are then fused. A weighted average method can be used, dynamically adjusting the weights based on the confidence level of each sensor to obtain the global velocity. The global position can be obtained by converting the data from each sensor to a global coordinate system and then fusing the target position. For example, the centroid position can be calculated after clustering radar point cloud data, and the region of interest of the target can be extracted from image data and converted into global coordinates.

[0077] Finally, using a unified timestamp as an index, the vehicle's unique identifier, global location, global speed, timestamp, and raw data from each sensor (such as visible light image paths, radar point clouds, etc.) are integrated to form multimodal spatiotemporal sequence data.

[0078] Step S402: Generate historical motion trajectories based on the multimodal spatiotemporal sequence data.

[0079] In this embodiment, the time series of timestamps and global positions are first extracted from the multimodal spatiotemporal sequence data and arranged in chronological order to form the original trajectory. Sensor noise is eliminated by Kalman filtering or Savitzky-Golay filtering, and cubic spline interpolation is performed on missing points caused by high-frequency downsampling to ensure trajectory continuity. Then, leveraging the advantages of multi-sensor fusion, the velocity components calculated by millimeter-wave radar velocity measurement and visible light optical flow method are combined, and the velocity characteristics of the trajectory are optimized by weighted averaging. Finally, a historical motion trajectory including, but not limited to, timestamps, vehicle IDs, coordinates, velocity vectors, and timestamps is obtained.

[0080] Step S403: Construct an occlusion compensation model, and use the occlusion compensation model to predict the vehicle's predicted trajectory and confidence level based on the historical motion trajectory.

[0081] The construction of the occlusion compensation model includes:

[0082] Step S30501: Construct a time series feature extraction module using a long short-term memory network.

[0083] In this embodiment, a time-series feature extraction module based on a Long Short-Term Memory (LSTM) network is constructed. This module utilizes a multi-layer LSTM network structure and a gating mechanism to capture long-term dependencies in multimodal spatiotemporal sequence data. The input layer converts multi-dimensional data such as global position, velocity, and sensor features (e.g., radar intensity, image, coordinates) into embedding vectors. The hidden layer transmits time-series information through a chain structure, and the output layer extracts motion pattern features (e.g., acceleration, curvature). Combining bidirectional LSTM or attention mechanisms enhances the modeling of the correlation between historical trajectories and future predictions.

[0084] Step S30502: Construct a confidence calculation module based on the differences in the input sample data.

[0085] In this embodiment, the difference in the input sample data refers to the degree of difference in the descriptions of the same physical object or scene by different sensors or data sources in a multimodal data fusion task. Its core objective is to quantify the inconsistencies in spatiotemporal alignment, semantic representation, and other aspects of different modal data.

[0086] The reliability calculation module satisfies the following formula:

[0087]

[0088]

[0089] in, Due to data discrepancies, For the number of samples, The location of the target new energy vehicle in the visible light image of the target vehicle. The location of the target new energy vehicle in the millimeter-wave radar point cloud data of the target vehicle. For confidence level, and All parameters are adjustable.

[0090] Adjustable parameters Controlling confidence level for data variability The degree of sensitivity, The larger the value, the more significant the impact of Mc on confidence.

[0091] Adjustable parameters It serves to adjust the position of the confidence curve, and can be adjusted according to actual data and application scenarios to determine the appropriate level of adjustment. Below that, the confidence level begins to decline rapidly.

[0092] This formula maps the differences in data to a confidence level range of 0 to 1. The lower the confidence level, the lower the consistency and the less reliable the fusion result; conversely, the higher the confidence level, the more reliable the fusion result.

[0093] Step S30503: Construct a loss function based on the time series feature extraction module and the confidence calculation module.

[0094] In this embodiment, the work loss function satisfies the following formula:

[0095]

[0096] in, loss function These are the weighting coefficients. Let be the mean square error function. To predict the trajectory, For the actual trajectory, These are the weighting coefficients. Let cross-entropy be the loss function. To predict confidence levels, This represents the true confidence level.

[0097] Step S30504: Construct an occlusion compensation model based on the time series feature extraction module, the confidence calculation module, and the loss function.

[0098] In this embodiment, when constructing the occlusion compensation model, a time-series feature extraction module, a confidence calculation module, and a loss function are comprehensively utilized to optimize the model's output. The time-series feature extraction module is responsible for capturing the target's behavioral patterns and appearance changes from consecutive video frames; these features are then used to predict the target's future state. The confidence calculation module evaluates the reliability of these predictions, especially when the target is partially or completely occluded. It assigns a confidence score to each prediction, reflecting the model's degree of confidence in its prediction. The loss function measures the deviation between the model's predictions and actual observations, guiding parameter optimization during training to minimize this deviation. The model's output includes the prediction results and corresponding confidence scores. This not only provides the tracking algorithm with predictions of the target's future position and state but also provides information on the reliability of the predictions. This design allows the system to handle prediction results more intelligently when facing uncertainty, such as target occlusion, thereby further improving prediction accuracy.

[0099] Before training the occlusion compensation model, historical data of the vehicle under normal driving and occlusion scenarios were continuously collected using multimodal sensors (visible light cameras, thermal imaging equipment, and millimeter-wave radar) inside the tunnel. This included the vehicle's global position, speed, timestamps, and raw sensor data (such as radar point clouds and images). Multimodal spatiotemporal sequence data was generated through spatiotemporal synchronous calibration. During data processing, historical motion trajectories were filtered, denoised, and interpolated to ensure trajectory continuity. Occlusion scenario data was labeled to simulate different degrees of occlusion. Subsequently, the processed sample data was input into the model. The time series feature extraction module used an LSTM network to capture long-term dependencies in motion patterns, and the confidence calculation module quantified prediction reliability based on multimodal data differences (such as visible light and radar position deviations). The loss function was optimized by combining the mean square error (MSE) of the predicted trajectory and the cross-entropy (CE) of the confidence. Finally, the occlusion compensation model was iteratively trained using sample data, enabling it to accurately compensate for the vehicle's motion trajectory when occluded by fusing the measured and predicted trajectories based on confidence.

[0100] Step S404: Based on the confidence level, the predicted trajectory of the vehicle and the historical motion trajectory are spliced ​​together to obtain the motion trajectory of each of the new energy vehicles.

[0101] The motion trajectory of the new energy vehicle satisfies the following formula:

[0102]

[0103] in, For the movement trajectory of new energy vehicles, This represents the historical trajectory of new energy vehicles. For confidence level, The occlusion compensation model is used to predict the trajectory of new energy vehicles.

[0104] Step S5: Perform time synchronization calibration on the motion trajectory and the battery temperature, and calculate the battery temperature gradient of the new energy vehicle using the results of the time synchronization calibration.

[0105] The process of calibrating the motion trajectory and the battery temperature in real time, and then calculating the battery temperature gradient of the new energy vehicle using the results of the real-time calibration, specifically includes the following sub-steps:

[0106] Step S501: Time synchronization calibration is performed on the motion trajectory and the battery temperature to obtain trajectory-temperature fusion data.

[0107] In this embodiment, the sampling frequencies of different sensors may differ, leading to inconsistent timestamps. To address this, interpolation is used for fine synchronization. If the sampling interval for battery temperature data is 5 seconds, while the sampling interval for motion trajectory data is 1 second, linear interpolation can be used to estimate the battery temperature value at the corresponding motion trajectory time point based on known battery temperature data points.

[0108] Linear interpolation satisfies the following formula:

[0109]

[0110] in, For time The interpolated temperature, at any time The corresponding temperature is ,time The corresponding temperature is .

[0111] After interpolation, the time synchronization results are verified. The correlation between the motion trajectory and battery temperature data at adjacent time points is calculated. If the correlation is lower than a set threshold, an error is considered to exist in the synchronization. In this case, the error can be corrected by adjusting the parameters of the interpolation algorithm or by performing coarse alignment again. Simultaneously, a feedback mechanism is introduced to periodically calibrate the sensor clock to ensure the long-term stability of time synchronization, ultimately yielding the trajectory-temperature fusion data.

[0112] Step S502: Extract the timestamp data and speed data from the trajectory temperature fusion data, and calculate the peak acceleration of the new energy vehicle based on the timestamp data and the speed data.

[0113] In this embodiment, by using data parsing technology, timestamp data and speed data are accurately separated according to the format and structure of the trajectory temperature fusion data. Specifically, a specific code script can be written to traverse the trajectory temperature fusion data and store the timestamp and speed into different arrays or lists to ensure the integrity and accuracy of the data.

[0114] The extracted timestamp and velocity data are preprocessed. The data is checked for missing or outlier values. Missing values ​​are filled using interpolation methods, such as linear interpolation. Outliers are identified and removed using statistical methods, such as those based on standard deviation. The data is also sorted to ensure the timestamps are in ascending order, laying the foundation for subsequent calculations.

[0115] According to the definition of acceleration in physics, acceleration is the ratio of the change in velocity to the time taken for that change to occur. To calculate acceleration, the process iterates through all data points, calculating the acceleration for each time interval. Finally, the peak acceleration is identified by comparing all calculated acceleration values; this maximum value is the peak acceleration of the new energy vehicle.

[0116] When calculating peak acceleration, two time points are selected in chronological order. and The time interval is The corresponding speeds are respectively and .

[0117] Peak acceleration satisfies the following formula:

[0118]

[0119] in, Peak acceleration for Speed ​​vector of new energy vehicles at all times for Speed ​​vector of new energy vehicles at all times for and The time interval.

[0120] Step S503: Extract the temperature data from the trajectory temperature fusion data, and calculate the measured battery temperature gradient of the new energy vehicle based on the timestamp data and the temperature data.

[0121] In this embodiment, the extraction and processing of temperature data is consistent with the speed data extraction method described above. When calculating the battery temperature gradient, two time points are selected in chronological order. and The time interval is The corresponding battery temperatures are respectively and .

[0122] The measured battery temperature gradient satisfies the following formula;

[0123]

[0124] The temperature gradient between every two adjacent time points was calculated sequentially, and finally, the calculation results were compiled. All calculated temperature gradient data were then summarized to obtain the measured battery temperature gradient.

[0125] Step S504: Calculate the aggressive driving level of the new energy vehicle using the peak acceleration to obtain the violent driving index.

[0126] The violent driving index satisfies the following formula:

[0127]

[0128] in, The index is for violent driving. The number of acceleration samples within a predetermined time period. For the first One acceleration value, This is the acceleration threshold.

[0129] In this embodiment, the acceleration threshold is used to distinguish between normal driving and aggressive driving. Only when the acceleration exceeds this threshold is it considered to be an action with aggressive driving characteristics.

[0130] Step S505: Calculate the impact coefficient of violent driving on battery temperature based on the violent driving index to obtain the driving interference coefficient.

[0131] The driving interference coefficient satisfies the following formula:

[0132]

[0133] in, The driving interference coefficient, This is the index for violent driving.

[0134] In this formula, As a base offset, even if the violent driving index is 0, meaning there is no violent driving behavior, the driving interference coefficient still has a base value of 0.5. 0.5 represents the basic degree of influence of factors other than violent driving on battery temperature, and provides an initial benchmark for the driving interference coefficient.

[0135] This is the scaling factor, used to adjust... The degree of influence of the function output value on the driving interference coefficient. The function's range is -1 to 1, which is scaled to -0.3 to 0.3, limiting the range of how aggressive driving behavior affects battery temperature and preventing excessive fluctuations in the calculation of the impact.

[0136] middle, It serves as a balancing point, allowing the violent driving index to fluctuate around a baseline of 2, through... The function is used to reasonably adjust the driving interference coefficient.

[0137] Step S506: The measured battery temperature gradient is corrected using the peak acceleration and the driving interference coefficient to obtain the battery temperature gradient of the new energy vehicle.

[0138] The battery temperature gradient satisfies the following formula:

[0139]

[0140] because,

[0141] Therefore, the battery temperature gradient satisfies the following formula:

[0142]

[0143] in, For the battery temperature gradient, To measure the battery temperature gradient, To correct the temperature weighting, The index is for violent driving. This represents the peak acceleration.

[0144] In this embodiment, by introducing a corrected temperature weight, the temperature correction module can be flexibly adjusted. Through repeated experiments and verification, A value of 0.1 is suitable for most tunnel conditions. Combining this with the peak acceleration, it is known that the larger the peak acceleration, the more temperature correction is required.

[0145] Step S6: Predict and issue an alarm for tunnel fires based on the battery temperature gradient.

[0146] The prediction and warning of tunnel fires based on the battery temperature gradient specifically includes the following sub-steps:

[0147] Step S601: Set a safety threshold and calculate the risk level of tunnel fire based on the safety threshold and the battery temperature gradient.

[0148] In this embodiment, a safety threshold for the battery temperature gradient is first set. The safety threshold represents a safe critical value for the rate of change of battery temperature. The battery temperature gradient is compared with the safety threshold, and the risk level of tunnel fire is quantified by the relationship between the two values. When the battery temperature gradient exceeds the safety threshold, the larger the excess, the more obvious the abnormal change in battery temperature, and the higher the risk of tunnel fire.

[0149] The degree of risk satisfies the following formula:

[0150]

[0151] in, Based on the level of risk, As a safety threshold, Amplify the risk weight.

[0152] Step S602: Assess the risk level and issue an alarm based on the level.

[0153] In this embodiment, the level of risk The value ranges from 0 to 1. When it is low risk, When it is low risk, It is a high-risk time. There is no risk at that time.

[0154] Different alerts are issued based on the risk level assessment: no risk, low risk, medium risk, and high risk.

[0155] When the assessment level is no risk, it means that the tunnel is in a relatively safe state, and at least the tunnel safety accident will not be caused by the fire of a new energy vehicle, so there is no need to notify the tunnel safety officer.

[0156] When the risk level is assessed as low, the tunnel safety officer is notified that the tunnel is in a low-risk state. To prevent accidents, the safety officer is reminded to keep a close eye on the tunnel's condition.

[0157] When the risk level is assessed as medium, the tunnel safety officer is notified that the tunnel is in a medium-risk state. The safety officer should be prepared to deal with possible fire accidents at any time. The tunnel has a built-in safety broadcast system to broadcast to the relevant vehicles by reading their license plates, notifying the vehicles involved to check their battery temperature to prevent the situation from escalating further.

[0158] When the risk level is assessed as high, the tunnel safety officer should be notified that the tunnel is in a high-risk state. The safety officer should take appropriate accident rescue measures, such as coordinating nearby fire trucks and ambulances to be on standby, and restricting the flow of vehicles in the opposite direction of the tunnel.

[0159] like Figure 2As shown, another aspect of the present invention provides a fire early warning system for new energy vehicles in tunnels, comprising: a processor, an input device, an output device, and a memory, wherein the processor, the input device, the output device, and the memory are interconnected, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is configured to invoke the program instructions to execute relevant steps of a relevant embodiment of the fire early warning method for new energy vehicles in tunnels of the present invention.

[0160] This invention provides a fire early warning system for new energy vehicles in tunnels. The functional components can be integrated into a single processing unit, or each component can exist independently, or two or more components can be integrated into one unit. The integrated components can be implemented in hardware or software, further enhancing the overall applicability and practical application capabilities of this invention.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A method for early warning of fires involving new energy vehicles in tunnels, characterized in that, The method includes: The YOLOv8 model is introduced, and new energy vehicles entering the tunnel are identified based on the YOLOv8 model. Visible light images, thermal images and millimeter-wave radar point cloud data of the new energy vehicles are acquired. A multi-target tracking algorithm is introduced to track the new energy vehicle based on the visible light image, the thermal imaging image, and the millimeter-wave radar point cloud data. Based on the target tracking results, the battery temperature of each of the new energy vehicles is calculated using the visible light image, the thermal imaging image, and the pre-acquired tunnel environment data. Based on the target tracking results, the visible light image, the thermal imaging image, and the millimeter-wave radar point cloud data are spatiotemporally synchronized and calibrated, and the motion trajectory of each new energy vehicle is obtained according to the spatiotemporal synchronization calibration results. The motion trajectory and the battery temperature are time-synchronized and calibrated, and the battery temperature gradient of the new energy vehicle is calculated using the results of the time-synchronization calibration, including: The motion trajectory and the battery temperature are time-synchronized and calibrated to obtain trajectory-temperature fusion data; Extract the timestamp data and speed data from the trajectory temperature fusion data, and calculate the peak acceleration of the new energy vehicle based on the timestamp data and the speed data; Extract the temperature data from the trajectory temperature fusion data, and calculate the measured battery temperature gradient of the new energy vehicle based on the timestamp data and the temperature data; The aggressive driving level of the new energy vehicle is calculated using the peak acceleration to obtain a violent driving index; The impact coefficient of violent driving on battery temperature is calculated based on the violent driving index to obtain the driving interference coefficient. The measured battery temperature gradient of the new energy vehicle is obtained by correcting the measured battery temperature gradient using the peak acceleration and the driving disturbance coefficient. The battery temperature gradient satisfies the following formula: The battery temperature gradient satisfies the following formula: in, For the battery temperature gradient, To measure the battery temperature gradient, To correct the temperature weighting, The index is for violent driving. Peak acceleration; Tunnel fires are predicted and alerted based on the battery temperature gradient.

2. The method for early warning of fires involving new energy vehicles in tunnels according to claim 1, characterized in that, The calculation of the battery temperature of each of the new energy vehicles using the visible light image, the thermal imaging image, and pre-acquired tunnel environment data includes: Semantic segmentation is performed on the visible light image to obtain the battery area mask; Feature points are extracted from the visible light image and the thermal imaging image respectively to obtain visible light feature points and thermal imaging feature points; The visible light feature points and the thermal imaging feature points are matched, and a homography matrix from the visible light feature points to the thermal imaging feature points is constructed based on the matching results. The battery area mask is mapped onto the thermal imaging image based on the homography matrix to obtain the thermal imaging battery area; Calculate the average temperature of the thermal imaging battery area to obtain the battery measurement temperature of the new energy vehicle; The battery temperature measurement is corrected using the tunnel environment data to obtain the battery temperature of each of the new energy vehicles.

3. The method for early warning of fires involving new energy vehicles in tunnels according to claim 2, characterized in that, The battery temperature satisfies the following formula: in, The value of the battery temperature. The temperature value of the battery is measured. This is the ambient temperature correction factor. This represents the real-time ambient temperature inside the tunnel. For reference temperature value, The airflow velocity influencing factor, This represents the airflow velocity inside the tunnel. The relative humidity factor This refers to the relative humidity of the air inside the tunnel. For reference relative humidity, This is the compensation value for the impact of thermal radiation from the tunnel environment on battery temperature.

4. The method for early warning of fires involving new energy vehicles in tunnels according to claim 1, characterized in that, The process of performing spatiotemporal synchronization calibration on the visible light image, the thermal imaging image, and the millimeter-wave radar point cloud data, and obtaining the motion trajectory of each of the new energy vehicles based on the results of the spatiotemporal synchronization calibration, includes: The visible light image, the thermal imaging image, and the millimeter-wave radar point cloud data are spatiotemporally synchronized and calibrated to generate multimodal spatiotemporal sequence data; Historical motion trajectories are generated based on the multimodal spatiotemporal sequence data; An occlusion compensation model is constructed, and the vehicle's predicted trajectory and confidence level are obtained by using the occlusion compensation model based on the historical motion trajectory. The predicted trajectory of the vehicle and the historical trajectory are spliced ​​together based on the confidence level to obtain the trajectory of each of the new energy vehicles.

5. A method for early warning of fires involving new energy vehicles in tunnels according to claim 4, characterized in that, The construction of the occlusion compensation model includes: A time series feature extraction module is constructed using a long short-term memory network; A confidence calculation module is constructed based on the differences in the input sample data; A loss function is constructed based on the time series feature extraction module and the confidence calculation module; An occlusion compensation model is constructed based on the time series feature extraction module, the confidence calculation module, and the loss function.

6. A method for early warning of fires involving new energy vehicles in tunnels according to claim 4, characterized in that, The motion trajectory of the new energy vehicle satisfies the following formula: in, For the movement trajectory of new energy vehicles, This represents the historical trajectory of new energy vehicles. For confidence level, The occlusion compensation model is used to predict the trajectory of new energy vehicles.

7. A method for early warning of fires involving new energy vehicles in tunnels according to claim 1, characterized in that, The method of predicting and alerting tunnel fires based on the battery temperature gradient includes: Set a safety threshold and calculate the risk level of tunnel fire based on the safety threshold and the battery temperature gradient; The risk level is assessed and an alert is issued based on the level.

8. A fire early warning system for new energy vehicles in tunnels, characterized in that, include: The system includes a processor, an input device, an output device, and a memory, all interconnected. The memory stores a computer program, which includes program instructions. The processor is configured to invoke the program instructions to execute a method for early warning of fires involving new energy vehicles in tunnels, as described in any one of claims 1 to 7.

Images