Fire early warning method and system for new energy vehicle in tunnel

Abstract

The invention relates to the field of tunnel fire prediction, in particular to a new energy vehicle fire early warning method and system in a tunnel. The method comprises the following steps: identifying a new energy vehicle entering a tunnel by using a YOLOv8 model, and obtaining a visible light image, a thermal imaging image and millimeter wave radar point cloud data of the new energy vehicle; a multi-target tracking algorithm is introduced, target tracking is conducted on the new energy vehicles, the battery temperature of each new energy vehicle is calculated in combination with tunnel environment data, space-time synchronous calibration is conducted on collected data, and then the motion trails of the new energy vehicles are calculated; time synchronization calibration is conducted on the movement track and the battery temperature, and the battery temperature gradient of the new energy vehicle is calculated according to the result obtained after time synchronization calibration; and predicting and alarming the tunnel fire according to the battery temperature gradient. The problem that in the prior art, fire early warning for the new energy vehicle in the tunnel is lacked is solved, and tunnel operation safety is improved.

Description

Technical Field

[0001] The present invention relates to the field of tunnel fire prediction, and specifically to a method and system for early warning of new energy vehicle fires in tunnels. Background Art

[0002] With the popularization of new energy vehicles, their safety issues have attracted increasing attention, especially the problem of battery fires. The batteries of new energy vehicles usually use lithium-ion batteries, which are prone to thermal runaway under conditions such as high temperature, collision, and overcharging, and then cause fires.

[0003] Among many scenarios, new energy vehicle fires in tunnels are particularly dangerous. The long and narrow confined space structure of the tunnel will exacerbate the spread speed of the fire, increase the accumulation and diffusion of smoke, making rescue work more difficult. Once a new energy vehicle catches fire in the tunnel, the fire may quickly spread to the entire tunnel, causing traffic paralysis and even endangering the lives and safety of personnel. At present, the prevention and response measures for new energy vehicle fires mainly focus on the improvement of battery materials, the protection design of battery packs, and the maintenance of vehicles. However, these measures cannot completely eliminate the fire risk. Therefore, it is particularly important to predict and early warn new energy vehicle fires in tunnels.

[0004] In the prior art, in the confined space of the tunnel, the systematic research on new energy vehicle fires is relatively less, especially in aspects such as battery combustion characteristics, fire plume characteristics, and the spread and control of toxic smoke. Therefore, developing an effective tunnel new energy vehicle fire prediction system not only has important theoretical significance, but also has great practical value for ensuring tunnel traffic safety and the safety of personnel's lives and property. Summary of the Invention

[0005] Aiming at the defects in the prior art, the present invention provides a method and system for early warning of new energy vehicle fires in tunnels, which solves the problem of the lack of early warning of new energy vehicle fires in tunnels in the prior art.

[0006] To achieve the above object, in one aspect, the present invention provides a method for early warning of new energy vehicle fires in a tunnel, and the method includes: introducing the YOLOv8 model, identifying new energy vehicles entering the tunnel based on the YOLOv8 model, and obtaining visible light images, thermal imaging images and millimeter wave radar point cloud data of the new energy vehicles; introducing a multi-object tracking algorithm, and using the multi-object tracking algorithm to perform target tracking on the new energy vehicles based on the visible light images, the thermal imaging images and the millimeter wave radar point cloud data; based on the results of the target tracking, calculating the battery temperature of each new energy vehicle by using the visible light images, the thermal imaging images and the pre-obtained tunnel environment data; based on the results of the target tracking, performing spatio-temporal synchronization calibration on the visible light images, the thermal imaging images and the millimeter wave radar point cloud data, and obtaining the movement trajectory of each new energy vehicle according to the results of the spatio-temporal synchronization calibration; performing time synchronization calibration on the movement trajectory and the battery temperature, and calculating the battery temperature gradient of the new energy vehicle by using the results after the time synchronization calibration; predicting and alarming tunnel fires according to the battery temperature gradient.

[0007] By introducing the technically mature YOLOv8 model, the present invention can accurately identify new energy vehicles entering the tunnel, and obtain visible light images, thermal imaging images and millimeter wave radar point cloud data of new energy vehicles, providing a rich data basis for subsequent analysis. By integrating visible light images, thermal imaging images and real-time environmental data, the battery temperature of new energy vehicles can be calculated more accurately, improving data reliability. The application of the multi-object tracking algorithm can accurately track new energy vehicles and obtain their movement trajectories, facilitating the grasp of the dynamic situation of vehicles in the tunnel. Calculating the battery temperature gradient based on the movement trajectory and the battery temperature can deeply analyze the changing trend of the battery state. Predicting and alarming tunnel fires according to the battery temperature gradient realizes early warning of potential fire risks in the tunnel, can effectively reduce the probability of fire occurrence, provides strong guarantee for the safety of people and vehicles in the tunnel, reduces the life and property losses caused by fires, and also helps to improve the safety and stability of tunnel operation.

[0008] Optionally, calculating the battery temperature of each of the new energy vehicles by using the visible light image, the thermal imaging image, and the pre-acquired tunnel environment data includes: performing semantic segmentation on the visible light image to obtain a battery area mask; respectively extracting feature points from the visible light image and the thermal imaging image to obtain visible light feature points and thermal imaging feature points; matching the visible light feature points and the thermal imaging feature points, and constructing a homography matrix from the visible light feature points to the thermal imaging feature points according to the matching result; mapping the battery area mask to the thermal imaging image according to the homography matrix to obtain a thermal imaging battery area; calculating the average temperature of the thermal imaging battery area to obtain the measured battery temperature of the new energy vehicle; and correcting the measured battery temperature by using the tunnel environment data to obtain the battery temperature of each of the new energy vehicles.

[0009] Through semantic segmentation of the visible light image, the present invention can accurately locate the battery area, providing a clear target range for subsequent analysis, excluding the influence of heat generation in other areas of the new energy vehicle, and improving the pertinence and accuracy of the analysis. Feature points of the visible light image and the thermal imaging image are respectively extracted and matched to construct a homography matrix, realizing the effective fusion and association of the two image information, enabling the battery area in the visible light image to be accurately mapped onto the thermal imaging image, and further obtaining the thermal imaging battery area. By calculating the average temperature of this area, the measured battery temperature is obtained, providing basic data for obtaining the battery temperature. Finally, the measured temperature is corrected by using the real-time environment data, fully considering the influence of environmental factors on the battery temperature, and further improving the accuracy and reliability of the battery temperature calculation.

[0010] Optionally, the battery temperature satisfies the following formula: Wherein, is the value of the battery temperature, is the value of the measured battery temperature, is the environmental temperature correction coefficient, is the value of the real-time environmental temperature in the tunnel, is the reference temperature value, is the air flow velocity influence factor, is the value of the air flow velocity in the tunnel, is the relative humidity influence factor, is the relative humidity of the air in the tunnel, is the reference relative humidity, is the influence compensation value of the tunnel environment thermal radiation on the battery temperature.

[0011] The battery temperature calculation formula of the present invention comprehensively corrects the measured battery temperature with multiple factors. By incorporating the influence of ambient temperature, air flow velocity, relative humidity, and thermal radiation, it can more accurately reflect the actual battery temperature.

[0012] Optionally, performing spatio-temporal synchronization calibration on the visible light image, the thermal imaging image, and the millimeter-wave radar point cloud data, and obtaining the motion trajectory of each new energy vehicle according to the result of the spatio-temporal synchronization calibration includes: performing spatio-temporal synchronization calibration on the visible light image, the thermal imaging image, and the millimeter-wave radar point cloud data to generate multi-modal spatio-temporal sequence data; generating a historical motion trajectory according to the multi-modal spatio-temporal sequence data; constructing an occlusion compensation model, and predicting using the occlusion compensation model according to the historical motion trajectory to obtain a vehicle prediction trajectory and a confidence level; splicing the vehicle prediction trajectory and the historical motion trajectory according to the confidence level to obtain the motion trajectory of each new energy vehicle.

[0013] The present invention makes different data unified in time and space by performing spatio-temporal synchronization calibration on the collected data of new energy vehicles, facilitating subsequent analysis. Generating a historical motion trajectory according to the multi-modal spatio-temporal sequence data improves the accuracy of motion state estimation. Using the occlusion compensation model to predict the driving trajectory during occlusion improves the accuracy and robustness of the motion trajectory calculation of new energy vehicles.

[0014] Optionally, constructing the occlusion compensation model includes: using a long short-term memory network to construct a time series feature extraction module; constructing a confidence level calculation module according to the difference of the to-be-input sample data; constructing a loss function based on the time series feature extraction module and the confidence level calculation module; constructing an occlusion compensation model based on the time series feature extraction module, the confidence level calculation module, and the loss function.

[0015] The time series feature extraction module constructed by the present invention using a long short-term memory network can effectively capture the time-dependent relationship of vehicle motion data, accurately extract key features, and provide strong support for prediction. Constructing a confidence level calculation module according to the difference of the to-be-input sample data can more reasonably evaluate the reliability of different sample data, making the model more adaptable to data. The loss function constructed based on these two modules can guide the model to continuously optimize. The occlusion compensation model jointly constructed by the three can accurately predict the motion trajectory based on features and confidence levels when the vehicle is occluded, enhancing the accuracy and robustness of multi-target tracking and improving the accuracy of the occlusion compensation model prediction.

[0016] Optionally, the motion trajectory of the new energy vehicle satisfies the following formula: Wherein, is the movement trajectory of a new energy vehicle, is the historical movement trajectory of the new energy vehicle, is the confidence level, is the predicted trajectory of the new energy vehicle by the occlusion compensation model.

[0017] The movement trajectory formula of the new energy vehicle in the present invention flexibly combines the measured and predicted trajectories according to the confidence level to determine the movement trajectory of the new energy vehicle. When the confidence level is high, the measured trajectory is adopted to ensure accuracy. When the confidence level is moderate, weighted fusion is carried out to balance the actual and predicted situations. When the confidence level is low, the predicted trajectory is used to avoid data loss, improving the reliability and adaptability of obtaining the movement trajectory.

[0018] Optionally, the time synchronization calibration of the movement trajectory and the battery temperature, and calculating the battery temperature gradient of the new energy vehicle by using the result after the time synchronization calibration includes: performing time synchronization calibration on the movement trajectory and the battery temperature to obtain trajectory-temperature fusion data; extracting the timestamp data and speed data in the trajectory-temperature fusion data, and calculating the peak acceleration value of the new energy vehicle according to the timestamp data and the speed data; extracting the temperature data in the trajectory-temperature fusion data, and calculating the measured battery temperature gradient of the new energy vehicle according to the timestamp data and the temperature data; calculating the aggressive driving degree of the new energy vehicle by using the peak acceleration value to obtain a violent driving index; calculating the influence coefficient of violent driving on the battery temperature according to the violent driving index to obtain a driving interference coefficient; and correcting the measured battery temperature gradient by using the peak acceleration value and the driving interference coefficient to obtain the battery temperature gradient of the new energy vehicle.

[0019] In the present invention, through time synchronization calibration, the timestamps of the movement trajectory and the battery temperature are aligned, directly correlating the battery temperature and the movement trajectory. By calculating the violent driving index through the peak acceleration value, the aggressive driving degree of the driver is quantified, and further the driving interference coefficient is calculated, fully considering the influence of the human driving factor on the battery temperature. Using the violent driving index and the driving interference coefficient to correct the measured battery temperature gradient makes the finally obtained battery temperature gradient more accurately reflect the true state of the vehicle battery, avoiding misjudgment caused by driving behavior and improving the accuracy of calculating the battery temperature gradient.

[0020] Optionally, the battery temperature gradient satisfies the following formula: The battery temperature gradient satisfies the following formula: Wherein, is the battery temperature gradient, is the measured battery temperature gradient, is the correction temperature weight, is the violent driving index, is the peak acceleration.

[0021] The battery temperature gradient formula of the present invention comprehensively considers factors such as the measured gradient, violent driving index, and peak acceleration, fully taking into account the influence of violent driving and environmental factors on the battery temperature, and improving the accuracy of calculating the battery temperature gradient.

[0022] Optionally, the predicting and alarming of tunnel fires using the battery temperature gradient includes: setting a safety threshold, calculating the risk level of tunnel fires according to the safety threshold and the battery temperature gradient; evaluating the level of the risk level, and giving an alarm according to the level.

[0023] By setting a safety threshold and calculating the risk level of tunnel fires accordingly, the present invention can convert the abstract data of the battery temperature gradient into a specific risk quantification index, which is convenient for intuitively evaluating the possibility of fire occurrence. Evaluating the level of the risk level realizes hierarchical management of risks, and different levels correspond to different degrees of danger. Giving an alarm based on the level allows managers and relevant personnel to quickly understand the risk situation and take corresponding measures in a timely manner, further improving the practical application ability of the present invention.

[0024] Another aspect of the present invention also provides a new energy vehicle fire warning system in a tunnel, including: a processor, an input device, an output device, and a memory, which are interconnected. Among them, the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute any one of the methods for warning new energy vehicle fires in a tunnel in the previous aspect of the present invention.

[0025] The new energy vehicle fire warning system in a tunnel of the present invention has a compact structure, stable performance, high integration, and simple composition, and can stably execute the method for warning new energy vehicle fires in a tunnel provided in the previous aspect of the present invention, further improving the overall applicability and practical application ability of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flowchart of a method for warning new energy vehicle fires in a tunnel according to an embodiment of the present invention; Figure 2 is a schematic structural diagram of a new energy vehicle fire warning system in a tunnel according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0027] Specific embodiments of the present invention will be described in detail below. It should be noted that the embodiments described here are only for illustrative purposes and are not used to limit the present invention. In the following description, in order to provide a thorough understanding of the present invention, a large number of specific details are set forth. However, it is obvious to those of ordinary skill in the art that the present invention does not have to employ these specific details. In other instances, well-known circuits, software, or methods have not been described in detail in order to avoid obscuring the present invention.

[0028] Throughout the specification, references to "one embodiment", "an embodiment", "an example", or "an example" mean that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment of the present invention. Thus, the phrases "in one embodiment", "in an embodiment", "an example", or "an example" appearing throughout the specification do not necessarily all refer to the same embodiment or example. Additionally, the particular features, structures, or characteristics may be combined in any suitable combination and / or sub-combination in one or more embodiments or examples. Further, those of ordinary skill in the art should understand that the diagrams provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0029] Please refer to Figure 1 , such as Figure 1 which is a flowchart of a method for early warning of new energy vehicle fires in a tunnel according to an embodiment of the present invention. In order to solve the problem of the lack of early warning of new energy vehicle fires in the tunnel in the prior art, the method includes the following steps: Step S1, introduce the YOLOv8 model, identify new energy vehicles entering the tunnel based on the YOLOv8 model, and obtain visible light images, thermal imaging images, and millimeter wave radar point cloud data of the new energy vehicles.

[0030] In this embodiment, the YOLOv8 model is an advanced and efficient object detection and image segmentation model developed by Ultralytics. At the same time, YOLOv8 itself is a pre-trained model with good recognition ability for many common objects. First, collect image data of new energy vehicles and other vehicles in the tunnel environment, covering different lighting, angles, positions, etc., and accurately label the vehicle positions and categories. Then, use the labeled data to fine-tune and train YOLOv8, and appropriately adjust the convolution kernel parameters of the backbone network, the feature fusion parameters of the neck network, and the anchor box parameters of the detection head, etc. to improve performance. During the training process, use the validation set to evaluate the model, and further optimize the parameters based on indicators such as accuracy, recall, and error analysis results. Finally, deploy the trained model to the tunnel detection system to achieve accurate recognition of new energy vehicles.

[0031] In terms of data acquisition, high-definition visible light cameras, thermal imaging equipment and millimeter-wave radars should be deployed at the entrances and exits of the tunnel and at key locations. When a new energy vehicle enters the tunnel, the YOLOv8 model can quickly and accurately identify the vehicle with its powerful target detection capabilities, and accurately capture the vehicle's visible light image, thermal imaging image and millimeter-wave radar point cloud data based on the model's output information.

[0032] The obtained visible light image can clearly present the vehicle's body color, model, license plate and other information, which is helpful for vehicle identification. Thermal imaging images focus on the temperature distribution of the vehicle, especially the heating status of key components such as batteries, motors, and tires. Millimeter-wave radar point cloud data provides precise position, speed and posture information of the vehicle in three-dimensional space. By analyzing these data, the vehicle's driving trajectory and movement status can be more accurately grasped. For example, when a vehicle is driving in a tunnel, the millimeter-wave radar point cloud data can provide real-time feedback on the vehicle's speed changes, distance to surrounding obstacles and other information, and combine with visible light images and thermal imaging images to achieve all-round and multi-angle monitoring of the vehicle.

[0033] Step S2, introducing a multi-target tracking algorithm, and tracking the new energy vehicle using the multi-target tracking algorithm based on the visible light image, the thermal imaging image and the millimeter wave radar point cloud data.

[0034] In this embodiment, the multi-target tracking algorithm refers to simultaneously detecting and tracking multiple target objects, namely, new energy vehicles, in a video sequence, and maintaining the continuity of the identity (ID) of each new energy vehicle between different frames.

[0035] The multi-target tracking algorithm tracks new energy vehicles based on the visible light images, thermal imaging images and millimeter wave radar point cloud data obtained by the visible light camera, thermal imaging camera and millimeter wave radar installed in the tunnel. First, the target detection model is used to detect the position and category of the vehicle from each frame of the image based on the vehicle's appearance features, contours, color and other visual information, and generate a detection frame. Then, the multi-target tracking algorithm tracks the detected new energy vehicles, assigns a unique ID to each new energy vehicle, and forms a unique identification of the new energy vehicle, thereby distinguishing the visible light image, thermal imaging image and millimeter wave radar point cloud data of each new energy vehicle.

[0036] Step S3, based on the target tracking result, the battery temperature of each of the new energy vehicles is calculated using the visible light image, the thermal imaging image and the pre-acquired tunnel environment data.

[0037] Among them, calculating the battery temperature of each of the new energy vehicles by using the visible light image, the thermal imaging image and the pre-acquired tunnel environment data specifically includes the following sub-steps: Step S301, perform semantic segmentation on the visible light image to obtain a battery area mask.

[0038] In this embodiment, based on the result of the target tracking, it is shown that the visible light images of each new energy vehicle have been correctly distinguished. The collected visible light images of the new energy vehicles are input into a trained semantic segmentation model, such as U-Net. This model is trained with a large number of vehicle image samples containing battery areas and can accurately learn the characteristics of the battery area. When the model runs, it judges whether each pixel belongs to the battery area according to the pixel characteristics and context information. After the segmentation is completed, a binary image is generated as the battery area mask, where the pixel value of the battery area is 1 and the other areas are 0.

[0039] Step S302, extract feature points from the visible light image and the thermal imaging image respectively to obtain visible light feature points and thermal imaging feature points.

[0040] In this embodiment, the scale-invariant feature transform algorithm can be used to extract feature points from the visible light image and the thermal imaging image respectively. This algorithm can effectively identify unique points in the visible light image, such as corner points on the vehicle contour edge, obvious feature points of components, etc., so as to obtain visible light feature points. For the thermal imaging image, since it reflects the temperature distribution information, a feature point extraction method based on the temperature gradient change can be used to detect the extreme value points of the temperature gradient as thermal imaging feature points in the areas where the temperature of heat-generating components such as batteries changes significantly. By extracting the feature points of the two images respectively, important basic data is provided for subsequent image matching and analysis, which helps to more accurately associate the vehicle information in the visible light and thermal imaging images.

[0041] Step S303, match the visible light feature points and the thermal imaging feature points, and construct a homography matrix from the visible light feature points to the thermal imaging feature points according to the matching result.

[0042] In this embodiment, after the visible light feature points and the thermal imaging feature points are extracted, the fast library for approximate nearest neighbor search FLANN can be used to calculate the similarity measure between them according to the descriptors of the feature points, and match the visible light feature points and the thermal imaging feature points with high similarity. After the feature point matching is completed, the random sample consensus algorithm RANSAC is used. It selects a small number of feature point pairs by random sampling to estimate the homography matrix, and then uses this estimated matrix to test all other feature point pairs. The point pairs that conform to the matrix transformation relationship are regarded as inliers, and those that do not conform are regarded as outliers. Through multiple samplings and calculations, the estimated matrix with the most inliers is selected as the final homography matrix.

[0043] The homography matrix describes the projective transformation relationship of points on a plane from one image to another. When the scene points corresponding to the two image planes are all on the same plane (the vehicle battery area is approximately regarded as a plane), then there is a two-dimensional projective transformation between these two images, and this transformation can be represented by a 3×3 matrix, that is, the homography matrix.

[0044] By constructing the homography matrix from visible light feature points to thermal imaging feature points, the battery area mask segmented in the visible light image can be accurately mapped onto the thermal imaging image. Since the thermal imaging image mainly reflects temperature information and it is difficult to directly and accurately locate the battery area itself, while the visible light image can provide clear vehicle appearance information for segmenting the battery area, the homography matrix is used to achieve the fusion of the two image information, thereby accurately locating the battery area in the thermal imaging image.

[0045] Step S304, map the battery area mask to the thermal imaging image according to the homography matrix to obtain the thermal imaging battery area.

[0046] In this embodiment, for each pixel point in the battery area mask, its coordinates are converted into homogeneous coordinate form, and then the homography matrix is used to perform transformation calculation on this point. The spatial transformation relationship contained in the homography matrix can accurately map the position information of the battery area on the visible light image to the corresponding position on the thermal imaging image.

[0047] It should be noted that during the mapping process, some boundary situations may be encountered. For example, the mapped point falls outside the boundary of the thermal imaging image. At this time, reasonable processing is required, such as discarding the point or correcting it according to the boundary rules. After performing the mapping operation on all pixel points in the battery area mask, a new area, that is, the thermal imaging battery area, is obtained on the thermal imaging image.

[0048] Step S305, calculate the average temperature of the thermal imaging battery area to obtain the battery measurement temperature of the new energy vehicle.

[0049] In this embodiment, each pixel point of the thermal imaging image corresponds to a temperature value. Therefore, traverse all pixel points within the thermal imaging battery area to obtain the temperature value of each pixel point. Then, sum up these temperature values and divide by the total number of pixel points within the thermal imaging battery area to obtain the average temperature value of this area, that is, the battery measurement temperature. This measured temperature reflects the overall thermal state of the battery area and provides important basic data for subsequent evaluation of battery performance, prediction of battery-related risks, and tunnel fire warning, etc.

[0050] Step S306: Use the tunnel environment data to correct the measured battery temperature to obtain the battery temperature of each new energy vehicle.

[0051] In this embodiment, the actual operating temperature of the new energy vehicle battery may be affected by various factors, including environmental temperature, air flow velocity, humidity, and thermal radiation, etc. The directly measured battery temperature may not accurately reflect its true operating state. Through correction, the actual temperature of the battery can be obtained more accurately.

[0052] The battery temperature satisfies the following formula: where, is the value of the battery temperature, is the value of the measured battery temperature, is the environmental temperature correction coefficient, is the value of the real-time environmental temperature in the tunnel, is the reference temperature value, is the air flow velocity influence factor, is the value of the air flow velocity in the tunnel, is the relative humidity influence factor, is the relative humidity of the air in the tunnel, is the reference relative humidity, is the compensation value of the tunnel environmental thermal radiation on the battery temperature.

[0053] The compensation value of the tunnel environmental thermal radiation on the battery temperature satisfies the following formula: where, is the compensation value of the tunnel environmental thermal radiation on the battery temperature, is the absorption coefficient of the battery surface to the tunnel thermal radiation, is the value of the tunnel environmental thermal radiation intensity, is the effective area value of the battery receiving thermal radiation, is the duration value of the battery affected by a specific thermal radiation intensity in the tunnel, is the heat capacity of the lithium battery.

[0054] In this embodiment, indicates that when the real-time environmental temperature in the tunnel is higher than the reference temperature, it is difficult for the battery to dissipate heat, and the battery temperature will rise. When the real-time environmental temperature in the tunnel is lower than the reference temperature, the battery dissipates heat faster, and the battery temperature will drop. The difference between the two indicates that based on the reference temperature, the greater the difference, the greater the rise or fall of the battery temperature. After data verification, the reference temperature is generally taken as 25 degrees Celsius. The relative humidity of the air inside the tunnel follows the same principle as the real-time ambient temperature inside the tunnel. The value generally is 50%.

[0055] Step S4: Based on the result of the target tracking, perform spatio-temporal synchronization calibration on the visible light image, the thermal imaging image, and the millimeter-wave radar point cloud data, and obtain the motion trajectory of each new energy vehicle according to the result of the spatio-temporal synchronization calibration.

[0056] Among them, performing spatio-temporal synchronization calibration on the visible light image, the thermal imaging image, and the millimeter-wave radar point cloud data, and obtaining the motion trajectory of each new energy vehicle according to the result of the spatio-temporal synchronization calibration specifically includes the following sub-steps: Step S401: Perform spatio-temporal synchronization calibration on the visible light image, the thermal imaging image, and the millimeter-wave radar point cloud data to generate multi-modal spatio-temporal sequence data.

[0057] In this embodiment, the multi-modal spatio-temporal sequence data includes the unique identifier of the new energy vehicle, the global position of the new energy vehicle, the global speed of the new energy vehicle, the timestamp of the new energy vehicle, and the original data of each sensor of the new energy vehicle.

[0058] Based on the result of the target tracking, it indicates that the visible light image, the thermal imaging image, and the millimeter-wave radar point cloud data of each new energy vehicle have been distinguished, and the visible light image, the thermal imaging image, and the millimeter-wave radar point cloud data of each new energy vehicle are processed separately. In terms of geometric alignment, tools such as calibration plates are used to calibrate the internal parameters (such as focal length, distortion coefficient) and external parameters (rotation matrix, translation vector) of the visible light image and the thermal imaging image. At the same time, by synchronizing the millimeter-wave radar point cloud data, the conversion parameters from the radar coordinate system to the visible light image coordinate system are obtained, and the data of each sensor is converted into the global coordinate system. In terms of time alignment, since the frequencies of different sensors are different, such as the frequency of the millimeter-wave radar is generally 10 - 20Hz, and the visible light camera is 30 - 60Hz, the method of software interpolation (such as linear interpolation) can be used to ensure that the timestamps of the data of each sensor are consistent.

[0059] For the calculation of the speed of the visible light image, techniques such as the optical flow method can be used. The optical flow method can analyze the pixel displacement of the target in adjacent frame images, and combine the internal and external parameters of the visible light image and the thermal imaging image to convert the pixel displacement into the actual physical displacement, and then calculate the speed of the target in the global coordinate system. For the millimeter-wave radar, the speed information of the target can be directly obtained from its point cloud data. The thermal imaging image can assist the visible light image in target recognition and positioning.

[0060] Then the calculated visible light speed, radar speed, etc. are fused. The weighted average method can be used to dynamically adjust the weight according to the confidence of each sensor to obtain the global speed. The global position can be obtained by converting the data of each sensor into the global coordinate system and fusing the target position. For example, the center of mass position is calculated after clustering the radar point cloud data, and the area of ​​interest of the target is extracted for the image data and converted into global coordinates.

[0061] Finally, using a unified timestamp as an index, the vehicle's unique identification, global position, global speed, timestamp, and the raw data of each sensor (such as visible light image path, radar point cloud, etc.) are integrated together to form multimodal spatiotemporal series data.

[0062] Step S402: Generate a historical motion trajectory according to the multimodal spatiotemporal sequence data.

[0063] In this embodiment, firstly, the time series of timestamps and global positions are extracted from the multimodal spatiotemporal sequence data and arranged in chronological order to form the original trajectory. Sensor noise is eliminated by Kalman filtering or Savitzky-Golay filtering, and cubic spline interpolation is performed on the missing points caused by high-frequency downsampling to ensure trajectory continuity. Then, by taking advantage of multi-sensor fusion, the velocity components calculated by millimeter-wave radar velocity measurement and visible light optical flow method are combined, and the velocity characteristics of the trajectory are optimized by weighted averaging, and finally a historical motion trajectory including but not limited to timestamps, vehicle IDs, coordinates, velocity vectors and timestamps is obtained.

[0064] Step S403: construct an occlusion compensation model, and use the occlusion compensation model to perform prediction according to the historical motion trajectory to obtain the vehicle prediction trajectory and confidence.

[0065] Among them, building an occlusion compensation model includes: Step S30501, the long short-term memory network constructs a time series feature extraction module.

[0066] In this embodiment, the time series feature extraction module built based on the long short-term memory network (LSTM) captures the long-term dependencies in the multimodal spatiotemporal sequence data by designing a multi-layer LSTM network structure and using a gating mechanism. The input layer converts multi-dimensional data such as global position, speed, sensor features (such as radar intensity, image, coordinates) into embedded vectors, the hidden layer transmits time series information through a chain structure, and the output layer extracts motion pattern features (such as acceleration and curvature). Combined with a bidirectional LSTM or attention mechanism, the correlation modeling between historical trajectories and future predictions can be enhanced.

[0067] Step S30502, constructing a confidence calculation module according to the differences of the sample data to be input.

[0068] In this embodiment, the difference in the sample data to be input refers to the degree of difference in the descriptions of the same physical object or scene by different sensors or data sources in the multi-modal data fusion task in the present invention. Its core objective is to quantify the inconsistencies in spatio-temporal alignment, semantic expression, etc. of different modal data.

[0069] The reliability calculation module satisfies the following formula: Wherein, is the difference in data, is the number of samples, is the position of the target new energy vehicle in the visible light image of the target vehicle, is the position of the target new energy vehicle in the millimeter wave radar point cloud data of the target vehicle, is the confidence level, and are both adjustable parameters.

[0070] The adjustable parameter controls the sensitivity of the confidence level to the data difference The larger , the more significant the influence of Mc on the confidence level.

[0071] The adjustable parameter plays a role in adjusting the position of the confidence level curve and can be adjusted according to the actual data and application scenarios to determine at what degree of the confidence level begins to decline rapidly.

[0072] This formula maps the difference in data to the confidence level interval from 0 to 1. The smaller the consistency of the data, the lower the consistency, and the less reliable the fusion result. Conversely, the more reliable the fusion result.

[0073] Step S30503, construct a loss function based on the time series feature extraction module and the confidence calculation module.

[0074] In this embodiment, the loss function satisfies the following formula: Wherein, is the loss function, is the weight coefficient, is the mean square error function, is the predicted trajectory, is the true trajectory, is the weight coefficient, is the cross-entropy loss function, is the predicted confidence level, is the true confidence level.

[0075] Step S30504: Construct an occlusion compensation model based on the time series feature extraction module, the confidence calculation module, and the loss work function.

[0076] In this embodiment, when constructing the occlusion compensation model, the time series feature extraction module, the confidence calculation module, and the loss function are comprehensively utilized to optimize the output of the model. The time series feature extraction module is responsible for capturing the behavior patterns and appearance changes of the target from consecutive video frames, and these features are subsequently used to predict the future state of the target. The confidence calculation module evaluates the reliability of these predictions. Especially when the target is partially or completely occluded, it assigns a confidence score to each prediction result, and this score reflects the confidence level of the model in its prediction. The loss function is used to measure the deviation between the model prediction and the actual observation, and it guides the parameter optimization of the model during the training process to minimize this deviation. The output of the model includes the prediction result and the corresponding confidence, which not only provides the prediction of the future position and state of the target for the tracking algorithm but also provides the credibility information of the prediction. This design allows the system to process the prediction results more intelligently in the face of uncertainties such as target occlusion, thereby further improving the accuracy of the prediction.

[0077] Before training the occlusion compensation model, historical data of the vehicle under normal driving and occlusion scenarios are continuously collected through multi-modal sensors (visible light cameras, thermal imaging devices, millimeter-wave radars) in the tunnel, including the global position, speed, timestamp of the vehicle, and the original sensor data (such as radar point clouds, images), and multi-modal spatio-temporal sequence data are generated through spatio-temporal synchronization calibration. During data processing, the historical motion trajectories are filtered, denoised, and interpolated to ensure trajectory continuity; the occlusion scenario data are labeled to simulate different degrees of occlusion states. Subsequently, the processed sample data are input into the model. Among them, the time series feature extraction module uses an LSTM network to capture the long-term dependence relationship of the motion pattern, the confidence calculation module quantifies the prediction reliability based on the multi-modal data difference (such as the position deviation between visible light and radar), and the loss function combines the mean square error (MSE) of the predicted trajectory and the cross-entropy (CE) of the confidence for optimization training. Finally, the occlusion compensation model is iteratively trained through the sample data so that it can accurately compensate for the motion trajectory of the vehicle when it is occluded by fusing the measured trajectory and the predicted trajectory according to the confidence.

[0078] Step S404: Stitch the vehicle prediction trajectory and the historical motion trajectory according to the confidence to obtain the motion trajectory of each new energy vehicle.

[0079] The motion trajectory of the new energy vehicle satisfies the following formula: where is the movement trajectory of a new energy vehicle, is the historical movement trajectory of the new energy vehicle, is the confidence level, is the predicted trajectory of the new energy vehicle by the occlusion compensation model.

[0080] Step S5, perform time synchronization calibration on the movement trajectory and the battery temperature, and calculate the battery temperature gradient of the new energy vehicle using the result after the time synchronization calibration.

[0081] Among them, performing time synchronization calibration on the movement trajectory and the battery temperature, and calculating the battery temperature gradient of the new energy vehicle using the result after the time synchronization calibration specifically includes the following sub-steps: Step S501, perform time synchronization calibration on the movement trajectory and the battery temperature to obtain trajectory-temperature fusion data.

[0082] In this embodiment, since the sampling frequencies of different sensors may vary, this will lead to inconsistent timestamps. For this situation, the interpolation method is used for fine synchronization. If the sampling interval of the battery temperature data is 5 seconds, and the sampling interval of the movement trajectory data is 1 second, the battery temperature value corresponding to the movement trajectory time point can be estimated by the linear interpolation method according to the known battery temperature data points.

[0083] The linear interpolation satisfies the following formula: Among them, is the interpolated temperature at time , the temperature corresponding to time is , and the temperature corresponding to time is .

[0084] After the interpolation process is completed, verify the result of the time synchronization. Calculate the correlation between the movement trajectory and the battery temperature data at adjacent time points. If the correlation is lower than the set threshold, it is considered that there is an error in the synchronization. At this time, the error can be corrected by adjusting the parameters of the interpolation algorithm or re-performing the coarse alignment. At the same time, introduce a feedback mechanism to regularly calibrate the clocks of the sensors to ensure the long-term stability of the time synchronization, and finally obtain the trajectory-temperature fusion data.

[0085] Step S502, extract the timestamp data and speed data from the trajectory-temperature fusion data, and calculate the peak acceleration of the new energy vehicle according to the timestamp data and the speed data.

[0086] In this embodiment, by means of data parsing technology, according to the format and structure of the trajectory temperature fusion data, the timestamp data and speed data are accurately separated. Specifically, a specific code script can be written to traverse the trajectory temperature fusion data and store the timestamp and speed into different arrays or lists respectively to ensure the integrity and accuracy of the data.

[0087] Preprocess the extracted timestamp data and speed data. Check whether there are missing values and outliers in the timestamp data and speed data. If there are missing values, interpolation methods such as linear interpolation can be used to fill them; if there are outliers, statistical methods such as the method based on standard deviation can be used to identify and remove them. At the same time, sort the data to ensure that the timestamps are arranged in ascending order, laying a foundation for subsequent calculations.

[0088] According to the definition of acceleration in physics, acceleration is the ratio of the change in velocity to the time taken for this change. Traverse all data points and calculate the acceleration for each time period in turn. Finally, find the acceleration peak value. Among all the calculated acceleration values, find the maximum value by comparing the magnitudes, and this maximum value is the acceleration peak value of the new energy vehicle.

[0089] When calculating the acceleration peak value, select two time points in chronological order and , the time interval is , and the corresponding speeds are and .

[0090] The acceleration peak value satisfies the following formula: where is the acceleration peak value, is the new energy vehicle speed vector at moment, is the new energy vehicle speed vector at moment, is and time interval.

[0091] Step S503, extract the temperature data from the trajectory temperature fusion data, and calculate the measured battery temperature gradient of the new energy vehicle according to the timestamp data and the temperature data.

[0092] In this embodiment, the extraction and processing of the temperature data are the same as the above method for extracting speed data. When calculating the battery temperature gradient, select two time points and in chronological order, the time interval is , and the corresponding battery temperatures are and .

[0093] The measured battery temperature gradient satisfies the following formula; The temperature gradient between each two adjacent time points is calculated in sequence, and finally, the calculation results are sorted out. All the calculated temperature gradient data are summarized to obtain the measured battery temperature gradient.

[0094] Step S504: Calculate the aggressive driving degree of the new energy vehicle using the acceleration peak value to obtain a violent driving index.

[0095] The violent driving index satisfies the following formula: in, is the violent driving index, is the number of acceleration samples within a predetermined time, For the acceleration value, is the acceleration threshold.

[0096] In this embodiment, the acceleration threshold is used to distinguish the acceleration limit between normal driving and aggressive driving. When the acceleration exceeds this threshold, it is considered as a behavior with aggressive driving characteristics.

[0097] Step S505, calculating the influence coefficient of violent driving on battery temperature according to the violent driving index to obtain a driving interference coefficient.

[0098] The driving interference coefficient satisfies the following formula: in, is the driving interference coefficient, The violent driving index.

[0099] In this formula, As the basic offset, even if the violent driving index is 0, that is, there is no violent driving behavior, the driving interference coefficient also has a basic value of 0.5, which represents the basic influence of other factors on the battery temperature except violent driving, and provides a starting benchmark for the driving interference coefficient.

[0100] is the scaling factor, used to adjust The influence of the function output value on the driving interference coefficient, The function has a range of -1 to 1, which becomes -0.3 to 0.3 after scaling, limiting the range of the impact of aggressive driving behavior on battery temperature and avoiding excessive fluctuations in the calculation of the impact.

[0101] Among them, it plays the role of a balance point, enabling the violent driving index to change up and down with 2 as the reference. Through the function to reasonably adjust the driving interference coefficient.

[0102] Step S506: Use the acceleration peak value and the driving interference coefficient to correct the measured battery temperature gradient, and obtain the battery temperature gradient of the new energy vehicle.

[0103] The battery temperature gradient satisfies the following formula: Because, Therefore, the battery temperature gradient satisfies the following formula: Wherein, is the battery temperature gradient, is the measured battery temperature gradient, is the correction temperature weight, is the violent driving index, is the acceleration peak value.

[0104] In this embodiment, by introducing the correction temperature weight, the temperature correction module is flexibly adjusted. Through repeated experiments and verifications, takes the value of 0.1, which is applicable to most tunnel conditions. Combining with the acceleration peak value, it can be known that the greater the acceleration peak value, the more temperature needs to be corrected.

[0105] Step S6: Predict and alarm the tunnel fire according to the battery temperature gradient.

[0106] Among them, predicting and alarming the tunnel fire according to the battery temperature gradient specifically includes the following sub-steps: Step S601: Set a safety threshold, and calculate the risk degree of the tunnel fire according to the safety threshold and the battery temperature gradient.

[0107] In this embodiment, first set the safety threshold of the battery temperature gradient. The safety threshold represents the safety critical value of the battery temperature change rate. The battery temperature gradient is compared with the safety threshold, and the risk degree of the tunnel fire is quantified through the relationship between the two values. When the battery temperature gradient exceeds the safety threshold, the greater the excess part, the more obvious the abnormal change of the battery temperature, and the higher the fire risk of the tunnel.

[0108] The risk degree satisfies the following formula: Wherein, is the risk level, is the safety threshold, is the risk amplification weight.

[0109] Step S602: Evaluate the rating of the risk level and issue an alarm according to the rating.

[0110] In this embodiment, the risk level takes values from 0 to 1. When it is a low risk, when it is a low risk, when it is a high risk, it is a risk-free.

[0111] According to the rating levels of risk-free, low risk, medium risk and high risk, different alarms are issued respectively.

[0112] When the rating level is risk-free, it indicates that the tunnel is in a relatively safe state, and at least there will be no safety accidents in the tunnel caused by the fire of new energy vehicles, so there is no need to notify the tunnel safety officer.

[0113] When the rating level is low risk, notify the tunnel safety officer that the tunnel is in a low risk state. To prevent unexpected situations from occurring, remind the safety officer to pay attention to the further state of the tunnel at any time.

[0114] When the rating level is medium risk, notify the tunnel safety officer that the tunnel is in a medium risk state. The safety officer should be ready to deal with possible fire accidents at any time. There is a safety broadcast in the tunnel, which broadcasts to the corresponding vehicles in the way of license plate reading, and notifies the involved vehicles to pay attention to checking the battery temperature of their own vehicles to prevent the situation from escalating further.

[0115] When the rating level is high risk, notify the tunnel safety officer that the tunnel is in a high risk state. The safety officer should take corresponding accident rescue measures, such as coordinating the nearby fire trucks and ambulances to standby, and restricting the flow of vehicles in the reverse direction of the tunnel.

[0116] As Figure 2 shown, another aspect of the present invention also provides a fire warning system for new energy vehicles in a tunnel, including: a processor, an input device, an output device and a memory. The processor, the input device, the output device and the memory are connected to each other. Among them, the memory is used to store a computer program, and the computer program includes program instructions. The processor is configured to call the program instructions to execute the relevant steps of the relevant embodiments in a fire warning method for new energy vehicles in a tunnel according to the present invention.

[0117] A fire warning system for new energy vehicles in tunnels provided by the present invention, each functional component can be integrated into a processing component, or each component can exist physically alone, or two or more components can be integrated into one component. The above-mentioned integrated components can be implemented in the form of hardware or in the form of software functions, further improving the overall applicability and practical application ability of the present invention.

[0118] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered by the scope of the claims and the specification of the present invention.

Claims

1. A fire warning method for new energy vehicles in a tunnel, characterized in that: The method comprises: A YOLOv8 model is introduced, and new energy vehicles entering the tunnel are identified based on the YOLOv8 model, and visible light images, thermal imaging images, and millimeter wave radar point cloud data of the new energy vehicles are obtained; Introducing a multi-target tracking algorithm, and tracking the new energy vehicle using the multi-target tracking algorithm based on the visible light image, the thermal imaging image, and the millimeter wave radar point cloud data; Based on the target tracking result, the battery temperature of each of the new energy vehicles is calculated using the visible light image, the thermal imaging image and the pre-acquired tunnel environment data; Based on the target tracking result, the visible light image, the thermal imaging image and the millimeter wave radar point cloud data are calibrated in time and space, and the motion trajectory of each of the new energy vehicles is obtained according to the result of the time and space synchronization calibration; Performing time synchronization calibration on the motion trajectory and the battery temperature, and calculating the battery temperature gradient of the new energy vehicle using the result of the time synchronization calibration; Tunnel fire is predicted and warned based on the battery temperature gradient.

2. A fire warning method for new energy vehicles in a tunnel according to claim 1, characterized in that: The method of calculating the battery temperature of each new energy vehicle by using the visible light image, the thermal imaging image and the pre-acquired tunnel environment data includes: Performing semantic segmentation on the visible light image to obtain a battery area mask; Extracting feature points from the visible light image and the thermal imaging image respectively to obtain visible light feature points and thermal imaging feature points; Matching the visible light feature points with the thermal imaging feature points, and constructing a homography matrix from the visible light feature points to the thermal imaging feature points according to the matching result; Mapping the battery area mask to the thermal imaging image according to the homography matrix to obtain a thermal imaging battery area; Calculating the average temperature of the thermal imaging battery area to obtain the battery measurement temperature of the new energy vehicle; The battery measurement temperature is corrected using the tunnel environment data to obtain the battery temperature of each of the new energy vehicles.

3. A fire warning method for new energy vehicles in a tunnel according to claim 2, characterized in that: The battery temperature satisfies the following formula: in, is the value of the battery temperature, a value of the temperature measured for said battery, is the ambient temperature correction factor, is the real-time ambient temperature in the tunnel, is the reference temperature value, is the air flow velocity influencing factor, is the value of the air velocity in the tunnel, is the relative humidity factor, is the relative humidity of the air in the tunnel, is the reference relative humidity, It is the compensation value for the influence of thermal radiation from tunnel environment on battery temperature.

4. A fire warning method for new energy vehicles in a tunnel according to claim 1, characterized in that: The step of performing spatiotemporal synchronous calibration on the visible light image, the thermal imaging image and the millimeter wave radar point cloud data, and obtaining the motion trajectory of each of the new energy vehicles according to the result of the spatiotemporal synchronous calibration comprises: Performing spatiotemporal synchronous calibration on the visible light image, the thermal imaging image, and the millimeter wave radar point cloud data to generate multimodal spatiotemporal sequence data; Generating a historical motion trajectory according to the multimodal spatiotemporal sequence data; Constructing an occlusion compensation model, and using the occlusion compensation model to make predictions based on the historical motion trajectory to obtain a vehicle prediction trajectory and a confidence level; The predicted vehicle trajectory and the historical motion trajectory are spliced ​​according to the confidence level to obtain the motion trajectory of each of the new energy vehicles.

5. A fire warning method for new energy vehicles in a tunnel according to claim 4, characterized in that: The constructing of the occlusion compensation model comprises: Use long short-term memory network to build time series feature extraction module; Construct a confidence calculation module according to the differences of the sample data to be input; Constructing a loss work function based on the time series feature extraction module and the confidence calculation module; An occlusion compensation model is constructed based on the time series feature extraction module, the confidence calculation module and the loss work function.

6. A fire warning method for new energy vehicles in a tunnel according to claim 4, characterized in that: The motion trajectory of the new energy vehicle satisfies the following formula: in, is the motion trajectory of new energy vehicles, is the historical movement trajectory of new energy vehicles, is the confidence level, Predicted trajectory of new energy vehicles using occlusion compensation model.

7. A fire warning method for new energy vehicles in a tunnel according to claim 1, characterized in that: The step of performing time synchronization calibration on the motion trajectory and the battery temperature, and calculating the battery temperature gradient of the new energy vehicle using the result of the time synchronization calibration includes: Performing time synchronization calibration on the motion trajectory and the battery temperature to obtain trajectory-temperature fusion data; Extracting timestamp data and speed data from the trajectory temperature fusion data, and calculating the acceleration peak value of the new energy vehicle according to the timestamp data and the speed data; Extracting temperature data from the trajectory temperature fusion data, and calculating a measured battery temperature gradient of the new energy vehicle according to the timestamp data and the temperature data; Calculating the aggressive driving degree of the new energy vehicle using the acceleration peak value to obtain a violent driving index; Calculate the influence coefficient of violent driving on battery temperature according to the violent driving index to obtain a driving interference coefficient; The measured battery temperature gradient is corrected using the acceleration peak value and the driving interference coefficient to obtain the battery temperature gradient of the new energy vehicle.

8. A fire warning method for new energy vehicles in a tunnel according to claim 7, characterized in that: The battery temperature gradient satisfies the following formula: The battery temperature gradient satisfies the following formula: in, is the battery temperature gradient, To measure the battery temperature gradient, To correct the temperature weight, is the violent driving index, is the peak acceleration.

9. A fire warning method for new energy vehicles in a tunnel according to claim 1, characterized in that: The predicting and alarming of tunnel fire according to the battery temperature gradient comprises: Setting a safety threshold, and calculating the risk level of tunnel fire according to the safety threshold and the battery temperature gradient; The risk level is graded and an alert is issued based on the grade.

10. A fire warning system for new energy vehicles in tunnels, characterized in that: include: A processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute a fire warning method for new energy vehicles in a tunnel as described in any one of claims 1 to 9.

Images