Multi-mode fusion fire early warning and automatic fire extinguishing method applied to battery car charging scene

Abstract

The invention provides a multi-modal fusion fire early warning and automatic fire extinguishing method applied to a battery car charging scene, which comprises the following steps that gas concentration data is acquired through a gas detection sensor array deployed in a battery car charging area, image data is acquired through an image detection camera array, and all the data carry timestamp information; a battery car target detection model is adopted to carry out battery car identification on the acquired image data, and dual-path detection analysis is executed according to an identification result; the method comprises the following steps: 1, executing multi-modal depth feature coding, cross-modal depth fusion and fire probability prediction if the battery car is successfully identified, and 2, independently detecting the gas concentration if the battery car is failed to be identified; and finally, executing an intelligent step early warning strategy based on a dual-path detection analysis result. According to the invention, deep cooperation between modes and advanced early warning of fire are realized, the problem of single-point fault is solved, single-mode false alarm information is effectively inhibited, and the false alarm rate is reduced to below 5%.

Description

Technical Field

[0001] This invention belongs to the field of fire protection Internet of Things technology, and in particular relates to a multimodal fusion fire early warning and automatic fire extinguishing method applied to electric vehicle charging scenarios. Background Technology

[0002] Electric bicycles have become an important means of transportation for urban residents due to their convenience. However, the safety hazards of charging electric bicycles are becoming increasingly prominent. According to statistics, 80% of electric bicycle fires occur during the charging process. When electric bicycles, especially lithium batteries, experience thermal runaway, they are characterized by rapid ignition, rapid spread, and great difficulty in firefighting. After a battery short circuit ignites, it can enter a stage of intense combustion within 12 seconds.

[0003] Currently, there are two main types of fire early warning systems that can be used for fire prevention and suppression: One type is a detection system based on various fire detection devices, such as smoke detectors, heat detectors, and combustible gas detectors. Existing technology discloses a solution for remote monitoring and early warning via the Internet of Things (IoT) in fire protection. This solution uses IoT sensing technology to remotely monitor fire alarm control panels, fire pools / tanks, sprinkler networks, fire hydrant networks, and independent smoke detectors / heat detectors / combustible gas detectors, and obtains alarm information for real-time early warning. Another type is based on image recognition. Existing technology discloses an intelligent fire early warning and automatic fire suppression method, which describes that the combustible gas concentration, the smoke image and the flame image are input into the fire early warning model. The fire early warning model identifies the combustible gas concentration, the smoke image and the flame image, and determines whether each high-temperature monitoring area has reached the fire early warning level. If it is determined that the fire early warning level has been reached, the corresponding fire alarm level is determined.

[0004] However, there are still obvious shortcomings when applying this type of existing technology to electric vehicle charging scenarios: 1. As in the existing technology, a remote monitoring and early warning solution for fire protection via the Internet of Things does not take into account the special characteristics of electric vehicle charging areas and the early characteristics of lithium battery thermal runaway, resulting in a problem of delayed early warning; 2. Existing intelligent fire early warning and automatic fire suppression methods mainly rely on image recognition, which is easily interfered with under complex lighting conditions. Such technologies also lack specificity for interference sources unique to electric vehicle charging areas, such as reflections from vehicle headlight lenses and people smoking, resulting in a high false alarm rate. 3. The technology relies on a single detection mode, and the entire system fails when the single mode fails, posing a high risk of single point of failure. 4. Both existing technologies have limited early warning capabilities and neither has optimized the configuration of the sensor array and the algorithm model for electric vehicle charging scenarios. In particular, neither has conducted targeted analysis based on the status of the electric vehicle, resulting in resource waste, inefficient early warning, and false fire alarms.

[0005] Therefore, improvements to existing technologies are necessary. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a fire early warning and automatic response system specifically for electric vehicle charging areas. Through multimodal data fusion analysis and scene adaptive recognition, it can achieve early, accurate and efficient fire early warning and response.

[0007] One technical solution adopted by this invention is to provide a multimodal fusion fire early warning and automatic fire extinguishing method for electric vehicle charging scenarios, comprising the following steps: S1. Gas concentration data is collected by a gas detection sensor array deployed in the electric vehicle charging area, and image data is collected by an image detection camera array. All data are accompanied by timestamp information. S2. Use the electric vehicle target detection model to identify electric vehicles from the collected image data. If the electric vehicle is successfully identified, proceed to step S3 for the area where the electric vehicle is identified. If the electric vehicle is not identified, proceed to step S4. S3. Perform multimodal deep feature encoding: The gas concentration data is input into the first Transformer encoder to extract the temporal feature representation of the gas concentration. The image data is input into the VisionTransformer encoder to extract the visual feature representation of the image. The output features of the two encoders are aligned and stitched together according to the timestamp to form a multimodal fusion feature. Step S5 is then executed. S4. Perform independent gas concentration detection: Gas concentration data is input into an LSTM neural network in time series form. The LSTM neural network predicts whether a fire will occur based on the gas concentration time series data and outputs the predicted value of the fire gas concentration characteristic signal. S5. Perform cross-modal deep fusion and fire probability prediction: The multimodal fusion features formed in step S3 are input into the second Transformer encoder to perform cross-modal feature interaction, capturing the deep correlation between gas concentration and visual features. The interacted features are then input into the LSTM decoder to output the probability value of fire occurrence. S6. Execute the intelligent tiered early warning strategy based on the output of step S4 or S5: When the probability value output in step S5 is in the range of 0.1-0.3, an L1 level warning is triggered, the data is recorded and a notification is given; When the probability value output in step S5 is in the range of 0.3-0.7 or the predicted value output in step S4 is greater than the set environmental gas concentration threshold, an L2 level warning is triggered, an audible and visual alarm is activated, and the personnel in charge are notified to confirm. When the probability value output in step S5 is in the range of 0.7-1.0, an L3 level warning is triggered, the audible and visual alarm is activated, and automatic fire extinguishing is performed.

[0008] Furthermore, in step S1, the gas detection sensor array is distributed and deployed at a low position in the electric vehicle charging area, within the height range of the electric vehicle battery.

[0009] Furthermore, in step S2, the output of the electric vehicle target detection model is the electric vehicle recognition confidence level. If the electric vehicle recognition confidence level reaches the set threshold, it is determined that the electric vehicle recognition is successful; if the electric vehicle recognition confidence level is lower than the set threshold, it is determined that the electric vehicle recognition fails.

[0010] Furthermore, in step S4, the LSTM neural network, which is trained separately using gas concentration data from the electric vehicle charging scenario, focuses on identifying fire characteristics from the temporal changes in gas concentration.

[0011] Furthermore, in step S6, the ambient gas concentration threshold is dynamically adjusted based on historical normal data to adapt to changes in the gas concentration baseline under different seasons and weather conditions.

[0012] Furthermore, in step S6, if the L2 level warning is triggered by independent detection of gas concentration, it also includes a warning escalation step: if no confirmation signal is received from the personnel in charge within a preset time and the predicted value continues to exceed the escalation threshold, the warning level will be escalated from L2 level warning to L3 level warning.

[0013] Furthermore, the target detection model for the electric vehicle is selected from one of the following models: YOLOv5s, YOLOv8m, and YOLOv8_VanillaBlock.

[0014] Furthermore, the YOLOv5s model is improved by replacing the Conv module in its backbone network with the GhostConv module, replacing the C3 module in its neck with the ShuffleNetV2 module, and adding a CA module before the SPP module in its backbone network. These improvements effectively reduce the computational load and improve the model's inference speed.

[0015] The beneficial effects of this invention, applied to the multimodal fusion fire early warning and automatic fire extinguishing method in electric vehicle charging scenarios, are as follows: 1. By using timestamps to accurately align data, fine-grained interaction between gas features and visual features is achieved through multimodal deep feature encoding and cross-modal deep fusion. This strengthens the correlation between slight increases in gas concentration and weak thermal anomalies in images, realizes deep collaboration between modalities, enables early warning of fires, and effectively suppresses false alarms from single modalities. 2. A dual-path detection mechanism is adopted, which can maintain the early warning function through independent gas concentration detection even when image recognition fails, thus solving the single-point failure problem. Cross-modal fusion prediction and independent gas detection complement each other, reducing the false alarm rate to below 5%. Attached Figure Description

[0016] Figure 1 This is a flowchart of the dual-path detection mechanism of the present invention; Figure 2 This is a flowchart of the multimodal fusion analysis of the present invention; Figure 3 This is a flowchart of the independent gas concentration detection process of the present invention; Figure 4 This is the network architecture of the YOLOv5s model used in the electric vehicle target detection model of this invention; Figure 5 This invention provides an improved network architecture for the YOLOv5s model. Detailed Implementation

[0017] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby providing a clearer and more explicit definition of the scope of protection of the present invention.

[0018] First embodiment of the present invention: Implementation method of outdoor electric vehicle charging station Sensor configuration: Eight gas detection sensors are distributed at a low position in the electric vehicle charging station (so that the gas detection sensors are within the height range of the electric vehicle battery), with a sampling frequency of 1Hz. Four image detection cameras with a resolution of 1080P are arranged on the rainproof structure on the top of the charging station. Other hardware configurations: The data processing unit uses edge computing devices and is equipped with a GPU accelerator; the fire extinguishing device uses an ultra-fine dry powder automatic fire extinguishing system.

[0019] Specific implementation details of each step in the software: In step S1, the gas detection sensor detects and records the concentrations of CO and H2 gases in the environment every 10 seconds. In terms of data format, the gas concentration data is input in the form of a time series, with each time point containing a timestamp and CO and H2 concentration values. The image detection camera captures images every 10 seconds, synchronously collecting image data to ensure time alignment. In step S2, the YOLOv5s model is used as the target detection model for electric bicycles to identify them. The confidence threshold for identification is set to 0.7. When the confidence is lower than 0.7, the electric bicycle identification is considered to have failed. In step S3, The image data input to the VisionTransformer encoder is cropped image data containing the target electric vehicle, which can effectively reduce the amount of computation and focus on the visual feature extraction of the electric vehicle region. The output features of two encoders are concatenated (at the same timestamp) using the torch function in Python to form a multimodal fusion feature. Specifically, the configuration of the first Transformer encoder is 2 input dimensions, 128 hidden layer dimensions, 8 attention heads, and 4 layers. The configuration of the VisionTransformer encoder is 224*224 image size, 16*16 block size, 256 hidden layer dimensions, 12 attention heads, and 6 layers. In step S4, The time series data of gas concentration is represented as a vector X(t)=[x(t1),x(t2),...,x(tn)], where x(tn) represents the concentration data of the nth gas at time t, and this vector is used as the input of the LSTM neural network; Specifically, the LSTM neural network configuration is: input dimension 2, hidden layer dimension 64, number of layers 2, and output dimension 1. In step S5, the first Transformer encoder is configured with an input dimension of 384, a hidden layer dimension of 512, 16 attention heads, and 8 layers, while the LSTM decoder is configured with an input dimension of 512, a hidden layer dimension of 256, 3 layers, and 1 output. In step S6, the intelligent stairwell early warning strategy is as follows: Level 1 warning: Probability value 0.1-0.3, log recorded, interface prompt; Level L2 warning: When the probability value is 0.3-0.7 or the predicted value is greater than the set environmental gas concentration threshold, a yellow audible and visual alarm will be activated, and a message will be pushed to the monitoring screen of the personnel in charge or to the App notification so that the personnel in charge can confirm in time. Level L3 warning: Probability value 0.7-1.0, triggers a strong red alarm, executes automatic fire suppression, and the message will also be synchronized to the personnel in charge. Specifically, the ambient gas concentration thresholds are set to an initial CO threshold of 50 ppm, an initial H2 threshold of 100 ppm, and a dynamic adjustment range of ±20%. Specifically, the nozzle angle of the fire extinguishing device can be automatically adjusted according to the identified location. After the automatic fire extinguishing is triggered, the spraying time is 5-8 seconds, and the charging power of the electric vehicle is cut off simultaneously with the spraying.

[0020] Dual-path model training: For steps S3 and S5, the model is trained using 2,000 hours of historically collected normal data and 20 real electric vehicle fire data. For step S4, the model is trained using gas concentration data, focusing on the changing trend characteristics of the gas concentration data.

[0021] Outdoor testing: Two schemes were designed: a real fire simulation and an interference scenario simulation. The real fire simulation simulated the early stage of lithium battery thermal runaway and the combustion of plastic parts of the electric vehicle. The simulation method was to trigger the thermal runaway of the lithium battery pack by external heating and stop it before the open flame appeared. Necessary safety measures were adopted during the simulation. The interference scenario test simulated strong light reflection interference and personnel smoking interference.

[0022] The results of the real fire simulation are shown in the table above. In 27 tests, multimodal fusion was successfully detected 23 times, gas was detected independently 4 times, and the average warning time was significantly advanced.

[0023] The test results for the interference scenarios are shown in the table above. In 55 tests, two false alarms of strong light reflection interference occurred when the sun was shining at an extremely low angle. One false alarm of human smoking interference occurred when the smoker was very close to the electric vehicle and the concentration was high. All three false alarms were canceled after manual confirmation by the handling personnel.

[0024] The second embodiment of the present invention improves the YOLOv5s model to enhance its recognition speed: the GhostConv module replaces the Conv module in its backbone network, the ShuffleNetV2 module replaces the C3 module in its neck, and a CA module is added before the SPP module in its backbone network. These improvements effectively reduce the amount of computation and improve the model's inference speed. The GhostConv module, as a computation module, obtains the output feature map by: setting m as the number of feature map channels, using basic convolution to generate m intrinsic feature maps; performing several linear operations on the intrinsic feature maps to generate s Ghost feature maps; and concatenating the intrinsic feature maps and Ghost feature maps to obtain the output feature map. The formula for generating the Ghost feature map is as follows: ,in, It is the intrinsic feature map of each channel, and the generated Ghost feature map is , It generates the j-th Ghost feature map. The j-th linear operation.

[0025] In other embodiments of the present invention, a high-performance real-time detection model specifically designed for two-wheeled vehicles such as electric bicycles—the YOLOv8_VanillaBlock model—can also be used as the target detection model for electric bicycles. This model offers a faster response speed but also requires more sophisticated equipment.

[0026] Performance comparison of the model before and after improvement: The beneficial effects of this invention, applied to the multimodal fusion fire early warning and automatic fire extinguishing method in electric vehicle charging scenarios, are as follows: 1. By using timestamps to accurately align data, fine-grained interaction between gas features and visual features is achieved through multimodal deep feature encoding and cross-modal deep fusion. This strengthens the correlation between slight increases in gas concentration and weak thermal anomalies in images, realizes deep collaboration between modalities, enables early warning of fires, and effectively suppresses false alarms from single modalities. 2. A dual-path detection mechanism is adopted, which can maintain the early warning function through independent gas concentration detection even when image recognition fails, thus solving the single-point failure problem. Cross-modal fusion prediction and independent gas detection complement each other, reducing the false alarm rate to below 5%. 3. Using an LSTM neural network to predict gas concentration changes can identify fire risks from subtle concentration change trends. At the same time, the dynamic environmental gas concentration threshold can adapt to environmental changes, ensuring prediction accuracy while providing an average early warning time of 120 seconds.

[0027] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A multi-modal fusion fire warning and automatic extinguishing method applied to a battery car charging scene, characterized in that, The method comprises the following steps: S1, collecting gas concentration data through a gas detection sensor array deployed in the electric vehicle charging area, and collecting image data through an image detection camera array, all data being attached with timestamp information; S2, using an electric vehicle target detection model to perform electric vehicle recognition on the collected image data, if the electric vehicle recognition is successful, performing step S3 on the area where the electric vehicle is recognized, if the electric vehicle recognition fails, performing step S4; S3, multi-modal deep feature coding: inputting the gas concentration data into a first Transformer encoder to extract time sequence feature representation of the gas concentration, inputting the image data into a VisionTransformer encoder to extract visual feature representation of the image, aligning and splicing the output features of the two encoders according to the timestamp to form multi-modal fusion features, and performing step S5; S4, gas concentration independent detection: inputting the gas concentration data in the form of time sequence into an LSTM neural network, and the LSTM neural network predicting whether a fire occurs according to the gas concentration time sequence data to output a predicted value of a fire gas concentration feature signal; S5, cross-modal deep fusion and fire probability prediction: inputting the multi-modal fusion features formed in step S3 into a second Transformer encoder for cross-modal feature interaction to capture deep correlations between the gas concentration and the visual features, inputting the interacted features into an LSTM decoder to output a probability value of fire occurrence; S6, executing an intelligent step-by-step warning strategy based on the output of step S4 or S5: when the probability value output by step S5 is in the range of 0.1-0.3, triggering L1-level warning, recording data and prompting; when the probability value output by step S5 is in the range of 0.3-0.7 or the predicted value output by step S4 is greater than a set environmental gas concentration threshold, triggering L2-level warning, starting sound-light alarm and notifying disposal personnel for confirmation; when the probability value output by step S5 is in the range of 0.7-1.0, triggering L3-level warning, starting sound-light alarm and performing automatic fire extinguishing.

2. The multi-modal fusion fire warning and automatic extinguishing method applied to the battery car charging scene according to claim 1, characterized in that, In step S1, the gas detection sensor array is distributedly deployed at a low position of the electric vehicle charging area, at the height range of the electric vehicle battery.

3. The multi-modal fusion fire warning and automatic extinguishing method applied to the battery car charging scene according to claim 1, characterized in that, In step S2, the output of the electric vehicle target detection model is an electric vehicle recognition confidence, if the electric vehicle recognition confidence reaches a set threshold, it is determined that the electric vehicle recognition is successful, if the electric vehicle recognition confidence is lower than the set threshold, it is determined that the electric vehicle recognition fails.

4. The multi-modal fusion fire warning and automatic extinguishing method applied to the battery car charging scene according to claim 1, characterized in that, In step S4, the LSTM neural network is trained separately using gas concentration data of the electric vehicle charging scene, and focuses on identifying fire features from the time sequence changes of the gas concentration.

5. The multi-modal fusion fire warning and automatic extinguishing method applied to the battery car charging scene according to claim 1, characterized in that, In step S6, the environmental gas concentration threshold is dynamically adjusted according to historical normal data to adapt to the baseline changes of the gas concentration under different seasons and weather conditions.

6. The multi-modal fusion fire warning and automatic extinguishing method applied to the battery car charging scene according to claim 1, characterized in that, In step S6, if the L2-level warning is triggered by the gas concentration independent detection, a warning escalation step is further included: if no confirmation signal of the disposal personnel is received within a preset time, and the predicted value continuously exceeds an escalation threshold, the warning level is escalated from L2-level warning to L3-level warning.

7. The multi-modal fusion fire warning and automatic fire extinguishing method applied to the electric vehicle charging scene according to any one of claims 1-6, characterized in that, The battery car target detection model selects one of a YOLOv5s model, a YOLOv8m model and a YOLOv8_VanillaBlock model.

8. The multi-modal fusion fire warning and automatic extinguishing method applied to the battery car charging scene according to claim 7, characterized in that, The YOLOv5s model is improved: a GhostConv module is used to replace a Conv module in a backbone network, a ShuffleNetV2 module is used to replace a C3 module in a neck, and a CA module is added before an SPP module in the backbone network.

9. The multi-modal fusion fire warning and automatic extinguishing method applied to the battery car charging scene according to claim 8, characterized in that, The GhostConv module is used as a calculation module to obtain an output feature map in the following manner: assuming that m is a channel number of a feature map, m intrinsic feature maps are generated by using a basic convolution; s Ghost feature maps are generated by using linear operations on the intrinsic feature maps; and the output feature map is obtained by splicing the intrinsic feature maps and the Ghost feature maps.

10. The multi-modal fusion fire warning and automatic extinguishing method applied to the battery car charging scene according to claim 9, characterized in that, The formula for generating the Ghost feature map is wherein, is the intrinsic feature map of each channel, and the generated Ghost feature map is , is the jth linear operation for generating the jth Ghost feature map .

Images