Battery thermal runaway flame detection method and system based on machine vision

Abstract

The invention is suitable for the technical field of battery safety detection, and provides a battery thermal runaway flame detection method and system based on machine vision, and the method comprises the steps: extracting the temperature field distribution characteristics and flame visual characteristics of a battery cluster based on an obtained image; inputting the extracted features and a preset spectral feature threshold value into a multi-modal fusion decision model, and outputting the flame confidence coefficient of the thermal runaway of the battery; based on the infrared thermal imaging image and the visible light image, calculating a three-dimensional space coordinate of the fire point through a binocular parallax principle; and according to the flame confidence coefficient and a preset risk grading threshold value, determining a thermal runaway early warning grade of the battery region, and triggering a corresponding control strategy in combination with the three-dimensional space coordinates. According to the method, the multi-dimensional fire behavior characteristics are determined in the early stage of thermal runaway, and the timeliness and reliability of detection are improved; through cooperation of three-dimensional space positioning and a grading early warning mechanism, the accuracy and timeliness of safety protection of the energy storage system are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of battery safety testing technology, and in particular to a method and system for detecting battery thermal runaway flames based on machine vision. Background Technology

[0002] As a key component of large-scale energy storage, battery energy storage systems are facing increasingly prominent safety issues. Thermal runaway is one of the most serious safety failures in battery systems, typically accompanied by localized high temperatures, smoke, and even open flames. Currently, monitoring of battery thermal runaway flames mainly relies on contact temperature sensors, which use thermocouples or digital temperature sensors placed on the battery surface or inside the battery compartment for point-based temperature measurement; another approach is image analysis technology based on visible light cameras, which identifies fires by recognizing the color and shape of the flames.

[0003] However, contact sensors have fixed deployment locations and limited coverage, making it difficult to capture early localized high temperatures in areas not covered by sensors within battery clusters. Furthermore, their response may be delayed due to heat conduction delays once a flame has formed. Visible light cameras are susceptible to changes in ambient light, smoke obstruction, and light interference, failing to effectively identify flames in low-light or smoldering stages, resulting in a high false alarm rate. Single infrared sensors can only provide single-point temperature data, unable to acquire two-dimensional temperature field distribution, making it difficult to distinguish between normal heat dissipation and a true fire point. This leads to insufficient detection sensitivity and a lack of localization capabilities in the early stages of thermal runaway, making it impossible to reliably identify smoldering states before open flames form, and failing to meet the high reliability requirements of energy storage systems for accurate early warning and rapid suppression of flames. Summary of the Invention

[0004] This invention provides a machine vision-based method and system for detecting battery thermal runaway flames, which addresses the problem of meeting the high reliability requirements of energy storage systems for accurate early warning and rapid suppression of flames.

[0005] The first aspect of this invention provides a machine vision-based method for detecting battery thermal runaway flames, comprising: Acquire infrared thermal images and visible light images of the battery area from dual-light fusion cameras deployed at different spatial locations; The infrared thermal imaging image is processed to extract the temperature field distribution characteristics of the battery cluster; and the visible light image is processed to extract the visual characteristics of the flame. The temperature field distribution features, the flame visual features, and the preset spectral feature thresholds are input into the multimodal fusion decision model. The multimodal fusion decision model is used to perform fusion calculations on multi-source features based on evidence theory and output the flame confidence level of battery thermal runaway. Based on the infrared thermal imaging image and the visible light image, the three-dimensional spatial coordinates of the fire point are calculated using the binocular parallax principle. Based on the flame confidence level and the preset risk classification threshold, the thermal runaway warning level of the battery area is determined, and a corresponding control strategy is triggered in combination with the three-dimensional spatial coordinates. The control strategy includes at least one of triggering a local alarm, reporting warning information, performing precise power cut-off or locating and activating the fire extinguishing system based on the three-dimensional spatial coordinates.

[0006] Furthermore, the processing of the infrared thermal imaging image to extract the temperature field distribution features of the battery cluster includes: The infrared thermal imaging image is segmented based on pre-stored prior information about the battery cluster structure to locate the individual battery regions in the image. Within the located single-cell area, temperature field parameters, including at least the area of ​​the high-temperature region, temperature gradient, and temperature rise rate, are extracted as temperature field distribution characteristics.

[0007] Furthermore, the visible light image is processed to extract visual features of the flame, including: Candidate flame regions were identified from the visible light image using an optimized YOLOv5 deep learning model. Temporal analysis is performed on the candidate flame regions in multiple consecutive visible light images to extract the corresponding flicker frequencies; The image color, shape outline, and flicker frequency of the candidate flame region are used as visual features of the flame.

[0008] Furthermore, the multimodal fusion decision model is constructed based on DS evidence theory and includes: The identification framework definition module is used to define the hypothesis space, which includes the presence of flames, the absence of flames, and uncertainties. The basic probability assignment module is used to assign a first basic probability assignment to the temperature field distribution feature, assign a second basic probability assignment to the flame visual feature, and assign a third basic probability assignment to the spectral feature determined based on the spectral feature threshold; wherein, the first basic probability assignment, the second basic probability assignment, and the third basic probability assignment all include assignments for the presence of flame, the absence of flame, and uncertainty; The evidence synthesis module is used to perform fusion calculations on the first basic probability assignment, the second basic probability assignment, and the third basic probability assignment based on the Dempster synthesis rule.

[0009] Furthermore, the flame confidence level for the thermal runaway of the output battery includes: The evidence synthesis module performs a fusion calculation on the first basic probability assignment, the second basic probability assignment, and the third basic probability assignment to obtain a comprehensive basic probability assignment. The flame confidence level is obtained by extracting the values ​​assigned to the flame hypothesis from the comprehensive basic probability assignment. When there is a conflict in the basic probability assignments of different features, the evidence synthesis strategy is adjusted by calculating the conflict coefficient, which is used to quantify the degree of inconsistency between the basic probability assignments.

[0010] Furthermore, before calculating the three-dimensional spatial coordinates of the fire point using the binocular parallax principle, the method further includes: At least three visual markers with known three-dimensional coordinates are pre-set inside the battery compartment; Based on the binocular parallax principle, the three-dimensional coordinates of each visual marker point are calculated in real time, and the error set with the known coordinates is obtained. When the total drift of the error set exceeds a first preset threshold, it is determined that the camera parameters have drifted, and the calibration process is triggered. The calibration process minimizes the error set, reverse optimizes and solves for the updated intrinsic and extrinsic parameters of the dual-light fusion camera, and replaces the original parameters with the updated intrinsic and extrinsic parameters for subsequent calculation of the three-dimensional spatial coordinates of the fire point.

[0011] Furthermore, the calculation of the three-dimensional spatial coordinates of the fire point using the binocular parallax principle includes: When performing cross-modal feature point matching between the infrared thermal image and the visible light image, physical model constraints are set; the physical model constraints include: Spectral constraints prioritize matching feature points in high-temperature regions where the temperature exceeds a temperature threshold in the infrared thermal imaging image. Spatiotemporal consistency constraint: the motion trajectory of the feature point between consecutive frames must conform to the laws of physical motion; The initial disparity map is calculated based on feature point matching pairs that satisfy the constraints of the physical model.

[0012] Furthermore, after calculating the initial disparity map based on feature point matching pairs satisfying the physical model constraints, the method further includes: Let the initial 3D coordinates of a pixel in the fire point candidate region be... The corresponding infrared image temperature is The optimized final three-dimensional coordinates Calculated using the following formula: in, This represents the total number of pixels within the candidate fire point region. The temperature weighting function is... For the effective temperature threshold, The highest temperature within the candidate ignition point area. The weighting adjustment factor is greater than zero.

[0013] Furthermore, the step of determining the thermal runaway warning level of the battery region based on the flame confidence level and a preset risk grading threshold, and triggering a corresponding control strategy in conjunction with the three-dimensional spatial coordinates, includes: The flame confidence level is compared with a preset first-level threshold and a second-level threshold, wherein the second-level threshold is greater than the first-level threshold; If the flame confidence level is greater than or equal to the second-level threshold, it is determined to be a level two warning, indicating an open flame stage, and the highest level of emergency response plan is executed. The emergency response plan includes: sending a command to the battery management system to cut off the power supply to the corresponding battery cluster based on the three-dimensional spatial coordinates of the fire point, and simultaneously controlling the fire extinguishing device to spray at a fixed point towards the three-dimensional spatial coordinates; If the flame confidence level is greater than or equal to the first-level threshold but less than the second-level threshold, it is determined to be a level one warning, indicating a smoldering stage, and an early intervention plan is implemented; the early intervention plan includes: While triggering local audible and visual alarms and reporting early warning information, the three-dimensional spatial coordinates are matched with the pre-stored battery cluster layout model to lock the target battery cell, and the data sampling frequency and monitoring level of the target battery cell and its adjacent battery cells are increased.

[0014] A second aspect of the present invention provides a battery thermal runaway flame detection system based on machine vision, comprising: The image acquisition unit is used to acquire infrared thermal images and visible light images of the battery area from dual-light fusion cameras deployed at different spatial locations; The feature extraction unit is used to process the infrared thermal imaging image to extract the temperature field distribution features of the battery cluster; and to process the visible light image to extract the visual features of the flame. The flame confidence output unit is used to input the temperature field distribution features, the flame visual features, and the preset spectral feature thresholds into the multimodal fusion decision model. The multimodal fusion decision model is used to perform fusion calculations on multi-source features based on evidence theory and output the flame confidence of battery thermal runaway. The three-dimensional spatial coordinate calculation unit is used to calculate the three-dimensional spatial coordinates of the fire point based on the infrared thermal imaging image and the visible light image, using the principle of binocular parallax. The thermal runaway early warning level determination unit is used to determine the thermal runaway early warning level of the battery area based on the flame confidence level and the preset risk classification threshold, and to trigger a corresponding control strategy in combination with the three-dimensional spatial coordinates; the control strategy includes at least one of triggering a local alarm, reporting early warning information, performing precise power cut-off or locating and activating the fire extinguishing system based on the three-dimensional spatial coordinates.

[0015] As can be seen from the above technical solutions, the present invention has the following advantages: This invention utilizes dual-light fusion cameras deployed at different spatial locations to simultaneously acquire infrared thermal and visible light images of the battery area. Temperature field distribution features and flame visual features are extracted respectively, and combined with spectral feature thresholds, these are input into a multimodal fusion decision model based on evidence theory to calculate flame confidence. Simultaneously, the three-dimensional spatial coordinates of the fire point are calculated based on the principle of binocular parallax. Finally, the warning level is determined based on the confidence level and risk threshold, and a graded control strategy is triggered by combining the three-dimensional coordinates. This invention achieves the fusion and analysis of multi-source heterogeneous information, enabling the simultaneous determination of fire characteristics from multiple dimensions—physical temperature rise, chemical products, and visual morphology—in the early stages of thermal runaway, improving the timeliness and reliability of detection. Through the combination of three-dimensional spatial positioning and a graded warning mechanism, it overcomes the shortcomings of traditional single detection methods, such as slow response and insufficient positioning capabilities, effectively improving the accuracy and timeliness of energy storage system safety protection. Attached Figure Description

[0016] Figure 1 This is a schematic flowchart of an embodiment of a battery thermal runaway flame detection method based on machine vision in this invention. Detailed Implementation

[0017] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “corresponding to,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0018] Example 1 The implementation method in this embodiment can be implemented in a system, on a server, or on a terminal; no specific limitation is made. The method in this application will be described below from the perspective of system implementation. Please refer to... Figure 1The method provided in this application includes the following steps: S1. Acquire infrared thermal images and visible light images of the battery area from dual-light fusion cameras deployed at different spatial locations; In this embodiment, the dual-light fusion camera refers to a composite imaging device that integrates an uncooled vanadium oxide infrared detector and a complementary metal-oxide-semiconductor (CMOS) visible light sensor. The infrared detection unit is responsible for acquiring temperature distribution data of the battery area, generating an infrared thermal imaging image stored in matrix form, where the value of each pixel represents the temperature information at that location. The visible light unit is responsible for acquiring visual information of the battery area, generating a visible light image containing color, texture, and shape features. The two sensors ensure spatiotemporal consistency of data acquisition through hardware synchronization and are jointly installed at different spatial locations on the top or side wall of the battery compartment, forming a binocular vision architecture with a fixed baseline. This provides a synchronous multimodal image data foundation for subsequent temperature field analysis, flame identification, and 3D positioning.

[0019] S2. Process the infrared thermal imaging image to extract the temperature field distribution characteristics of the battery cluster; and process the visible light image to extract the visual features of the flame; This step extracts key features characterizing the battery's thermal runaway state from the acquired dual-light images. Temperature field distribution features refer to physical parameters, derived from infrared thermal imaging data, that reflect the spatial distribution and dynamic changes of abnormal battery heating; flame visual features refer to visual patterns and dynamic behaviors consistent with typical flame properties, identified from visible light images.

[0020] The extraction process of temperature field distribution characteristics specifically includes: 1. The infrared thermal imaging image is segmented based on the pre-stored prior information of the battery cluster structure in order to locate the individual battery regions in the image; 2. Within the located single-cell area, extract temperature field parameters, including at least the area of ​​the high-temperature region, temperature gradient, and temperature rise rate, as temperature field distribution characteristics.

[0021] Specifically, based on pre-stored prior information about the battery cluster structure, including the size, spacing, and overall layout of individual cells, a method combining Hough transform and template matching is used to segment the infrared thermal imaging image, defining the contour regions of each individual cell and effectively eliminating background interference. Within the located individual cell regions, connected high-temperature pixel regions are calculated and extracted, and their high-temperature areas are statistically analyzed. The temperature gradient of the region is calculated by solving for the rate of change between adjacent pixels in the temperature field. Furthermore, the temperature rise rate is calculated based on the temperature changes in the same region within a continuous frame sequence. These parameters collectively constitute the temperature field distribution characteristics used for subsequent decision-making.

[0022] The process of extracting visual features of flames specifically includes: 1. Identify candidate flame regions from visible light images using an optimized YOLOv5 deep learning model; 2. Perform temporal analysis on candidate flame regions in multiple consecutive visible light images to extract the corresponding flicker frequencies; 3. The image color, shape outline, and flicker frequency of the candidate flame region are used as visual features of the flame.

[0023] Specifically, visible light images are input into a YOLOv5 deep learning model optimized through transfer learning. This model has been trained on a dataset containing numerous energy storage compartment flame scenes and can quickly and accurately locate and select candidate flame regions in images. The system performs temporal analysis on the same candidate region in multiple consecutive frames, extracting the corresponding flicker frequency by calculating the periodic fluctuations of its brightness or red, green, and blue color components; typically, the flame flicker frequency is ≥2Hz. Finally, the system integrates the image color, shape contour, and calculated flicker frequency of the region to construct a vector representing the visual characteristics of the flame.

[0024] S3. Input the temperature field distribution characteristics, flame visual characteristics, and preset spectral feature thresholds into the multimodal fusion decision model. The multimodal fusion decision model is used to perform fusion calculations on multi-source features based on evidence theory and output the flame confidence level of battery thermal runaway. A multimodal fusion decision model is used to uniformly quantify and collaboratively analyze features extracted from different physical dimensions, addressing the uncertainty of single-feature discrimination. Ultimately, it outputs a comprehensive quantitative index reflecting the probability of battery thermal runaway flames, namely, the flame confidence score. The preset spectral feature threshold is a key criterion derived from statistical analysis of extensive battery thermal runaway experimental data. It is set as the concentration change corresponding to the characteristic spectral radiance of CO2 gas in the 4.26 μm band, typically set at 500 ppm·m. This threshold is used to determine the presence of specific chemical products generated by the thermal decomposition of battery materials, serving as a crucial basis for identifying the smoldering stage. The multimodal fusion decision model is a mathematical framework based on DS evidence theory, specifically designed to process multi-source information from different sensors with varying uncertainties. Through rigorous mathematical synthesis rules, it derives a comprehensive and quantifiable decision conclusion.

[0025] The specific architecture and implementation principle of the multimodal fusion decision model are as follows: 1. Identify the framework definition module, which is used to define the hypothesis space including the presence of flames, the absence of flames, and uncertainties; 2. The basic probability assignment module is used to assign a first basic probability assignment to the temperature field distribution features, a second basic probability assignment to the flame visual features, and a third basic probability assignment to the spectral features judged based on the spectral feature threshold; wherein, the first basic probability assignment, the second basic probability assignment, and the third basic probability assignment all include assignments for the presence of flame, the absence of flame, and uncertainty; 3. Evidence synthesis module, used to perform fusion calculation of the first basic probability assignment, the second basic probability assignment, and the third basic probability assignment based on Dempster synthesis rules.

[0026] First, the model's recognition framework definition module constructs a complete hypothesis space containing all possible propositions: the existence of flames, the non-existence of flames, and an uncertain state representing cognitive uncertainty. This provides a complete domain of discourse for subsequent probability assignment.

[0027] Secondly, the basic probability assignment module is responsible for converting various feature evidences into probabilistic forms that the model can process. This module assigns the first basic probability value to the temperature field distribution features. Specifically, the system comprehensively judges and assigns corresponding probability qualities to the three propositions of "existence of flame," "non-existence of flame," or "uncertainty" based on whether the extracted high-temperature region area exceeds a threshold, whether the temperature gradient is steep, and whether the temperature rise rate is drastic. For example, when a local high-temperature region is detected simultaneously and the temperature rise rate exceeds 2 degrees Celsius per second, a higher probability value is assigned to the proposition of "existence of flame." Similarly, this module assigns the second basic probability value to the visual features of flame, based on the matching degree of the color and shape of the candidate region identified by the YOLOv5 model with the standard flame, and whether the calculated flicker frequency is within the typical flame frequency range. Finally, this module assigns the third basic probability value to the spectral features, directly based on whether the detected CO2 spectral radiation intensity exceeds the aforementioned preset spectral feature threshold. Each basic probability assignment is a probability distribution, and their sum is 1, quantifying the degree of support of that path of evidence for the three propositions.

[0028] Finally, the evidence synthesis module uses Dempster's synthesis rule as its core algorithm. This module receives the pre-assigned first, second, and third basic probability assignments, treating them as three independent evidence bodies. The synthesis rule fuses all evidence by calculating the sum of the probability-quality products of the intersection of all propositions in these evidence bodies and performing normalization processing, ultimately outputting a comprehensive basic probability assignment that reflects the combined effect of all modal information.

[0029] In this embodiment, the output of the flame confidence level for battery thermal runaway includes the following steps: 1. The first basic probability assignment, the second basic probability assignment, and the third basic probability assignment are fused and calculated through the evidence synthesis module to obtain the comprehensive basic probability assignment; 2. Extract the values ​​for the hypothesis of flame existence from the comprehensive basic probability assignments as the flame confidence level; When there is a conflict in the basic probability assignments of different features, the evidence synthesis strategy is adjusted by calculating the conflict coefficient, which is used to quantify the degree of inconsistency between the basic probability assignments.

[0030] Specifically, the comprehensive assignment reflects the final probabilistic judgment based on the combined effect of all feature evidence. The flame confidence score is a value between 0 and 1, with a higher value indicating a greater likelihood of flame presence according to the system's comprehensive assessment. It is particularly important to note that during the entire fusion calculation process, when feature evidence from different modalities contradicts each other—that is, when the basic probability assignments of different features conflict—the system will initiate a conflict handling mechanism. This mechanism quantifies the degree of inconsistency between the evidence by calculating an index called the conflict coefficient. If the conflict coefficient indicates that the conflict is within an acceptable range, the evidence is synthesized according to standard rules. If the conflict is significant, the system will dynamically adjust the synthesis strategy, such as discounting modal evidence with relatively low reliability, to ensure the robustness and reliability of the fusion result and avoid overall decision-making errors due to momentary misjudgments from a single sensor.

[0031] S4. Based on infrared thermal imaging images and visible light images, calculate the three-dimensional spatial coordinates of the fire point using the principle of binocular parallax; This step utilizes the principle of binocular parallax, based on infrared thermal imaging and visible light images acquired by dual-light fusion cameras, to calculate the precise three-dimensional spatial coordinates of the fire point. The principle of binocular parallax involves using two cameras located at different spatial positions to image the same scene. By calculating the difference in pixel position (parallax) of the same physical point in the two images, and combining this with the camera's intrinsic and extrinsic parameters such as focal length and baseline distance, a geometric triangulation relationship is constructed, thereby solving for the three-dimensional coordinates of the point.

[0032] In this embodiment, to ensure the measurement accuracy of the binocular vision system during long-term operation, an online adaptive calibration mechanism is introduced, as follows: 1. Pre-set at least three visual markers with known three-dimensional coordinates inside the battery compartment; 2. Based on the principle of binocular parallax, calculate the three-dimensional coordinates of each visual marker point in real time and obtain the error set with the known coordinates; 3. When the total drift of the error set exceeds the first preset threshold, it is determined that the camera parameters have drifted and the calibration process is triggered. The calibration process minimizes the error set, reverse optimizes and solves the updated intrinsic and extrinsic parameters of the dual-light fusion camera, and replaces the original parameters with the updated intrinsic and extrinsic parameters for subsequent calculation of the three-dimensional spatial coordinates of the fire point.

[0033] Specifically, at least three visual markers with known precise 3D coordinates are deployed within the battery compartment. These markers employ high-contrast, specific patterns, such as QR codes or circular symbols, and their spatial positions are pre-determined and recorded into the system using precision measuring instruments during system installation. During system operation, these markers are continuously identified using current binocular images through feature extraction and matching techniques. Based on the principle of binocular parallax, the 3D coordinates of each marker in the camera coordinate system are calculated. The calculated coordinates are compared with pre-stored known coordinates to obtain an error set containing deviation values ​​for each coordinate axis. This error set can be characterized as the set of deviation components of each marker on each coordinate axis. When the comprehensive index of this error set, such as the sum of the squares of the deviations of all markers in each direction, exceeds a first preset threshold set based on the system's allowable positioning accuracy—the first preset threshold is determined according to the system's positioning requirements, for example, set to five millimeters—the system determines that the camera's internal and external parameters have drifted beyond the allowable range and triggers an automatic calibration process. The calibration process employs a nonlinear optimization algorithm, with the objective function being to minimize the overall error between the calculated coordinates and known coordinates of the marker points. Through iterative calculations, the optimal combination of camera intrinsic and extrinsic parameters is determined, including focal length, principal point coordinates, distortion coefficients, and the relative pose parameters between the two cameras. The updated parameters are immediately applied to all subsequent 3D coordinate calculations, effectively compensating for parameter drift caused by environmental temperature changes or mechanical vibrations, ensuring the long-term stability of fire point positioning accuracy.

[0034] In this embodiment, the three-dimensional spatial coordinates of the fire point are calculated using the binocular parallax principle, including the following: 1. When performing cross-modal feature point matching between infrared thermal imaging images and visible light images, physical model constraints are set. The physical model constraints include: spectral constraints, which prioritize matching feature points in high-temperature regions of infrared thermal imaging images where the temperature is higher than the temperature threshold; and spatiotemporal consistency constraints, which require that the motion trajectory of feature points between consecutive frames conforms to the laws of physical motion. 2. Calculate the initial disparity map based on feature point matching pairs that satisfy the physical model constraints.

[0035] First, when performing cross-modal feature point matching between infrared thermal imaging images and visible light images, physical model constraints are applied to improve the accuracy and robustness of the matching. These physical model constraints include two aspects: First, spectral constraints. The system sets an effective temperature threshold, determined by adding a safety margin to the upper limit of the battery's surface temperature during normal operation. In the infrared image, feature points are extracted only from areas with temperatures above this threshold, and feature matching is preferentially performed with corresponding spatial locations in the visible light image. This ensures that the matching process focuses on potential thermal anomaly areas. Second, spatiotemporal consistency constraints. The system tracks the motion trajectories of successfully matched feature points in consecutive video frames. By analyzing the displacement vectors of feature points between adjacent frames, the system verifies whether their motion conforms to the continuity and smoothness of physical motion, eliminating abnormal matching points with abrupt changes in motion trajectories or that do not conform to dynamic laws. Based on feature point matching pairs that simultaneously satisfy the above dual physical model constraints, the system calculates the disparity value of each matching pair, that is, the difference in horizontal coordinates of the corresponding pixels of the same physical point in the left and right camera images, and then generates an initial disparity map that reflects the scene depth information. The disparity map is a matrix of the same size as the original image, where each pixel value represents the disparity of that point.

[0036] In this embodiment, after obtaining the initial three-dimensional coordinates of each pixel within the candidate fire point region through the initial disparity map, the final fire point coordinates are further refined using a temperature field-weighted optimization model, as follows: Let the initial 3D coordinates of a pixel in the fire point candidate region be... The corresponding infrared image temperature is The optimized final three-dimensional coordinates Calculated using the following formula: in, This represents the total number of pixels within the candidate fire point region. The temperature weighting function is... For the effective temperature threshold, The highest temperature within the candidate ignition point area. The weighting adjustment factor is greater than zero.

[0037] Specifically, this optimization model is based on the physical fact that the core region of a flame has the highest temperature and is most likely to represent the actual fire source location. The design principle of the temperature weighting function is that when the temperature of a pixel is lower than the effective temperature threshold, the pixel is considered not to belong to the effective fire point region and is assigned zero weight; when the temperature is not lower than the effective temperature threshold, the weight increases exponentially with the temperature, the growth rate is controlled by a weight adjustment factor greater than zero, and normalization is performed based on the highest temperature in the region. This exponential weighting ensures that the coordinates of high-temperature pixels contribute more to the final result, thus making the optimized coordinates more inclined to the higher-temperature core region. Through this optimization process, coordinate deviations caused by parallax calculation errors, image noise, or edge blurring can be effectively suppressed, outputting a high-precision three-dimensional spatial coordinate that is closer to the actual fire source center.

[0038] S5. Based on the flame confidence level and the preset risk classification threshold, determine the thermal runaway early warning level of the battery area, and trigger the corresponding control strategy in combination with the three-dimensional spatial coordinates; the control strategy includes triggering a local alarm, reporting early warning information, performing precise power cut-off or locating and activating the fire extinguishing system based on the three-dimensional spatial coordinates, at least one of the following.

[0039] The purpose of this step is to transform the quantitative indicator of flame confidence obtained from the aforementioned analysis into a clear risk level judgment and drive corresponding control actions. The preset risk classification thresholds here are critical values ​​derived from statistical analysis of a large amount of battery thermal runaway experimental data, used to classify risk stages of different severity. Specifically, the first-level threshold is used to identify early smoldering signs of thermal runaway, while the second-level threshold is used to determine the emergency state that has developed into an open flame. The value of the second-level threshold is set higher than that of the first-level threshold to ensure clear classification of alarm levels and accurate triggering of response strategies.

[0040] 1. Compare the flame confidence score with a preset first-level threshold and a second-level threshold, wherein the second-level threshold is greater than the first-level threshold; 2. If the flame confidence level is greater than or equal to the second-level threshold, it is determined to be a level two warning, indicating an open flame stage, and the highest level emergency response plan is executed. The emergency response plan includes: based on the three-dimensional spatial coordinates of the fire point, sending instructions to the battery management system to cut off the power to the corresponding battery cluster, and simultaneously controlling the fire extinguishing device to spray at a fixed point in the three-dimensional spatial coordinates. 3. If the flame confidence level is greater than or equal to the first-level threshold but less than the second-level threshold, it is determined to be a first-level warning, indicating a smoldering stage, and an early intervention plan is implemented. The early intervention plan includes: while triggering the local audible and visual alarm and reporting the warning information, matching the three-dimensional spatial coordinates with the pre-stored battery cluster layout model to lock the target battery cell, and increasing the data sampling frequency and monitoring level of the target battery cell and its adjacent battery cells.

[0041] First, the system compares the calculated flame confidence score with preset first-level and second-level thresholds in real time. The first-level threshold is set to a relatively low value to sensitively capture potential risks; the second-level threshold is set to a higher value to confirm the occurrence of high-risk events.

[0042] Secondly, if the system determines that the flame confidence level is greater than or equal to the second-level threshold, it immediately issues a second-level warning, confirming that the battery area has entered the open flame stage. At this point, the system will execute the highest-level emergency response plan. The core of this plan is based on the high-precision three-dimensional spatial coordinates calculated in the aforementioned steps. On one hand, the system sends instructions to the battery management system, requesting it to cut off the power supply to the specific battery cluster where the fire point is located, achieving precise power cut-off and preventing the accident from escalating; on the other hand, it sends the three-dimensional coordinates of the fire point to the fire extinguishing control unit, driving the fire extinguishing device to adjust the spray angle and spray at the target coordinates for precise point spraying, suppressing the open flame as quickly as possible.

[0043] Finally, if the system determines that the flame confidence level is greater than or equal to the first-level threshold but less than the second-level threshold, it is classified as a first-level warning, indicating the early smoldering stage of thermal runaway. At this point, the system executes an early intervention plan centered on early warning and precise monitoring. This plan includes immediately triggering local audible and visual alarms for on-site warning, while simultaneously uploading the warning information and key data to the remote monitoring center. More importantly, the system quickly matches the three-dimensional spatial coordinates of the fire point with a pre-stored digital layout model of the battery cluster, thereby accurately locating the target battery cell exhibiting the anomaly. Based on this, the system instructs the data acquisition unit to increase the data sampling frequency of the target battery cell and its adjacent battery cells, and to increase its display and alarm priority on the monitoring interface, thereby enhancing monitoring of the potential hazard area, providing maintenance personnel with precise targets for handling, and buying time for possible escalation responses.

[0044] Example 2 An embodiment of the battery thermal runaway flame detection system based on machine vision of the present invention includes: The image acquisition unit is used to acquire infrared thermal images and visible light images of the battery area from dual-light fusion cameras deployed at different spatial locations; The feature extraction unit is used to process infrared thermal imaging images to extract the temperature field distribution features of the battery clusters; and to process visible light images to extract the visual features of the flames. The flame confidence output unit is used to input the temperature field distribution characteristics, flame visual characteristics and preset spectral feature thresholds into the multimodal fusion decision model. The multimodal fusion decision model is used to perform fusion calculation of multi-source features based on evidence theory and output the flame confidence of battery thermal runaway. The three-dimensional spatial coordinate calculation unit is used to calculate the three-dimensional spatial coordinates of the fire point based on infrared thermal imaging images and visible light images, using the principle of binocular parallax. The thermal runaway early warning level determination unit is used to determine the thermal runaway early warning level of the battery area based on the flame confidence level and the preset risk classification threshold, and to trigger the corresponding control strategy in combination with the three-dimensional spatial coordinates; the control strategy includes triggering a local alarm, reporting early warning information, performing precise power cut-off or locating and activating the fire extinguishing system based on the three-dimensional spatial coordinates, at least one of the following.

[0045] For specific limitations regarding the system, please refer to the method limitations described above, which will not be repeated here. Each module in the above system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0046] It is understood that those skilled in the art can combine various implementation methods in the above embodiments under the guidance of the above examples to obtain technical solutions with multiple implementation methods.

[0047] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A machine vision-based method for detecting battery thermal runaway flames, characterized in that, include: Acquire infrared thermal images and visible light images of the battery area from dual-light fusion cameras deployed at different spatial locations; The infrared thermal imaging image is processed to extract the temperature field distribution characteristics of the battery cluster; and the visible light image is processed to extract the visual characteristics of the flame. The temperature field distribution features, the flame visual features, and the preset spectral feature thresholds are input into the multimodal fusion decision model. The multimodal fusion decision model is used to perform fusion calculations on multi-source features based on evidence theory and output the flame confidence level of battery thermal runaway. Based on the infrared thermal imaging image and the visible light image, the three-dimensional spatial coordinates of the fire point are calculated using the binocular parallax principle. Based on the flame confidence level and the preset risk classification threshold, the thermal runaway warning level of the battery area is determined, and a corresponding control strategy is triggered in combination with the three-dimensional spatial coordinates. The control strategy includes at least one of triggering a local alarm, reporting warning information, performing precise power cut-off or locating and activating the fire extinguishing system based on the three-dimensional spatial coordinates.

2. The battery thermal runaway flame detection method based on machine vision according to claim 1, characterized in that, The process of processing the infrared thermal imaging image to extract the temperature field distribution features of the battery cluster includes: The infrared thermal imaging image is segmented based on pre-stored prior information about the battery cluster structure to locate the individual battery regions in the image. Within the located single-cell area, temperature field parameters, including at least the area of ​​the high-temperature region, temperature gradient, and temperature rise rate, are extracted as temperature field distribution characteristics.

3. The battery thermal runaway flame detection method based on machine vision according to claim 1, characterized in that, The process of processing the visible light image to extract visual features of the flame includes: Candidate flame regions were identified from the visible light image using an optimized YOLOv5 deep learning model. Temporal analysis is performed on the candidate flame regions in multiple consecutive visible light images to extract the corresponding flicker frequencies; The image color, shape outline, and flicker frequency of the candidate flame region are used as visual features of the flame.

4. The battery thermal runaway flame detection method based on machine vision according to claim 1, characterized in that, The multimodal fusion decision model is constructed based on DS evidence theory and includes: The identification framework definition module is used to define the hypothesis space, which includes the presence of flames, the absence of flames, and uncertainties. The basic probability assignment module is used to assign a first basic probability assignment to the temperature field distribution feature, assign a second basic probability assignment to the flame visual feature, and assign a third basic probability assignment to the spectral feature determined based on the spectral feature threshold; wherein, the first basic probability assignment, the second basic probability assignment, and the third basic probability assignment all include assignments for the presence of flame, the absence of flame, and uncertainty; The evidence synthesis module is used to perform fusion calculations on the first basic probability assignment, the second basic probability assignment, and the third basic probability assignment based on the Dempster synthesis rule.

5. The battery thermal runaway flame detection method based on machine vision according to claim 4, characterized in that, The flame confidence level for the thermal runaway of the output battery includes: The evidence synthesis module performs a fusion calculation on the first basic probability assignment, the second basic probability assignment, and the third basic probability assignment to obtain a comprehensive basic probability assignment. The flame confidence level is obtained by extracting the values ​​assigned to the flame hypothesis from the comprehensive basic probability assignment. When there is a conflict in the basic probability assignments of different features, the evidence synthesis strategy is adjusted by calculating the conflict coefficient, which is used to quantify the degree of inconsistency between the basic probability assignments.

6. The battery thermal runaway flame detection method based on machine vision according to claim 1, characterized in that, Before calculating the three-dimensional spatial coordinates of the fire point using the binocular parallax principle, the method further includes: At least three visual markers with known three-dimensional coordinates are pre-set inside the battery compartment; Based on the binocular parallax principle, the three-dimensional coordinates of each visual marker point are calculated in real time, and the error set with the known coordinates is obtained. When the total drift of the error set exceeds a first preset threshold, it is determined that the camera parameters have drifted, and the calibration process is triggered. The calibration process minimizes the error set, reverse optimizes and solves for the updated intrinsic and extrinsic parameters of the dual-light fusion camera, and replaces the original parameters with the updated intrinsic and extrinsic parameters for subsequent calculation of the three-dimensional spatial coordinates of the fire point.

7. The battery thermal runaway flame detection method based on machine vision according to claim 6, characterized in that, The calculation of the three-dimensional spatial coordinates of the fire point using the binocular parallax principle includes: When performing cross-modal feature point matching between the infrared thermal image and the visible light image, physical model constraints are set; the physical model constraints include: Spectral constraints prioritize matching feature points in high-temperature regions where the temperature exceeds a temperature threshold in the infrared thermal imaging image. Spatiotemporal consistency constraint: the motion trajectory of the feature point between consecutive frames must conform to the laws of physical motion; The initial disparity map is calculated based on feature point matching pairs that satisfy the constraints of the physical model.

8. The battery thermal runaway flame detection method based on machine vision according to claim 7, characterized in that, After calculating the initial disparity map based on feature point matching pairs that satisfy the physical model constraints, the method further includes: Let the initial 3D coordinates of a pixel in the candidate fire point region be... The corresponding infrared image temperature is The optimized final three-dimensional coordinates Calculated using the following formula: in, This represents the total number of pixels within the candidate fire point region. The temperature weighting function is... For the effective temperature threshold, The highest temperature within the candidate ignition point area. The weighting adjustment factor is greater than zero.

9. The battery thermal runaway flame detection method based on machine vision according to claim 1, characterized in that, The step involves determining the thermal runaway warning level of the battery region based on the flame confidence level and a preset risk classification threshold, and triggering a corresponding control strategy in conjunction with the three-dimensional spatial coordinates, including: The flame confidence level is compared with a preset first-level threshold and a second-level threshold, wherein the second-level threshold is greater than the first-level threshold; If the flame confidence level is greater than or equal to the second-level threshold, it is determined to be a level two warning, indicating an open flame stage, and the highest level of emergency response plan is executed. The emergency response plan includes: sending a command to the battery management system to cut off the power supply to the corresponding battery cluster based on the three-dimensional spatial coordinates of the fire point, and simultaneously controlling the fire extinguishing device to spray at a fixed point towards the three-dimensional spatial coordinates; If the flame confidence level is greater than or equal to the first-level threshold but less than the second-level threshold, it is determined to be a level one warning, indicating a smoldering stage, and an early intervention plan is implemented; the early intervention plan includes: While triggering local audible and visual alarms and reporting early warning information, the three-dimensional spatial coordinates are matched with the pre-stored battery cluster layout model to lock the target battery cell, and the data sampling frequency and monitoring level of the target battery cell and its adjacent battery cells are increased.

10. A battery thermal runaway flame detection system based on machine vision, characterized in that, The method according to any one of claims 1 to 9 comprises: The image acquisition unit is used to acquire infrared thermal images and visible light images of the battery area from dual-light fusion cameras deployed at different spatial locations; The feature extraction unit is used to process the infrared thermal imaging image to extract the temperature field distribution features of the battery cluster; and to process the visible light image to extract the visual features of the flame. The flame confidence output unit is used to input the temperature field distribution features, the flame visual features, and the preset spectral feature thresholds into the multimodal fusion decision model. The multimodal fusion decision model is used to perform fusion calculations on multi-source features based on evidence theory and output the flame confidence of battery thermal runaway. The three-dimensional spatial coordinate calculation unit is used to calculate the three-dimensional spatial coordinates of the fire point based on the infrared thermal imaging image and the visible light image, using the principle of binocular parallax. The thermal runaway early warning level determination unit is used to determine the thermal runaway early warning level of the battery area based on the flame confidence level and the preset risk classification threshold, and to trigger a corresponding control strategy in combination with the three-dimensional spatial coordinates; the control strategy includes at least one of triggering a local alarm, reporting early warning information, performing precise power cut-off or locating and activating the fire extinguishing system based on the three-dimensional spatial coordinates.

Images