Computer vision-based energy storage fire early identification method and system

Abstract

The application discloses a kind of energy storage fire early identification method and system based on computer vision, obtain the visible light image sequence, infrared thermal image sequence and near-infrared micro smoke enhancement image sequence of the same target monitoring area of energy storage cabin, and according to battery cluster, busbar, cable joint, fire-fighting nozzle, vent and cabinet door observation window generate area semantic graph;Time alignment, field of view registration and semantic area mapping are carried out using area semantic graph, and multiple modal area image sequence is obtained;Region graph state vector is constructed and matched with fire precursor event chain template, and event chain matching strength is obtained;Through graph causal decoupling model, thermal runaway precursor strength, smoke precursor strength, arc light precursor strength, interference counterproof strength, graph propagation consistency strength and propagation consistency residual are output, and the early fire risk value of region is calculated, and when meeting double-threshold hysteresis grading early warning condition, abnormal position, fire early grade and linkage disposal instruction are output.The application can improve the accuracy of energy storage fire early identification and reduce false alarm.

Description

Technical Field

[0001] This invention relates to the fields of energy storage safety monitoring, computer vision, and early fire warning technology, and in particular to a method and system for early identification of energy storage fires based on computer vision. Background Technology

[0002] With the widespread application of electrochemical energy storage power stations, energy storage containers, and energy storage compartments, the safe operation of energy storage systems has become increasingly prominent. Energy storage compartments typically house multiple battery clusters, busbars, cable connectors, fire sprinklers, ventilation openings, and observation windows on doors. If abnormal temperature rises, insulation deteriorates, partial discharge occurs, or thermal runaway precursors appear in individual battery cells, battery cluster connections, or cable connectors, it can quickly trigger smoke, arcing, heat spread, or even a fire. Therefore, identifying early signs of energy storage fires before open flames form is crucial for ensuring the safety of energy storage systems.

[0003] Current methods for monitoring fires in energy storage systems primarily employ temperature sensors, smoke detectors, gas detectors, or conventional video recognition. Temperature sensors and smoke detectors typically require an anomaly to reach a certain level before triggering an alarm, which can lead to response delays. While gas detectors can detect some thermal runaway releases, their placement, airflow path, and sampling delay can affect detection accuracy. Conventional video recognition methods often rely on changes in flames, dense smoke, or brightness, making it difficult to accurately identify early signs of thermal runaway such as low-concentration smoke, localized hot spot expansion, arc flashes, and weak diffusion trends.

[0004] Furthermore, the presence of numerous metal casings, cabinet doors and observation windows, inspection lights, fans, personnel inspections, and door opening and closing within the energy storage compartment can easily generate reflections, obstructions, vibrations, sudden changes in local brightness, or smoke-like disturbances. Existing visual recognition methods typically treat these disturbances as ordinary negative samples, lacking recognition constraints specific to the structural relationships within the energy storage compartment and the mechanisms of fire precursor propagation, resulting in a high false alarm rate. On the other hand, even when existing multimodal fire identification methods use both visible light and infrared images simultaneously, they often remain at the image-level feature fusion level, failing to consider spatial adjacency, cable connections, airflow propagation, heat diffusion, and visual obstruction relationships between battery clusters, busbars, cable joints, vents, fire nozzles, and observation windows as criteria for judgment.

[0005] Therefore, it is necessary to propose an early identification method and system for energy storage fires that can combine semantic relationships of energy storage compartment structure, multimodal visual precursors, event chain temporal evolution, interference counterfactual verification, and graph propagation consistency judgment, so as to more accurately distinguish between real fire precursors and non-fire interferences before the fire forms, and improve the accuracy and stability of early warning. Summary of the Invention

[0006] The purpose of this invention is to provide a computer vision-based method and system for early identification of energy storage fires, which solves the problems of existing energy storage fire identification methods, such as lagging identification of early signs of thermal runaway, unstable identification of low-concentration smoke and arc flash, high false alarm rate of non-fire interference such as reflective obstruction, and lack of constraints on the propagation relationship of energy storage compartment structure.

[0007] This invention establishes a semantic graph of the energy storage compartment area, mapping structural components such as battery clusters, busbars, cable connectors, fire nozzles, ventilation openings, and cabinet door observation windows as semantic nodes, and mapping spatial adjacency, cable connections, airflow propagation, heat diffusion, and visual obstruction relationships as directed semantic edges. Based on this, semantic region mapping is performed on visible light image sequences, infrared thermal image sequences, and near-infrared micro-smoke enhanced image sequences to construct a regional graph state vector. Combined with a fire precursor event chain template, a graph causal decoupling model, counterfactual interference verification, and graph propagation residual calculation, an early fire risk value for the region is obtained, thereby achieving early, stable, and low false alarm identification and coordinated response to energy storage fires.

[0008] To achieve the aforementioned objective, this invention provides a computer vision-based method for early identification of energy storage fires, comprising: S1, acquiring visible light image sequences, infrared thermal image sequences, and near-infrared smoke enhancement image sequences of the same target monitoring area in the energy storage compartment, and generating a region semantic map based on battery clusters, busbars, cable connectors, fire nozzles, ventilation openings, and cabinet door observation windows; S2, using the region semantic map to perform time alignment, field-of-view registration, and semantic region mapping on the visible light image sequences, infrared thermal image sequences, and near-infrared smoke enhancement image sequences to obtain a multimodal region image sequence; S3, constructing a region map state vector based on the multimodal region image sequence, and assigning the region map state vector to the region map state vector. S4. The vector is matched with a preset fire precursor event chain template to obtain the event chain matching strength; S5. The regional graph state vector is input into the graph causal decoupling model to output the thermal runaway precursor strength, smoke precursor strength, arc flash precursor strength, interference counter-evidence strength, graph propagation consistency strength, and propagation consistency residual; S6. The regional early fire risk value is calculated based on the event chain matching strength, thermal runaway precursor strength, smoke precursor strength, arc flash precursor strength, interference counter-evidence strength, graph propagation consistency strength, and propagation consistency residual; S7. When the regional early fire risk value meets the dual-threshold hysteresis graded early warning condition, the abnormal location, early fire level, and linkage response instruction of the corresponding semantic region are output.

[0009] To achieve the aforementioned objective, this invention provides a computer vision-based early fire identification system for energy storage, comprising: an image acquisition module for acquiring visible light image sequences, infrared thermal image sequences, and near-infrared smoke enhancement image sequences of the same target monitoring area in an energy storage compartment; a region semantic modeling module for generating a region semantic map based on battery clusters, busbars, cable connectors, fire nozzles, ventilation openings, and cabinet door observation windows; a multimodal region mapping module for performing time alignment, field-of-view registration, and semantic region mapping on the visible light image sequences, infrared thermal image sequences, and near-infrared smoke enhancement image sequences using the region semantic map to obtain a multimodal region image sequence; a region state construction module for constructing a region map state vector based on the multimodal region image sequence; and an event chain matching module. The system comprises the following modules: a module for matching the regional graph state vector with a preset fire precursor event chain template to obtain the event chain matching strength; a graph causal decoupling module for inputting the regional graph state vector into the graph causal decoupling model and outputting the intensity of thermal runaway precursors, smoke precursors, arc flash precursors, interference counter-evidence, graph propagation consistency, and propagation consistency residuals; a risk calculation module for calculating the early fire risk value of the region based on the event chain matching strength, thermal runaway precursor strength, smoke precursor strength, arc flash precursor strength, interference counter-evidence, graph propagation consistency, and propagation consistency residuals; and an early warning linkage module for outputting the abnormal location, early fire level, and linkage response instructions of the corresponding semantic region when the early fire risk value of the region meets the dual-threshold hysteresis graded early warning conditions.

[0010] Compared with the prior art, the present invention has at least the following beneficial effects:

[0011] First, this invention does not simply rely on smoke, flames, or temperature anomalies in images for alarm triggering. Instead, it generates a semantic map of the area based on battery clusters, busbars, cable connectors, fire nozzles, ventilation openings, and cabinet door observation windows, thus constraining the early fire identification process to the actual structural relationships of the energy storage compartment. By using spatial adjacency, cable connections, airflow propagation, heat diffusion, and visual obstruction relationships as the basis for subsequent registration, event chain matching, and propagation consistency judgment, the accuracy of anomaly area location can be improved.

[0012] Second, this invention maps visible light image sequences, infrared thermal image sequences, and near-infrared smoke enhancement image sequences to the same semantic region coordinate system, enabling local temperature rise, low-concentration smoke, arc flash, and visual interference to be coupled and analyzed within the same semantic region, thus avoiding misjudgments caused by field deviation, time deviation, or regional correspondence errors between different modal images.

[0013] Third, this invention matches the state vector of the region map with a preset fire precursor event chain template, enabling the identification of early fire evolution relationships such as thermal anomaly leaders, follow-up smoke diffusion, arcing, and regional propagation. Compared to single-frame identification or single-threshold alarms, this invention can utilize the temporal evolution characteristics within consecutive frames to determine whether anomalies conform to the early development patterns of energy storage fires, thereby improving the reliability of early identification.

[0014] Fourth, this invention employs a graph-causal decoupling model to output the intensity of thermal runaway precursors, smoke precursors, arc flash precursors, interference counter-evidence, graph propagation consistency, and propagation consistency residuals, thereby separating fire precursor features from non-fire interference features during the identification process. This reduces false alarms caused by non-fire factors such as inspection light reflections, cabinet door observation window reflections, personnel obstruction, camera shake, door opening and closing, and fan disturbances.

[0015] Fifth, this invention utilizes the concept of counterfactual interference verification. Suspected reflective areas, obstructed areas, shaking areas, or reflection areas from cabinet door observation windows are masked, replaced, or treated equivalently. The changes in the precursory responses before and after the treatment are then compared to obtain the strength of the counterfactual interference. This method does not simply identify the type of interference, but rather determines whether the current anomaly can be explained by interference, thereby further improving the ability to suppress false alarms.

[0016] Sixth, this invention introduces graph propagation consistency strength and propagation consistency residual. By judging whether precursor anomalies form reasonable propagation along directed semantic edges in the regional semantic graph, it distinguishes between real fire precursors and isolated reflections, transient obstructions, or image noise. For anomalies that do not conform to the laws of spatial propagation, cable propagation, airflow propagation, or heat diffusion, this invention can reduce the regional early fire risk value.

[0017] Seventh, this invention calculates the early fire risk value of a region based on the event chain matching strength, thermal runaway precursor strength, smoke precursor strength, arc flash precursor strength, interference counter-evidence strength, graph propagation consistency strength, and propagation consistency residual. This allows risk assessment to simultaneously consider precursor enhancement, event chain rationality, propagation rationality, and interference interpretability, avoiding instability issues caused by alarms triggered by a single feature.

[0018] Eighth, the present invention adopts a dual-threshold hysteresis graded early warning condition to output the early fire level and linkage response instructions, which can avoid the risk value fluctuating near the threshold, causing frequent alarms, alarm cancellations or level jumps, making the early warning output of energy storage fires more stable, and facilitating linkage with response measures such as ventilation, power outage, fire spraying and operation and maintenance alarms. Attached Figure Description

[0019] Figure 1 A flowchart of a computer vision-based early identification method for energy storage fires provided in the first embodiment of the present invention.

[0020] Figure 2 This is a block diagram of a computer vision-based early fire identification system for energy storage provided in the second embodiment of the present invention. Detailed Implementation

[0021] The technical solution of the present invention will be described in detail below with reference to embodiments. This embodiment uses an energy storage compartment as the application object, which contains structural components such as battery clusters, busbars, cable connectors, fire nozzles, ventilation openings, and cabinet door observation windows. Multimodal image sequences of the same target monitoring area are acquired through visible light imaging, infrared thermal imaging, and near-infrared micro-smoke enhancement imaging. The spatial adjacency relationships, cable connection relationships, airflow propagation relationships, heat diffusion relationships, and visual obstruction relationships among the structural components within the energy storage compartment are constructed into a regional semantic map. Based on this regional semantic map, semantic region mapping is performed on the multimodal image sequences to construct a regional graph state vector. Through event chain matching, graph causal decoupling, counterfactual interference verification, graph propagation consistency judgment, and dual-threshold hysteresis graded early warning, the early precursors of energy storage fires can be identified, located, and coordinated for response.

[0022] In this embodiment, a multimodal sensor health self-verification, adaptive update of regional semantic graph edge weights, precursor credibility backtracking, low-sample domain adaptation, and linkage response feedback correction mechanism can be further introduced. This enables the identification process to not only determine whether the current image anomaly belongs to an early precursor of a fire, but also whether the image source is reliable, whether the propagation relationship changes with the operating status of the energy storage compartment, and whether the alarm result is verified by the subsequent response effect. Thus, the early identification process of energy storage fires forms a closed-loop data processing chain of "image acquisition—semantic mapping—regional mapping—state construction—event chain matching—causal decoupling—counterfactual verification—propagation residual judgment—risk calculation—hysteresis warning—response feedback correction".

[0023] First Embodiment

[0024] like Figure 1 As shown, the first embodiment provides a computer vision-based method for early identification of energy storage fires, including the following steps.

[0025] S1: Acquire visible light image sequence, infrared thermal image sequence, and near-infrared micro-smoke enhanced image sequence of the same target monitoring area of ​​the energy storage compartment, and generate a semantic map of the area based on battery clusters, busbars, cable joints, fire nozzles, ventilation openings, and cabinet door observation windows.

[0026] In this step, the target monitoring area can be the interior of an energy storage compartment, the interior of a battery cabinet, the area containing a battery cluster, the area containing a busbar, or a concentrated area of ​​cable connectors. Visible light image sequences are used to acquire visual information such as the structural outline of the energy storage compartment, abrupt changes in brightness, personnel obstruction, cabinet door opening and closing, reflections from the cabinet door observation window, and arc flashes. Infrared thermal image sequences are used to acquire temperature distribution, hot spot centers, hot spot boundaries, and temperature rise trends in the battery cluster, busbar, and cable connector areas. Near-infrared smoke enhancement image sequences are used to acquire low-concentration smoke boundaries, sparse smoke particles, and smoke drift direction. Near-infrared smoke enhancement image sequences can be acquired by a near-infrared camera, near-infrared illumination components, and narrowband filter components, or they can be acquired by an imaging device with a near-infrared channel and then obtained through image enhancement algorithms.

[0027] Specifically, in the first At each frame, a visible light image is acquired. Infrared thermal images Near-infrared smoke enhancement images A visible light image sequence is formed in chronological order. Infrared thermal image sequence Near-infrared smoke enhancement image sequence .in, Indicates the first Visible light image of a frame, Indicates the first Infrared thermal image of the frame, Indicates the first Near-infrared smoke enhancement image of the frame. Indicates the image frame number.

[0028] To avoid misjudgments caused by anomalies in a single imaging channel, this step also performs sensor health self-verification on the visible light image sequence, infrared thermal image sequence, and near-infrared smoke-enhanced image sequence. Sensor health self-verification includes assessing image sharpness, exposure stability, thermal imaging drift, near-infrared illumination uniformity, and timestamp continuity to obtain multimodal health weights. These multimodal health weights are used to weight the credibility of different image modalities during subsequent multimodal region mapping and risk calculation.

[0029] Multimodal health weights can be expressed as follows:

[0030]

[0031] in, Indicates the first The image modality in the th... Multimodal health weights of frames. This represents the health weight mapping function. Indicates the first The image modality in the th... Image clarity of the frame Indicates the first The image modality in the th... Frame exposure or thermal response stability, Indicates the first The image modality in the th... Frame fill light or background uniformity, Indicates the first The image modality in the th... The time deviation between the frame and the reference timestamp Indicates the image modality number. Indicates the image frame number. Image modality number. It can correspond to visible light image mode, infrared thermal image mode, and near-infrared smoke enhancement image mode. (The same letters are used.) In this embodiment, all letters represent image frame numbers, and different letters represent different meanings.

[0032] A semantic map of the area is generated based on the battery clusters, busbars, cable connectors, fire sprinklers, ventilation openings, and observation windows of the cabinet doors within the energy storage compartment. .in, Represents a semantic graph of regions. Represents a set of semantic nodes. This represents the set of directed semantic edges. The set of semantic nodes. This includes nodes for battery clusters, busbars, cable connectors, fire sprinklers, ventilation openings, and cabinet door observation windows. It is a set of directed semantic edges. This includes spatial adjacency edges, cable connection edges, airflow propagation edges, heat diffusion edges, and visible obstruction edges.

[0033] Spatial adjacency edges are used to indicate that two semantic regions are physically adjacent in the energy storage compartment; cable connection edges are used to indicate the electrical connection path between battery clusters, busbars, and cable connectors; airflow propagation edges are used to indicate the path that smoke, hot airflow, or particulate matter may propagate along the ventilation direction; heat diffusion edges are used to indicate the direction in which local temperature rise may spread along structural components or adjacent spaces; and visual obstruction edges are used to indicate the obstruction relationship between the cabinet door observation window, personnel passing by, the opening and closing of the compartment door, or equipment obstruction on the camera's field of view.

[0034] Furthermore, the directed semantic edges in the regional semantic graph can be assigned initial edge weights and adaptively updated based on the wind turbine operating status, door opening and closing status, charging and discharging conditions, and historical precursor propagation records of the energy storage compartment. This adaptive edge weight update allows the airflow propagation edges and heat diffusion edges to be adjusted according to actual operating conditions, thereby improving the accuracy of propagation consistency judgment.

[0035] The edge weights of a region semantic graph can be represented as:

[0036]

[0037] in, Indicates the first The semantic region to the first The semantic region in the first Frame edge weights Indicates the first The semantic region to the first Initial edge weights for each semantic region Indicates the first The semantic region up to the first The semantic region in the first The amount of influence of wind direction or airflow on the frame. Indicates the first The semantic region up to the first The semantic region in the first The impact of the frame's door opening / closing or passageway connectivity. Indicates the first The semantic region up to the first The semantic region in the first The reliability of the historical propagation of frames , , These represent the updated weights for the influence of wind direction or airflow, the influence of door opening or closing or passage connectivity, and the historical propagation reliability, respectively. Indicates the semantic region number. Indicates the number of adjacent semantic regions. Indicates the image frame number. Same letters. Both represent edge weights, and different subscripts and superscripts are used to distinguish different semantic regions and different frame times.

[0038] In this step, the region semantic map Edge weight and multimodal health weights The output is used for subsequent multimodal region mapping in S2, region graph state vector construction in S3, graph causal decoupling model in S4, and early regional fire risk value calculation in S5. Thus, the region semantic graph not only describes the spatial structure of the energy storage compartment but also serves as a common constraint for multimodal image registration, event chain matching, graph propagation consistency judgment, and interference rebuttal calculation.

[0039] This step transforms the structural objects within the energy storage compartment, propagation paths, image modal health status, and changes in operating conditions into a computable data structure. This allows subsequent identification processes to no longer rely solely on image pixel features, but are simultaneously constrained by the actual structural relationships within the energy storage compartment and the reliability of the image source. This can improve the accuracy of anomaly location positioning and reduce misjudgments caused by image modal failure, abnormal supplementary lighting, thermal imaging drift, or fixed edge weights not adapting to operating conditions.

[0040] S2, using the region semantic map, perform time alignment, field registration, and semantic region mapping on the visible light image sequence, infrared thermal image sequence, and near-infrared smoke enhancement image sequence to obtain a multimodal region image sequence.

[0041] In this step, the visible light image sequence, infrared thermal image sequence, and near-infrared smoke enhancement image sequence are uniformly mapped to a region semantic map. The defined semantic region coordinate system. Since different imaging devices may have different installation locations, field of view, resolutions, and sampling frequencies, directly fusing the three types of images can easily lead to region correspondence errors. Therefore, this step first performs temporal alignment, then field of view registration, and finally semantic region mapping.

[0042] Time alignment includes aligning visible light images according to the image acquisition timestamp. Infrared thermal images Near-infrared smoke enhancement images Align to the same frame moment When different image sequences have different sampling frequencies, neighboring frame matching, linear interpolation, or frame buffer synchronization can be used to ensure that the three types of images at the same time correspond to the same target monitoring area. For image frames with timestamp deviations exceeding a preset deviation threshold, the multimodal health weight of the corresponding image modality can be reduced. Alternatively, the frame can be treated as a low-confidence frame and used in subsequent calculations.

[0043] Field registration includes using region semantic maps The boundaries of the battery clusters, cabinet, ventilation openings, and cabinet door observation windows are used as fixed structural constraints to map the hot spot centers and boundaries in the infrared thermal image sequence, the smoke boundaries and sparse smoke particles in the near-infrared micro-smoke enhanced image sequence, and the structural contours and brightness abrupt change regions in the visible light image sequence to the same semantic region coordinate system. For the cabinet door observation window area, this step also establishes an observation window reflection mask for subsequent counterfactual interference verification.

[0044] Specifically, a mapping relationship can be established from the visible light image coordinate system, the infrared thermal image coordinate system, and the near-infrared smoke-enhanced image coordinate system to the semantic region coordinate system:

[0045]

[0046] in, Indicates the first The semantic region in the first Multimodal region image of a frame. Represents the semantic graph of the affected region Constrained temporal alignment, field of view registration, and semantic region mapping functions. Indicates the first Visible light image of a frame, Indicates the first Infrared thermal image of the frame, Indicates the first Near-infrared smoke enhancement image of the frame. Indicates the first The extent of a semantic region in the semantic region coordinate system Represents the visible light image mode in the 1st... Multimodal health weights of frames. Indicates the infrared thermal image mode in the 1st... Multimodal health weights of frames. This indicates that the near-infrared smoke enhancement image mode is in the first... Multimodal health weights of frames. Indicates the semantic region number. Indicates the image frame number. Same letters. All represent multimodal health weights, with different subscripts. , , These represent the visible light image mode, the infrared thermal image mode, and the near-infrared smoke-enhanced image mode, respectively.

[0047] Through the above mapping, a multimodal region image sequence is obtained. Each of them All images contain visible light region images, infrared thermal region images, and near-infrared smoke-enhanced region images within the same semantic region. Multimodal region image sequence. The output is then sent to S3 to construct the regional graph state vector, and then sent to S4 for counterfactual interference verification and graph causal decoupling.

[0048] Furthermore, for cases where the modalities of three types of images within the same semantic region are inconsistent, this step can generate a modal conflict identifier. For example, if a bright spot appears in the visible light region image but no temperature rise appears in the infrared thermal region image, and no smoke boundary appears in the near-infrared smoke enhancement region image, this region is marked as a suspected reflective region; if a temperature rise appears in the infrared thermal region image but the visible light region image is stable and no smoke appears in the near-infrared smoke enhancement region image, this region is marked as a suspected thermal anomaly region. This modal conflict identifier is transmitted to S3 and S4 for event chain matching and interference counter-evidence strength calculation.

[0049] This step unifies image data from different modalities, viewpoints, and sampling times into a single semantic region, and utilizes multimodal health weights and modal conflict identifiers to preserve image source reliability information. This reduces the impact of positional bias, temporal bias, and sensor anomalies in multimodal image fusion, and lowers misjudgments caused by region mismatch or single-modal distortion.

[0050] S3, construct a region map state vector based on the multimodal region image sequence, and match the region map state vector with a preset fire precursor event chain template to obtain the event chain matching strength.

[0051] In this step, the multimodal region image sequence is processed. Temporal feature extraction is performed to obtain the region map state vector for each semantic region within consecutive frames. The region map state vector is used to describe the visual state within the same semantic region related to early precursors of energy storage fires and non-fire interference.

[0052] Specifically, for the first The semantic region in the first Multimodal region image of a frame Extract the temperature rise slope, hotspot area growth rate, hotspot boundary irregularity, smoke boundary drift direction, smoke sparse particle density, arc flicker duration (frames), arc position overlap, occlusion disturbance intensity, reflection disturbance intensity, field-of-view stability, modal conflict intensity, and modal health decay to form a region map state vector:

[0053]

[0054] in, Indicates the first The semantic region in the first The region map state vector of the frame. Indicates the first The semantic region in the first The temperature rise slope of the frame, Indicates the first The semantic region in the first frame hotspot area growth rate Indicates the first The semantic region in the first frame hotspot boundary irregularity, Indicates the first The semantic region in the first The direction of smoke boundary drift in the frame. Indicates the first The semantic region in the first The sparse particle density of the smoke in the frame, Indicates the first The semantic region in the first The arc flashing lasts for [number] frames. Indicates the first The semantic region in the first The degree of overlap of the arc positions of the frames. Indicates the first The semantic region in the first The intensity of frame occlusion perturbation. Indicates the first The semantic region in the first The intensity of reflective disturbance in the frame. Indicates the first The semantic region in the first Frame field-of-view stability, Indicates the first The semantic region in the first The modal conflict intensity of the frame, Indicates the first The semantic region in the first The modal health decay of a frame. Indicates the semantic region number. Indicates the image frame number.

[0055] Among them, the temperature rise slope It can be obtained from the changes in the average temperature, maximum temperature, or hotspot center temperature of continuous infrared thermal region images; hotspot area growth rate It can be obtained from the change in the area of ​​pixel regions exceeding the adaptive temperature threshold in the infrared thermal image; hot spot boundary irregularity. It can be obtained from the perimeter, area, and shape factor of the hot spot boundary; the drift direction of the smoke boundary. It can be obtained from the optical flow direction of the smoke boundary in the near-infrared micro-smoke enhancement region image; the sparse particle density of the smoke. The number of particle scattering points in the near-infrared smoke enhancement area image can be obtained from the ratio of the area area to the number of particle scattering points; arc flicker duration frames. It can be obtained from the number of consecutive frames of short-duration brightness abrupt changes in the visible light region image; arc position overlap. The intensity of the shading disturbance can be obtained from the overlap ratio between the brightness abrupt change area and the cable connector area or busbar area. It can be obtained from the change in the area of ​​the foreground occlusion mask; reflective perturbation intensity It can be obtained from the shape of the bright area, the direction of specular reflection, and the degree of overlap between the cabinet door observation window area; field stability It can be obtained from the structural contour offset of adjacent frames or the results of global motion estimation; modal conflict intensity It can be obtained from the differences in anomalous responses of the three image modalities; modal health decay. It can be determined by multimodal health weights , , The degree of decline was obtained.

[0056] After obtaining the state vector of the region map Then, it is matched with a preset fire precursor event chain template. The preset fire precursor event chain template includes thermal anomaly leader events, smoke diffusion follow-up events, arc flash accompanying events, and area propagation events. Thermal anomaly leader events can be determined by the temperature rise slope. Hot spot area growth rate and hot spot boundary irregularity Characterization; the smoke diffusion following event can be represented by the direction of smoke boundary drift. and the density of sparse smoke particles Characterization; Arc-related events can be represented by the number of frames the arc flicker lasts. overlap with arc position Representation; regional propagation events can be represented by regional semantic graphs. The representation of the state changes of adjacent semantic regions under the constraints of spatial adjacency edges, cable connection edges, airflow propagation edges, or heat diffusion edges.

[0057] Furthermore, the preset fire precursor event chain template can include multiple sub-templates. The first sub-template is a thermal runaway progression template where localized temperature rise, hot spot expansion, and smoke diffusion occur sequentially; the second sub-template is an electrical connection anomaly template where arc flashes at cable joints, localized temperature rise, and temperature rise in adjacent busbar areas occur sequentially; the third sub-template is a smoke propagation template where smoke drift in the ventilation direction, smoke enhancement in adjacent areas, and consistent airflow propagation edges; the fourth sub-template is an observation window reflection elimination template where there is a sudden brightening of the cabinet door observation window, no temperature rise, no smoke, and increased reflective disturbance intensity. The observation window reflection elimination template is used to reduce the event chain matching intensity caused by reflections from the cabinet door observation window.

[0058] Event chain matching strength can be obtained using the following formula:

[0059]

[0060] in, Indicates the first The semantic region in the first Frame event chain matching strength, Indicates the event chain matching function, Indicates the first The semantic region from the first Frame to the The region map state vector sequence of the frame. Indicates the length of the historical frames used for event chain matching. Represents a semantic graph of regions. This represents a set of preset fire precursor event chain templates. Indicates the first The semantic region to the first The semantic region in the first Frame edge weights Indicates the semantic region number. Indicates the number of adjacent semantic regions. This indicates that the event chain matches the start frame number. Indicates the current image frame number.

[0061] When at least two of the following events—thermal anomaly leader event, smoke diffusion follow-up event, arc flash accompanying event, and regional propagation event—meet a preset temporal order within the same semantic region or adjacent semantic regions connected by directed semantic edges, the event chain matching strength is increased. For example, if a sustained temperature rise occurs first in the battery cluster area, followed by low-concentration smoke drifting along the vent direction, the event chain matching strength is increased; if an arc flash occurs first in the cable connector area, followed by a localized temperature rise in the same area or adjacent busbar areas, the event chain matching strength is increased; if a bright sudden change only occurs in the cabinet door observation window area and is not accompanied by a temperature rise, light smoke, or reasonable propagation, the event chain matching strength is not increased.

[0062] This step also allows for credibility backtracking of event chain matching results. Credibility backtracking refers to tracing back the region map state vectors of that semantic region and its adjacent semantic regions in historical frames when a semantic region receives a high event chain matching strength in the current frame. This process determines whether there are any underestimated early temperature rises, weak smoke, or short-duration arc events. If so, the credibility of the precursor corresponding to that historical event is increased, and this precursor credibility is used as an auxiliary input for subsequent S5 risk calculations. Credibility backtracking helps prevent weak early precursors from being ignored due to low intensity in a single frame.

[0063] In this step, the region map state vector The graph causal decoupling model output to S4, and the event chain matching strength. The early fire risk value of the region is output to S5. Thus, S3 not only completes the conversion from multimodal regional images to structured state data, but also converts the evolution relationship of fire precursors within consecutive frames into event chain matching intensity that can participate in risk calculation.

[0064] This step organizes localized temperature rise, low-concentration smoke, arc flash, propagation trend, modal conflict, and image health status into a time-series event chain, rather than isolating single-frame anomalies. This allows for the identification of anomalies that better align with the early development patterns of energy storage fires and reduces false alarms caused by transient reflections, short-term obstructions, image noise, or sensor deterioration.

[0065] S4, input the region graph state vector into the graph causal decoupling model, and output the intensity of thermal runaway precursor, smoke precursor, arc light precursor, interference counter-evidence intensity, graph propagation consistency intensity, and propagation consistency residual.

[0066] In this step, the graph causal decoupling model is used to analyze the region graph state vector. Decoupling identification is performed to distinguish between early precursors of real fires and non-fire interference. The graph causal decoupling model includes a shared multimodal coding network, a regional graph attention network, a thermal runaway causal branch, a micro-smoke causal branch, an arc light causal branch, a counterfactual interference branch, and a graph propagation residual branch.

[0067] Shared multimodal coding networks are used for region map state vectors and its corresponding multimodal region image A unified encoding process is performed to obtain shared features. The shared multimodal encoding network can be implemented using a convolutional neural network (CNN), a Transformer network, or a temporal convolutional network. A region map attention network is used to analyze the region semantic map. The directed semantic edges in the graph weight the shared features, allowing structural relationships related to battery clusters, busbars, cable connectors, vents, and cabinet door observation windows to participate in feature propagation. The region graph attention network can be implemented using either a GNN (Graph Neural Network) or a GAT (Graph Attention Network), where GNN stands for Graph Neural Network and GAT stands for Graph Attention Network.

[0068] The thermal runaway causal branch is used to output the intensity of the precursor to thermal runaway. This intensity is related to the temperature rise slope, the hotspot area growth rate, the hotspot boundary irregularity, and the thermal diffusion trend of adjacent semantic regions. The micro-smoke causal branch is used to output the smoke precursor intensity. This intensity is related to the smoke boundary drift direction, the density of sparse smoke particles, and the direction of airflow propagation. The arc causal branch is used to output the arc precursor intensity. This intensity is related to the number of frames the arc flicker lasts, the degree of overlap of the arc positions, and the location of the cable connector area or busbar area. The counterfactual interference branch is used to output the counterfactual interference intensity. This intensity characterizes the extent to which the current anomaly can be explained by reflections, obstructions, vibrations, or reflections from the cabinet door observation window. The graph propagation residual branch is used to output the graph propagation consistency intensity. and propagation consistency residual Among them, the graph propagation consistency strength The propagation consistency residual is used to characterize whether the current precursor has propagated reasonably along the directed semantic edges in the region semantic graph. This is used to characterize the deviation between the overall precursor intensity observed in actual observations and the graph propagation prediction.

[0069] The output of the graph causal decoupling model can be represented as:

[0070]

[0071] in, Indicates the first The semantic region in the first The intensity of the thermal runaway precursor in the frame. Indicates the first The semantic region in the first The intensity of the smoke precursor in the frame. Indicates the first The semantic region in the first The intensity of the arc precursor in the frame. Indicates the first The semantic region in the first The strength of the frame interference evidence. Indicates the first The semantic region in the first Frame graph propagation consistency strength, Indicates the first The semantic region in the first Frame propagation consistency residuals The parameter is The graph causal decoupling model, This represents the set of trainable parameters for a graph causal decoupling model. Indicates the first The semantic region in the first The region map state vector of the frame. Represents a semantic graph of regions. Indicates the first The semantic region in the first Multimodal region image of a frame. Represents the visible light image mode in the 1st... Multimodal health weights of frames. Indicates the infrared thermal image mode in the 1st... Multimodal health weights of frames. This indicates that the near-infrared smoke enhancement image mode is in the first... Multimodal health weights of frames. Indicates the semantic region number. Indicates the image frame number.

[0072] In this invention, the counterfactual interference branch performs masking replacement on the reflective area, occlusion area, jitter area, and reflection area of ​​the cabinet door observation window, respectively, to obtain a counterfactual region image sequence. Masking replacement can employ neighborhood texture filling, historical background frame replacement, brightness suppression replacement, stabilization replacement, or background replacement based on historical frames without anomalies within the same semantic region. The counterfactual region image sequence and the original multimodal region image sequence are input into the same graph causal decoupling model, and the difference in precursor responses before and after processing is compared to obtain the counterfactual interference strength. .

[0073] For example, the strength of the evidence to refute interference can be calculated using the following formula:

[0074]

[0075] in, Indicates the first The semantic region in the first The strength of the frame interference evidence. This represents the precursor response value obtained after inputting the original multimodal region image sequence into the causal decoupling model. This represents the precursor response value obtained after inputting a counterfactual region image sequence into the same graph causal decoupling model. Indicates the first The semantic region in the first Confidence of interference region in the frame. Indicates the semantic region number. Indicates the image frame number.

[0076] If the current megahertz response value decreases significantly after masking replacement, and the confidence level of the interference area is high, it indicates that the current anomaly can be explained by reflection, obstruction, vibration, or reflection from the cabinet door observation window, and the interference intensity is relatively low. High; if the current mega-response value remains high after masking replacement, it indicates that the current anomaly is not easily explained by interference, and the interference strength is a counter-evidence. Lower.

[0077] Furthermore, the counterfactual interference branch can also generate interference cause labels, which include reflections from cabinet door observation windows, reflections from inspection lights, personnel obstruction, camera shake, door opening and closing, and fan disturbances. Interference cause labels are not directly used as the fire severity output, but rather as the source of the interference counterfactual strength in S5, and as the basis for determining whether the coordinated response instructions in S6 require manual review.

[0078] In this invention, the graph propagation residual branch is based on the region semantic graph. Directed semantic edges and adaptive edge weights in Calculate the graph propagation consistency strength and propagation consistency residual:

[0079]

[0080] in, Indicates the first The semantic region in the first Frame graph propagation consistency strength, Indicates the first The semantic region in the first Frame propagation consistency residuals Indicates the relationship with the first Each semantic region has a set of adjacent semantic regions connected by directed semantic edges. express Adjacent semantic regions numbering in the data. Indicates the first The semantic region to the first The semantic region in the first Frame edge weights Indicates the first The semantic region up to the first The semantic region in the first Precursor propagation consistency of frames Indicates the first The semantic region in the first The overall precursor intensity of the frame, Indicates the first The semantic region in the first The overall precursor intensity of the frame, Indicates the first The frame before the frame, Indicates the semantic region number. Indicates the image frame number.

[0081] Overall precursor intensity It can be obtained by weighting the intensity of thermal runaway precursors, smoke precursors, and arc precursors:

[0082]

[0083] in, Indicates the first The semantic region in the first The overall precursor intensity of the frame, Indicates the first The semantic region in the first The intensity of the thermal runaway precursor in the frame. Indicates the first The semantic region in the first The intensity of the smoke precursor in the frame. Indicates the first The semantic region in the first The intensity of the arc precursor in the frame. , , These represent the combined weights of the intensity of thermal runaway precursors, smoke precursors, and arc precursors, respectively. Indicates the semantic region number. Indicates the image frame number.

[0084] When a precursor anomaly undergoes a reasonable evolution along spatial adjacent edges, cable connection edges, airflow propagation edges, or heat diffusion edges, the consistency of precursor propagation is... High, graph propagation consistency strength High; when the overall precursor intensity of the current semantic region is high. When the propagation prediction result deviates significantly from that of the previous frame in the adjacent semantic region, the propagation consistency residual... The magnitude is relatively large, indicating that this anomaly does not conform to the propagation pattern of real fire precursors.

[0085] In this invention, the training samples for the graph causal decoupling model include normal operation samples, thermal runaway precursor samples, low-concentration smoke samples, arc light samples, reflective interference samples, occlusion interference samples, camera shake samples, cabinet door observation window reflection samples, and fan disturbance samples. Each type of sample includes visible light image sequences, infrared thermal image sequences, near-infrared micro-smoke enhanced image sequences, regional semantic icon annotations, abnormal region annotations, precursor category annotations, interference category annotations, and propagation direction annotations.

[0086] During training, the same temporal alignment, field-of-view registration, and semantic region mapping as in S2 are first performed on the three types of image sequences to obtain the training multimodal region image sequences. Then, the training region graph state vector is constructed based on S3. Subsequently, the training region graph state vector is input into the graph causal decoupling model to calculate the precursor classification loss, region localization loss, event chain matching loss, counterfactual consistency loss, interference counter-evidence loss, and graph propagation residual loss, and the model parameters are updated through backpropagation. .

[0087] The training objective can be expressed as:

[0088]

[0089] in, This represents the total training loss of the graph causal decoupling model. Indicates precursor classification loss, Indicates regional positioning loss. Indicates the event chain matching loss. Indicates counterfactual consistency loss. This indicates the loss due to interference with the evidence. This represents the residual loss during graph propagation. This represents the modal health constraint loss. , , , , , , These represent the loss weights for precursor classification loss, region localization loss, event chain matching loss, counterfactual consistency loss, interference counter-evidence loss, graph propagation residual loss, and modal health constraint loss, respectively.

[0090] Among them, the precursor classification loss is used to constrain the classification results of thermal runaway precursors, smoke precursors, and arc flash precursors; the region localization loss is used to constrain the consistency between abnormal locations and labeled semantic regions; the event chain matching loss is used to constrain the consistency between the event chain matching strength and the evolution order of real fire precursors; the counterfactual consistency loss is used to constrain the response consistency between the counterfactual region image sequence and the original multimodal region image sequence in non-interference regions; the interference counter-evidence loss is used to improve the interference counter-evidence strength of samples such as reflection, occlusion, jitter, cabinet door observation window reflection, and fan disturbance; the graph propagation residual loss is used to constrain real fire precursor samples to form a smaller propagation consistency residual along the directed semantic edges of the region semantic graph, and to constrain non-fire interference samples to form a larger propagation consistency residual; and the modal health constraint loss is used to constrain the contribution of low health weight image modes to the final risk judgment, avoiding false alarms caused by defocusing, overexposure, thermal imaging drift, or near-infrared illumination anomalies.

[0091] Furthermore, when the target energy storage compartment lacks sufficient samples of precursors to thermal runaway, low-sample domain adaptive training can be employed. Low-sample domain adaptive training involves using samples obtained from other energy storage compartments, experimental compartments, or simulation compartments as source domain samples, and a small number of normal operation samples and a small number of abnormal samples from the target energy storage compartment as target domain samples. By aligning the state vector distributions of the region graphs in the source and target domains, the parameters of the shared multimodal coding network and the region graph attention network are updated, allowing the model to adapt to differences in camera installation angles, cabinet structures, ventilation methods, and background lighting within the target energy storage compartment. This training method reduces the dependence on the number of real fire samples from the target energy storage compartment.

[0092] In this step, the thermal runaway precursor intensity output by the graph causal decoupling model is... Smoke precursor intensity Arc precursor intensity Strength of interference with counter-evidence Graph Propagation Consistency Strength and propagation consistency residual All data are output to S5 for calculating early-stage fire risk values ​​for the region.

[0093] This step separates fire precursor information and non-fire interference information from the regional map state vector, and improves the recognition credibility through counterfactual interference verification, graph propagation residual judgment, multimodal health constraints and low sample domain adaptation. It can reduce false alarms caused by inspection light reflection, cabinet door observation window reflection, personnel obstruction, camera shake, hatch opening and closing, fan disturbance and image modal anomalies, while retaining the response of real early signs of thermal runaway.

[0094] S5. Calculate the early fire risk value of the region based on the event chain matching strength, thermal runaway precursor strength, smoke precursor strength, arc flash precursor strength, interference counter-evidence strength, graph propagation consistency strength, and propagation consistency residual.

[0095] In this step, the event chain matching strength obtained from S3 is... The thermal runaway precursor intensity obtained from S4 Smoke precursor intensity Arc precursor intensity Strength of interference with counter-evidence Graph Propagation Consistency Strength Propagation consistency residual The common input risk calculation process yields the first... The semantic region in the first Frame of regional early fire risk values .

[0096] The early fire risk value of a region can be calculated using the following formula:

[0097]

[0098] in, Indicates the first The semantic region in the first Early fire risk value for the area of ​​the frame. Represents the normalized mapping function. Indicates the first The semantic region in the first The intensity of the thermal runaway precursor in the frame. Indicates the first The semantic region in the first The intensity of the smoke precursor in the frame. Indicates the first The semantic region in the first The intensity of the arc precursor in the frame. Indicates the first The semantic region in the first Frame graph propagation consistency strength, Indicates the first The semantic region in the first Frame event chain matching strength, Indicates the first The semantic region in the first The strength of the frame interference evidence. Indicates the first The semantic region in the first Frame propagation consistency residuals Indicates the first The semantic region in the first The modal health decay of a frame. , , , , , , , These represent the weights of the thermal runaway precursor intensity, smoke precursor intensity, arc flash precursor intensity, graph propagation consistency intensity, event chain matching intensity, interference counter-evidence intensity, propagation consistency residual, and modal health decay, respectively. Indicates the semantic region number. Indicates the image frame number.

[0099] In this formula, the intensity of the precursor to thermal runaway Smoke precursor intensity Arc precursor intensity Graph Propagation Consistency Strength Matching strength with event chain It is included in the calculation as a positive risk term; it interferes with the strength of the counter-evidence. Propagation consistency residual and modal health decay It is included in the calculation as a negative correction term. When a region simultaneously exhibits continuous temperature rise, low-concentration smoke, arc flashes, a reasonable event chain sequence, and a propagation direction consistent with the regional semantic map, the early fire risk value of the region is... Increase; when the current anomaly can be explained by reflection, obstruction, shaking, or reflection from the cabinet door observation window, or when the current anomaly does not conform to the propagation pattern defined by the regional semantic map, or when the image modality health status declines, the regional early fire risk value increases. reduce.

[0100] Furthermore, this step can perform cross-regional risk retrospective correction on the early-stage fire risk value of the region. Cross-regional risk retrospective correction refers to, when the... If the early fire risk value of a semantic region increases in the current frame and its adjacent semantic regions have weak precursor responses in historical frames, the confidence of the weak precursor response is increased based on the directed semantic edges of the region semantic graph; if the current risk increase only occurs in the region corresponding to the visible occlusion edge and there is no heat diffusion edge, airflow propagation edge or cable connection edge to support it, the confidence of the risk increase is reduced.

[0101] In this step, the regional early fire risk value The output is sent to S6 for dual-threshold hysteresis graded early warning judgment. Thus, S5 integrates multimodal image precursors, temporal event chains, region map propagation, interference counter-evidence, modal health status, and historical weak precursor backtracking into a single risk value that can be used for coordinated control.

[0102] This step unifies the intensity of precursors and counter-evidence from different sources into the same risk assessment framework, avoiding direct alarm triggering by a single temperature threshold, a single smoke threshold, a single image recognition result, or a low-quality image, thereby improving the stability and anti-interference capability of early fire identification.

[0103] S6, when the early fire risk value in the area meets the dual-threshold hysteresis graded early warning conditions, output the abnormal location, early fire level and linkage response instructions of the corresponding semantic area.

[0104] In this step, the regional early fire risk value is calculated for each semantic region. A dual-threshold hysteresis-based graded early warning system is used to determine the fire risk level. This system includes an increasing threshold and a decreasing threshold, with the increasing threshold being higher than the decreasing threshold. When the early fire risk value of a region is higher than the corresponding increasing threshold for multiple consecutive frames, the early fire risk level is raised; conversely, when the early fire risk value is lower than the corresponding decreasing threshold for multiple consecutive frames, the early fire risk level is lowered or eliminated.

[0105] Specifically, three levels of early warning can be set: Level 1, Level 2, and Level 3. A Level 1 warning indicates the presence of suspicious early warning signs; a Level 2 warning indicates the presence of strong thermal runaway or smoke precursors; and a Level 3 warning indicates the presence of a high-risk early fire condition requiring coordinated response. Coordinated response instructions may include increasing sampling frequency, sending maintenance alarms, activating local ventilation, reducing charging and discharging power, disconnecting abnormal battery cluster circuits, activating fire sprinkler standby status, or triggering the fire control system.

[0106] The dual-threshold hysteresis graded early warning can be represented as:

[0107]

[0108] in, Indicates the first The semantic region in the first Frame fire early level, Indicates the first The semantic region in the first Frame fire early level, Indicates the first The semantic region in the first Early fire risk value for the area of ​​the frame. Indicates the early fire level from the previous frame. The required advancement threshold to the next level. This indicates maintaining or reducing the early fire level from the previous frame. The descent threshold used, This indicates the number of consecutive frames required for the risk value to be no less than the rising threshold. This indicates the number of consecutive frames required for the risk value to be no greater than the drop threshold. Indicates the semantic region number. Indicates the image frame number. Indicates the first The frame before the previous frame.

[0109] When an early fire level is generated Then, this level is compared with the regional semantic map. semantic region numbering The system correlates and outputs the anomaly location, early fire warning level, and coordinated response instructions for the corresponding semantic region. For example, when the anomaly location is in the cable joint area and the early fire warning level reaches level two, it outputs the anomaly location of the cable joint area and coordinated response instructions to reduce charging and discharging power and push maintenance alarms; when the anomaly location is in the battery cluster area and the early fire warning level reaches level three, it outputs the anomaly location of the battery cluster area and coordinated response instructions to cut off the abnormal battery cluster circuit and activate the fire sprinkler standby state; when the anomaly is mainly located in the cabinet door observation window area and the interference intensity is high, it does not raise the early fire warning level or only outputs inspection and verification prompts.

[0110] Furthermore, this step can use the risk changes after the coordinated response as feedback data to return to the subsequent training of the regional semantic graph edge weight adaptive update and graph causal decoupling model. If the intensity of the smoke precursor decreases along the ventilation direction after local ventilation is implemented, the response feedback of the airflow propagation edge is recorded as effective; if the intensity of the thermal runaway precursor in the battery cluster area decreases after reducing the charging and discharging power, the anomaly is recorded as being related to the thermal runaway precursor of the battery cluster; if manual verification confirms that it is a reflection from the cabinet door observation window, the corresponding frame is added as a reflection interference sample to the subsequent incremental training of the model. Through this coordinated response feedback correction mechanism, the system can gradually improve its adaptability to the energy storage cabin scenario during long-term operation.

[0111] This step converts continuous frame risk values ​​into stable early fire levels and executable linkage response instructions. By correcting subsequent identification processes through response feedback, it avoids frequent alarms, alarm cancellations, or level jumps caused by risk value fluctuations near the threshold, improves the engineering stability of early warning output, and enhances the system's long-term adaptability to different energy storage compartment site environments.

[0112] Second Embodiment

[0113] like Figure 2 As shown, the second embodiment also provides an early identification system for energy storage fires based on computer vision. The system includes an image acquisition module, a regional semantic modeling module, a multimodal regional mapping module, a regional state construction module, an event chain matching module, a graph causal decoupling module, a risk calculation module, and an early warning linkage module.

[0114] The image acquisition module acquires visible light image sequences, infrared thermal image sequences, and near-infrared smoke-enhanced image sequences of the same target monitoring area within the energy storage compartment. The image acquisition module may include a visible light camera, an infrared thermal imaging device, a near-infrared camera, a near-infrared illumination component, and an image synchronization unit. The image acquisition module may also include a sensor health self-calibration unit, which generates multimodal health weights based on image sharpness, exposure stability, thermal imaging drift, near-infrared illumination uniformity, and timestamp continuity. The image acquisition module outputs the visible light image sequences, infrared thermal image sequences, near-infrared smoke-enhanced image sequences, and multimodal health weights to the multimodal region mapping module.

[0115] The regional semantic modeling module generates a regional semantic map based on battery clusters, busbars, cable connectors, fire sprinklers, ventilation openings, and cabinet door observation windows. This module abstracts structural components of the energy storage compartment as semantic nodes and spatial adjacency relationships, cable connection relationships, airflow propagation relationships, heat diffusion relationships, and visual occlusion relationships as directed semantic edges, thus generating the regional semantic map. The regional semantic modeling module may also include an edge weight adaptive update unit, which updates the edge weights of directed semantic edges based on the wind turbine operating status, hatch opening and closing status, charging and discharging conditions, and historical precursor propagation records. The edge weights are output to the multimodal region mapping module, the region state construction module, the event chain matching module, the graph causal decoupling module, and the risk calculation module.

[0116] The multimodal region mapping module is used to utilize region semantic graphs. Temporal alignment, field-of-view registration, and semantic region mapping were performed on visible light image sequences, infrared thermal image sequences, and near-infrared smoke-enhanced image sequences to obtain multimodal region image sequences. The multimodal region mapping module is also used to generate modal conflict identifiers based on the differences in anomalous responses among the three types of image modalities, and output the multimodal region image sequence and modal conflict identifiers to the region state construction module and the graph causal decoupling module.

[0117] The region state construction module is used to construct region map state vectors based on multimodal region image sequences. The region state construction module extracts the following features from the multimodal region image sequence: temperature rise slope, hotspot area growth rate, hotspot boundary irregularity, smoke boundary drift direction, smoke sparse particle density, arc flicker duration (number of frames), arc position overlap, occlusion perturbation intensity, reflection perturbation intensity, field-of-view stability, modal conflict intensity, and modal health decay. These features are then combined into a region graph state vector. The region state construction module outputs this region graph state vector to the event chain matching module and the graph causal decoupling module.

[0118] The event chain matching module is used to match the state vector of the region map. The event chain matching strength is obtained by matching the event chain with a preset fire precursor event chain template. The event chain matching module determines whether the current anomaly conforms to the early evolution pattern of an energy storage fire based on the temporal relationships between thermal anomaly precursor events, smoke diffusion follow-up events, arc flash accompanying events, and regional propagation events, as well as the directed semantic edges in the regional semantic graph. The event chain matching module may also include a credibility backtracking unit, which is used to trace historical weak precursor responses when the risk in the current frame increases, and correct the event chain matching strength based on the propagation relationship of adjacent semantic regions. The event chain matching module outputs the event chain matching strength to the risk calculation module.

[0119] The graph cause-effect decoupling module is used to decouple the state vector of the region graph. Input a graph-causal decoupling model, output the intensity of thermal runaway precursors. Smoke precursor intensity Arc precursor intensity Strength of interference with counter-evidence Graph Propagation Consistency Strength and propagation consistency residual The graph causality decoupling module includes a shared multimodal coding network, a region graph attention network, a thermal runaway causality branch, a micro-smoke causality branch, an arc light causality branch, a counterfactual interference branch, and a graph propagation residual branch. The module may also include a low-sample-domain adaptive training unit to update model parameters based on source and target domain samples, adapting the model to the on-site environment of the target energy storage chamber. The graph causality decoupling module transmits its output to the risk calculation module.

[0120] The risk calculation module is used for event chain matching strength. Intensity of precursors to thermal runaway Smoke precursor intensity Arc precursor intensity Strength of interference with counter-evidence Graph Propagation Consistency Strength Propagation consistency residual and modal health decay Calculate the early fire risk value of the area The risk calculation module can also perform cross-regional risk retrospective correction to adjust the current risk value based on historical weak precursor responses and regional semantic map propagation relationships. The risk calculation module outputs the regional early fire risk value to the early warning linkage module.

[0121] The early warning linkage module is used to assess the early fire risk value in the region. When the dual-threshold hysteresis graded early warning conditions are met, the system outputs the abnormal location, early fire level, and linkage response instructions for the corresponding semantic region. The early warning linkage module can communicate with energy storage management systems, fire control systems, operation and maintenance alarm systems, ventilation systems, and power outage control systems to implement alarm push notifications, sampling frequency increases, local ventilation, power reduction, power outages, or fire sprinkler preparation. The early warning linkage module may also include a response feedback correction unit, used to return the risk changes after response, manual review results, and response effects to the regional semantic modeling module and the graph causal decoupling module to update edge weights, training samples, and risk calculation parameters.

[0122] After adopting the above method and system, the visible light image sequence, infrared thermal image sequence, and near-infrared smoke enhancement image sequence inside the energy storage compartment are no longer processed in isolation, but are mapped to the same semantic region coordinate system through a regional semantic graph. Local temperature rise, smoke diffusion, arc flash, and non-fire interference are all converted into regional graph state vectors and participate in event chain matching and graph causal decoupling identification, respectively. The temporal evolution pattern of early-stage energy storage fires is identified by the intensity of event chain matching; false alarms caused by reflection, obstruction, shaking, and reflection from cabinet door observation windows are suppressed by the intensity of interference counter-evidence; and whether precursor anomalies propagate reasonably along the structural relationship of the energy storage compartment are determined by the graph propagation consistency intensity and propagation consistency residual. Finally, a regional early fire risk value is formed and a stable graded early warning result is output.

[0123] In a further implementation, the multimodal sensor health self-calibration can identify problems such as defocusing, overexposure, thermal imaging drift, near-infrared illumination anomalies, and timestamp discontinuities, and reduce the impact of low-confidence image modalities on risk assessment; the regional semantic graph edge weight adaptive update can adjust propagation relationships based on wind turbine operation, hatch opening and closing, charging and discharging conditions, and historical propagation records; the precursor confidence backtracking can trace historical weak precursors after the current risk increases, improving the retention capacity of early weak anomalies; the low sample domain adaptation can reduce the impact of insufficient real fire samples in the target energy storage compartment on model training; and the linkage response feedback correction can continuously optimize edge weights, training samples, and risk calculation parameters by utilizing the risk changes after response and the results of manual review.

[0124] In this implementation, the steps form a continuous data processing chain: the regional semantic graph, multimodal health weights, and edge weights generated in S1 are used for registration mapping in S2, event chain matching in S3, graph causal decoupling in S4, and risk calculation in S5; the multimodal regional image sequence and modal conflict identifiers generated in S2 are used for regional graph state vector construction in S3 and counterfactual interference verification in S4; the regional graph state vector and event chain matching strength generated in S3 are used for model recognition in S4 and risk calculation in S5, respectively; the various precursor intensities, interference counterfactual strengths, graph propagation consistency strengths, and propagation consistency residuals generated in S4 are used to form early regional fire risk values ​​in S5; the early regional fire risk values ​​obtained in S5 are used to generate early fire levels and coordinated response instructions in S6; the coordinated response feedback results in S6 are returned to the adaptive update of regional semantic graph edge weights in S1 and the incremental training of the model in S4. Thus, the entire identification process forms a complete closed loop around the technical issues of early precursor identification, false alarm suppression, stable early warning, and long-term scenario adaptation for energy storage fires.

[0125] The above embodiments are only used to illustrate the technical solution of the present invention and do not limit the scope of protection of the present invention. Without departing from the technical concept of the present invention, the parameters, thresholds, weights and model structures of each module can be adjusted according to the energy storage compartment structure, camera equipment deployment method, number of early warning levels, linkage response strategy, training sample size and on-site operating conditions; these adjustments can all be implemented while keeping the data coupling relationship between the regional semantic graph, multimodal regional image sequence, regional graph state vector, event chain matching strength, graph causal decoupling output, regional early fire risk value and linkage response feedback unchanged.

Claims

1. A method for early identification of energy storage fires based on computer vision, characterized in that, include: S1. Acquire visible light image sequences, infrared thermal image sequences, and near-infrared smoke enhancement image sequences for the same target monitoring area of ​​the energy storage compartment, and generate a region semantic map based on battery clusters, busbars, cable connectors, fire nozzles, ventilation openings, and cabinet door observation windows; S2. Use the region semantic map to perform time alignment, field-of-view registration, and semantic region mapping on the visible light image sequences, infrared thermal image sequences, and near-infrared smoke enhancement image sequences to obtain a multimodal region image sequence; S3. Construct a region map state vector based on the multimodal region image sequence, and match the region map state vector with a preset fire precursor event chain template to obtain... S4. Input the regional graph state vector into the graph causal decoupling model and output the intensity of thermal runaway precursor, smoke precursor, arc flash precursor, interference counter-evidence intensity, graph propagation consistency intensity, and propagation consistency residual; S5. Calculate the regional early fire risk value based on the event chain matching strength, thermal runaway precursor intensity, smoke precursor intensity, arc flash precursor intensity, interference counter-evidence intensity, graph propagation consistency intensity, and propagation consistency residual; S6. When the regional early fire risk value meets the dual-threshold hysteresis graded early warning condition, output the abnormal location, early fire level, and linkage response instruction of the corresponding semantic region.

2. The computer vision-based early identification method for energy storage fires according to claim 1, characterized in that, The regional semantic graph includes semantic nodes and directed semantic edges. The semantic nodes correspond to the battery cluster area, busbar area, cable joint area, fire nozzle area, ventilation area, and cabinet door observation window area. The directed semantic edges include spatial adjacency edges, cable connection edges, airflow propagation edges, heat diffusion edges, and visual obstruction edges.

3. The computer vision-based early identification method for energy storage fires according to claim 2, characterized in that, In S2, the battery cluster boundary, cabinet boundary, ventilation opening boundary, and cabinet door observation window boundary in the region semantic map are used as fixed structural constraints to map the hot spot center and hot spot boundary in the infrared thermal image sequence, the smoke boundary and sparse smoke particles in the near-infrared micro-smoke enhancement image sequence, and the structural contour and brightness change region in the visible light image sequence to the same semantic region coordinate system to form the multimodal region image sequence.

4. The computer vision-based early identification method for energy storage fires according to claim 1, characterized in that, The region map state vector includes the temperature rise slope, hot spot area growth rate, hot spot boundary irregularity, smoke boundary drift direction, smoke sparse particle density, arc flicker duration in consecutive frames, arc position overlap, occlusion disturbance intensity, reflection disturbance intensity, and field of view stability for the same semantic region.

5. The computer vision-based early identification method for energy storage fires according to claim 1, characterized in that, The preset fire precursor event chain template includes thermal anomaly leading events, smoke diffusion following events, arc flash accompanying events, and regional propagation events. When at least two of the thermal anomaly leading events, smoke diffusion following events, arc flash accompanying events, and regional propagation events meet the preset temporal order in the same semantic region or in adjacent semantic regions connected by directed semantic edges, the event chain matching strength of the corresponding semantic region is determined to be increased or the event chain matching strength of the corresponding semantic region is improved.

6. The method for early identification of energy storage fires based on computer vision according to claim 1, characterized in that, The graph causal decoupling model includes a shared multimodal coding network, a region graph attention network, a thermal runaway causal branch, a smoke causal branch, an arc light causal branch, a counterfactual interference branch, and a graph propagation residual branch. The shared multimodal coding network encodes the region graph state vector and the multimodal region images corresponding to the region graph state vector in the multimodal region image sequence. The encoding results are weighted by the region graph attention network according to the directed semantic edges in the region semantic graph, and then input into the thermal runaway causal branch, the smoke causal branch, the arc light causal branch, the counterfactual interference branch, and the graph propagation residual branch, respectively.

7. The computer vision-based early identification method for energy storage fires according to claim 6, characterized in that, The counterfactual interference branch performs masking replacement on the reflective area, the obstructed area, the shaking area, and the reflection area of ​​the cabinet door observation window to obtain a counterfactual area image sequence. The counterfactual area image sequence and the multimodal area image sequence without masking replacement are input into the same graph causal decoupling model, and the difference in the precursor response before and after the processing is compared to obtain the counterfactual interference strength.

8. The computer vision-based early identification method for energy storage fires according to claim 6, characterized in that, The graph causal decoupling model is trained using normal operation samples, thermal runaway precursor samples, low-concentration smoke samples, arc light samples, reflective interference samples, occlusion interference samples, camera shake samples, cabinet door observation window reflection samples, and fan disturbance samples. It is jointly constrained by precursor classification loss, region localization loss, event chain matching loss, counterfactual consistency loss, interference counter-evidence loss, and graph propagation residual loss.

9. The method for early identification of energy storage fires based on computer vision according to claim 1, characterized in that, In S5, the first step is calculated according to the following formula. The semantic region in the first Early fire risk values ​​for the area of ​​the frame: ， in, Indicates the first The semantic region in the first Early fire risk value for the area of ​​the frame. Represents the normalized mapping function. Indicates the first The semantic region in the first The intensity of the thermal runaway precursor in the frame. Indicates the first The semantic region in the first The intensity of the smoke precursor in the frame. Indicates the first The semantic region in the first The intensity of the arc precursor in the frame. Indicates the first The semantic region in the first Frame graph propagation consistency strength, Indicates the first The semantic region in the first Frame event chain matching strength, Indicates the first The semantic region in the first The strength of the frame interference evidence. Indicates the first The semantic region in the first Frame propagation consistency residuals Indicates the modal health decay. , , , , , , , These represent the weights of the thermal runaway precursor intensity, smoke precursor intensity, arc flash precursor intensity, graph propagation consistency intensity, event chain matching intensity, interference counter-evidence intensity, propagation consistency residual, and modal health decay, respectively. Indicates the semantic region number. Indicates the image frame number.

10. A computer vision-based early detection system for energy storage fires, characterized in that, include: The image acquisition module is used to acquire visible light image sequences, infrared thermal image sequences, and near-infrared smoke enhanced image sequences of the same target monitoring area in the energy storage compartment; The regional semantic modeling module is used to generate regional semantic maps based on battery clusters, busbars, cable connectors, fire nozzles, ventilation openings, and cabinet door observation windows; the multimodal regional mapping module is used to perform time alignment, field registration, and semantic regional mapping on the visible light image sequence, infrared thermal image sequence, and near-infrared smoke enhancement image sequence using the regional semantic maps to obtain a multimodal regional image sequence. The region state construction module is used to construct a region map state vector based on the multimodal region image sequence; the event chain matching module is used to match the region map state vector with a preset fire precursor event chain template to obtain the event chain matching strength. The graph-causal decoupling module is used to input the regional graph state vector into the graph-causal decoupling model and output the intensity of thermal runaway precursor, smoke precursor, arc flash precursor, interference counter-evidence intensity, graph propagation consistency intensity, and propagation consistency residual; the risk calculation module is used to calculate the early fire risk value of the region based on the event chain matching intensity, thermal runaway precursor intensity, smoke precursor intensity, arc flash precursor intensity, interference counter-evidence intensity, graph propagation consistency intensity, and propagation consistency residual. The early warning linkage module is used to output the abnormal location, early fire level, and linkage response instructions of the corresponding semantic region when the early fire risk value of the area meets the dual threshold hysteresis graded early warning conditions.

Images