A battery health diagnosis method based on multi-modal image fusion

Abstract

The application discloses a battery health diagnosis method based on multi-modal image fusion, and relates to the technical field of battery health diagnosis. The method comprises the following steps: synchronously collecting visible light and infrared thermal imaging images of a battery, combining prior data of the battery to divide local diagnosis units; based on the local diagnosis units, respectively extracting visible light and infrared thermal imaging feature vectors, calling a pre-constructed cross-modal consistency constraint model, and generating a cross-modal consistency constraint; combining the feature vectors and the cross-modal constraint, calculating the double-modal correlation degree of each unit, and determining the unit failure mode; when the failure mode is a high-confidence failure, a spatial clustering algorithm based on graph connectivity is used to analyze the spatial expansion trend thereof; and comprehensively generating a battery health diagnosis result by combining the unit failure mode and the spatial expansion trend. Through the collaborative analysis and consistency constraint of the multi-modal images, the influence of environmental interference and incidental abnormalities is significantly reduced.

Description

Technical Field

[0001] This invention relates to the field of battery health diagnosis technology, specifically to a battery health diagnosis method based on multimodal image fusion. Background Technology

[0002] With the rapid development of new energy vehicles and large-scale energy storage technologies, the health status of batteries, as core energy storage components, directly affects the safety and reliability of the entire system. During long-term cycling and calendar use, irreversible chemical side reactions and physical structural changes occur inside the battery, leading to performance degradation such as capacity decay and increased internal resistance, and may even trigger serious safety issues such as thermal runaway. Therefore, achieving accurate and early diagnosis of battery health status has become a key issue in the field of battery management technology.

[0003] Existing battery health diagnostic methods are typically based on single-modal images. These methods use visible light cameras to monitor the battery and identify anomalies by analyzing the acquired visible light images. Specifically, visible light images are used to detect physical changes in the battery's appearance, such as casing bulging, surface cracks, or electrolyte leakage. The images are analyzed using preset feature thresholds to obtain battery health diagnostic results.

[0004] However, such battery health diagnosis methods based on single-modal images rely solely on physical dimension information such as visible light images for judgment. The actual aging or failure of batteries is often a complex process involving multiple coupled factors. Due to the lack of collaborative verification from another modality, it is difficult to eliminate misjudgments caused by occasional interference, ultimately resulting in low reliability and robustness of battery health diagnosis results in complex industrial application scenarios. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a battery health diagnosis method based on multimodal image fusion, which solves the problems existing in the background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a battery health diagnosis method based on multimodal image fusion, comprising the following steps: Step S1: Acquire visible light and infrared thermal images of the battery, collect prior battery data, divide the battery based on the prior battery data, and obtain local diagnostic units; Step S2: Through the local diagnostic unit, feature extraction is performed on the visible light image and infrared thermal imaging image of the battery respectively to obtain the visible light feature vector and infrared thermal imaging feature vector of the battery. The pre-built cross-modal consistency constraint model is called to generate cross-modal consistency constraints. Step S3: Based on the visible light feature vector, infrared thermal imaging feature vector and cross-modal consistency constraint, calculate the bimodal correlation degree of each local diagnostic unit and determine the failure mode of each local diagnostic unit; Step S4: When the failure mode of the local diagnostic unit is a high-confidence failure, the spatial expansion trend of the high-confidence failure is obtained by analyzing it using a spatial clustering algorithm based on graph connectivity. Step S5: Combine the failure modes of each local diagnostic unit with the spatial expansion trend of high-confidence failures to generate battery health diagnostic results.

[0007] Preferably, acquiring visible light and infrared thermal images of the battery includes: Visible light images of the battery under test are simultaneously acquired from the same observation angle using a visible light camera and an infrared thermal imager. and initial infrared thermal imaging images ; Visible light image of the battery and initial infrared thermal imaging images Spatial registration is performed using a feature-point-based registration method. This involves extracting salient feature points from the two images, establishing the correspondence between these feature points, and solving for the optimal affine transformation model to map the initial infrared thermal image to the visible light image coordinate system. The registration process is achieved using the following affine transformation formula: ; in, Represents the original infrared initial image The coordinates of the middle pixel; Represents the initial image of infrared thermal imaging. The corresponding coordinates in the registered image; These are matrix parameters that collectively determine rotation, scaling, and shear transformations; The vector parameters determine the translation transformation; After registration, an infrared thermal image of the battery was obtained for subsequent analysis. .

[0008] Preferably, collecting prior battery data and dividing the battery based on the prior battery data to obtain local diagnostic units includes: Collect prior data of the battery under test. It is a data set that includes the three-dimensional structural model of the battery, the precise location of the tabs and solder joints, and information on historically vulnerable areas. After image registration, based on prior data The battery's three-dimensional structural model, the precise locations of the tabs and solder joints, and information on historically degraded areas were used to divide the battery surface. During this division, the boundaries of stress concentration areas were identified. Simultaneously, historically degraded areas were also treated as independent or key units, further dividing the battery surface into... A set of non-overlapping local diagnostic units. ,in, Represents the set of all local diagnostic units; This represents the total number of units.

[0009] Preferably, the local diagnostic unit extracts features from the visible light image and infrared thermal imaging image of the battery to obtain the visible light feature vector and infrared thermal imaging feature vector of the battery, respectively, including: For each local diagnostic unit obtained in step S1 In the visible light image of the battery Infrared thermal imaging images of batteries The image patch corresponding to the unit region is cropped, and then visible light feature vectors reflecting its appearance integrity and surface texture are extracted from the visible light image patch. Infrared thermal imaging feature vectors reflecting temperature distribution and thermal gradient are extracted from infrared image patches. .

[0010] Preferably, the process of generating cross-modal consistency constraints by invoking a pre-built cross-modal consistency constraint model includes: To establish the correlation between visible light features and infrared thermal imaging features, a cross-modal consistency constraint model needs to be pre-constructed. This model is based on a fundamental physical principle: under the battery health condition, there is a stable correspondence between appearance and thermal behavior features. The construction of this model depends on the statistical regularities learned from the battery's historical health data. Specifically, the battery historical health data refers to the periodic acquisition of visible light and infrared thermal imaging images of the battery within its known healthy operating cycle. After processing in step S1 and feature extraction, the visible light feature vector and infrared thermal imaging feature vector pairs of each local diagnostic unit are obtained. These paired vectors constitute the basic dataset reflecting cross-modal correlations under healthy conditions, denoted as... ,in The total number of historical healthy samples, superscript Used for indexing the historical dataset One sample pair; Based on the aforementioned historical battery health data, visible light characteristics were fitted using a multivariate statistical analysis method. To infrared thermal imaging features mapping relationship And determine its normal fluctuation range, thereby constructing a cross-modal consistency constraint model, which can be expressed as: for a new, local diagnostic unit to be diagnosed... Features Calculate its residual vector Mahalanobis distance , This is the cross-modal consistency constraint, which should be less than a threshold determined based on the distribution of health data. .

[0011] Preferably, based on the visible light feature vector, the infrared thermal imaging feature vector, and the cross-modal consistency constraint, the bimodal correlation degree of each local diagnostic unit is calculated as follows: Combining unimodal anomaly scores with crossmodal consistency constraints Weighted fusion is performed to obtain the final bimodal correlation degree. : ; in, Represents a local diagnostic unit The bimodal correlation degree has a range of [0,1]; It is the Sigmoid function, used to smoothly map input values ​​to the [0,1] interval; These are the weights of the visible light single-mode anomaly scores after standardization using the Sigmoid function; The weights are the infrared single-mode anomaly scores after standardization using the Sigmoid function. It is the weight of the cross-modal consistency constraint after being standardized by the Sigmoid function.

[0012] Preferably, determining the failure modes of each local diagnostic unit includes: In obtaining bimodal correlation Then, the failure mode of the local diagnostic unit is determined according to the preset threshold rules, with a high-confidence failure threshold. and low confidence failure threshold The settings are based on statistical analysis of historical data to ensure a balance between false positive and false negative rates. The judgment rules are as follows: If bimodal correlation ≥ High-confidence failure threshold Then determine the local diagnostic unit. The failure mode is "high confidence failure"; If the low reliability failure threshold ≤ Bimodal correlation High-Confidence Failure Threshold Then determine the local diagnostic unit. The failure mode is "low confidence failure"; If bimodal correlation Low-confidence failure threshold Then determine the local diagnostic unit. The failure mode is "normal".

[0013] Preferably, when the failure mode of the local diagnostic unit is a high-confidence failure, analysis using a spatial clustering algorithm based on graph connectivity reveals that the spatial expansion trend of high-confidence failures includes: For local diagnostic units identified as high-confidence failures, a bimodal correlation-weighted graph connectivity spatial clustering algorithm is introduced to analyze the spatial distribution characteristics of these failure units on the battery surface and assess whether the failure has a risk of propagation. The specific process includes three sub-steps: weighted spatial adjacency graph construction, spatial clustering based on weighted connected components, and spatial expansion trend assessment based on weighted clustering scale.

[0014] Preferably, the construction of the weighted spatial adjacency graph includes: Constructing a weighted undirected graph model ,in, This represents the set of nodes corresponding to all high-confidence failure units. The set representing edges; Represents the set of node weights; Represents the set of edge weights; For any two high-confidence failure units and When the spatial adjacency condition is met, an edge is established between the corresponding nodes. .

[0015] Preferably, the battery health diagnostic results generated by combining the failure modes of each local diagnostic unit with the spatial expansion trend of high-confidence failures include: The failure modes and spatial expansion trends of high-confidence failures in each local diagnostic unit are quantitatively integrated to form three key judgment indicators, including the proportion of high-confidence failure units. Proportion of low-reliability failure units and spatial expansion trend indicators ; Overall battery health status Based on the proportion of high-reliability failure units as a quantitative indicator Proportion of low-reliability failure units Spatial Expansion Trend Indicator Determined by rules; Generate battery health diagnostic results, including the following: Overall battery health status Local diagnostic units for high-confidence failures; proportion of high-confidence failure units. Proportion of low-reliability failure units Spatial Expansion Trend Indicator The specific value.

[0016] This invention provides a battery health diagnosis method based on multimodal image fusion, involving machine learning and deep learning technologies, which has the following beneficial effects: (1) This battery health diagnosis method based on multimodal image fusion uses the isolated forest algorithm to perform unsupervised anomaly detection on the extracted visible light and infrared thermal imaging feature vectors. This method can efficiently identify multidimensional features that deviate from the health baseline state. Compared with traditional methods that rely on fixed thresholds, this algorithm constructs a randomly divided decision tree forest and adaptively learns the distribution boundary of health data, thereby accurately locating potential early failure points.

[0017] (2) This battery health diagnosis method based on multimodal image fusion treats each high-confidence failure unit as a graph node, constructs connecting edges based on their spatial adjacency, and then uses a graph traversal algorithm to identify connected components. This method achieves quantitative analysis of the spatial distribution of failure modes. This algorithm can accurately aggregate discrete abnormal units into different spatial clusters, thereby intuitively revealing whether the failure is distributed as isolated points or has formed a locally connected region.

[0018] (3) A battery health diagnosis method based on multimodal image fusion introduces multimodal correlation as the weight of edges and nodes during spatial clustering. This method deeply integrates the spatial location information and severity information of failure. The weighting mechanism enables the algorithm to consider not only geometric connectivity but also the strength of the failure correlation between connected units when determining whether units belong to the same cluster. This significantly improves the recognition accuracy and focus on high-confidence abnormal clustering areas, enabling the diagnosis results to more keenly capture early high-risk lesions that are small in scale but highly correlated, thereby optimizing the priority of risk warning and supporting more accurate and forward-looking maintenance decisions. Attached Figure Description

[0019] Figure 1 This is a flowchart of a battery health diagnosis method based on multimodal image fusion proposed in this invention.

[0020] Figure 2 This invention provides a hierarchical graph of bimodal correlation degree for a battery health diagnosis method based on multimodal image fusion.

[0021] Figure 3 This is a hierarchical diagram of the battery health diagnosis results obtained in a battery health diagnosis method based on multimodal image fusion proposed in this invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Please see Figures 1-3 This invention provides a technical solution: a battery health diagnosis method based on multimodal image fusion. Specifically, the following battery health diagnosis method based on multimodal image fusion is provided; please refer to [link / reference]. Figure 1 The method includes the following steps: Step S1: Acquire visible light and infrared thermal images of the battery, collect prior battery data, divide the battery based on the prior battery data, and obtain local diagnostic units; This step aims to establish a precise and physically meaningful spatial basis for subsequent analysis. Its core logic is as follows: first, simultaneously acquire initial visible light and infrared thermal images of the battery, along with relevant prior data; then, ensure the two images are precisely aligned in space to obtain the battery image for formal analysis; finally, based on the prior data, divide the battery surface into multiple local diagnostic units with clear physical meaning.

[0024] Visible light images of the battery under test are simultaneously acquired from the same observation angle using a visible light camera and an infrared thermal imager. and initial infrared thermal imaging images Simultaneously, prior data of the tested battery are collected. The purpose of this action is to obtain multi-dimensional observation data that can be correlated in time and space, and to provide a basis for subsequent refined partitioning.

[0025] in, A two-dimensional pixel matrix characterizing the surface appearance of a battery; An initial two-dimensional pixel matrix characterizing the temperature distribution on the battery surface; It is a data set that includes the three-dimensional structural model of the battery, the precise location of the tabs and solder joints, and information on historically vulnerable areas.

[0026] Because the physical parameters and viewing angles of imaging devices may differ, it is necessary to examine the visible light images of the battery. and initial infrared thermal imaging images Spatial registration is performed. A feature-point-based registration method is adopted, which extracts significant feature points (such as battery corner points and edge intersections) from the two images, establishes the correspondence between feature points, and solves the optimal affine transformation model to map the initial infrared thermal image to the coordinate system of the initial visible light image. The registration process is achieved through the following affine transformation formula: ; in, Represents the initial image of infrared thermal imaging. The coordinates of the middle pixel; Represents the initial image of infrared thermal imaging. The corresponding coordinates in the registered image; These are matrix parameters that collectively determine rotation, scaling, and shear transformations; The vector parameters determine the translation transformation. The initial infrared thermal imaging image is then processed using the aforementioned affine transformation model. Perform a spatial transformation to make it compatible with the visible light image. Precise alignment was achieved in the pixel coordinate system. Through registration processing, a formal infrared thermal imaging image of the battery was obtained for subsequent analysis. .

[0027] After image registration, based on prior data The battery's three-dimensional structural model (which can be used to identify stress concentration areas), the precise locations of the tabs and solder joints, and information on historically degraded areas are used to divide the battery surface. During this division, the boundaries of stress concentration areas (such as casing edges and electrode connections) are identified; simultaneously, historically degraded areas are also treated as independent or key units, further dividing the battery surface into... A set of non-overlapping local diagnostic units. ,in, Represents the set of all local diagnostic units; This represents the total number of units. Each unit... Defined by the area of ​​image pixels it covers, it can be recorded in practice using the coordinates of the polygon vertices of that area.

[0028] For example: Suppose that, based on prior knowledge, the "positive tab connection area" on the battery surface is designated as an independent high-risk monitoring unit. In the registered visible light image In the pixel coordinate system, this region can be defined as a rectangle enclosed by four vertices. Its coordinate sequence can be recorded as: .in, to The pixel coordinates of the quadrilateral boundary of this unit in the image.

[0029] This independent strategy based on small-area segmentation aims to transform the overall state assessment of the battery into targeted monitoring of known high-risk areas, thereby improving the sensitivity of early local fault detection and the engineering interpretability of diagnosis.

[0030] Step S2: Through the local diagnostic unit, feature extraction is performed on the visible light image and infrared thermal imaging image of the battery to obtain the visible light feature vector and infrared thermal imaging feature vector of the battery. The pre-built cross-modal consistency constraint model is then called to generate cross-modal consistency constraints.

[0031] This step builds upon the spatial framework established in step S1. Its core task is to extract key features characterizing the health status of the battery from the visible light image and the infrared thermal image within each local diagnostic unit, and to apply a pre-built cross-modal consistency constraint model. Specifically, firstly, based on the registered image and segmentation results output from step S1, quantitative feature vectors are extracted from both types of images; then, a cross-modal consistency constraint model trained offline based on a large amount of historical health data is invoked to generate cross-modal consistency constraints. For each local diagnostic unit obtained in step S1 In the visible light image of the battery Infrared thermal imaging images of batteries The image patch corresponding to the unit region is cropped, and then visible light feature vectors reflecting its appearance integrity and surface texture are extracted from the visible light image patch. Infrared thermal imaging feature vectors reflecting temperature distribution and thermal gradient are extracted from infrared image patches. .

[0032] Visible light feature vector The extraction includes the following five texture and gradient statistical features: ; in, and These represent the mean and standard deviation of the local binary pattern texture features within the unit region, respectively, and are used to quantify the uniformity and roughness of the surface texture. and These represent the contrast and energy characteristics of the gray-level co-occurrence matrix, respectively, and are used to describe the local changes and uniformity of the gray-level distribution in an image, which can detect surface cracks or bulges. The mean of the histogram of oriented gradients is used to capture edge and shape information. The purpose of this feature vector is to transform visual information such as the appearance texture and structural regularity within a cell into a quantitative numerical description.

[0033] Infrared thermal imaging feature vector The extraction includes the following six temperature and gradient statistical features: ; in, These represent the highest temperature, lowest temperature, average temperature, and temperature standard deviation within a unit area, respectively, and are used to characterize the overall thermal level and uniformity of the area. and These represent the mean and standard deviation of the temperature gradient amplitude, respectively, and are used to quantify the severity of spatial variations in heat, serving as key indicators for identifying local hotspots or cooling anomalies. The purpose of this eigenvector is to transform thermophysical information such as the thermal state and uniformity of heat distribution within a cell into numerical descriptions of variables.

[0034] To establish the correlation between visible light characteristics and infrared thermal imaging characteristics, a cross-modal consistency constraint model needs to be pre-constructed. This model is based on a fundamental physical principle: under battery health conditions, there is a stable correspondence between specific physical appearance and specific thermal behavior characteristics. The construction of this model relies on statistical regularities learned from historical battery health data. Specifically, the battery historical health data refers to the visible light and infrared thermal imaging images of the battery collected periodically within its known healthy operating cycle. After processing in step S1 and feature extraction, the visible light feature vector and infrared thermal imaging feature vector pairs for each local diagnostic unit are obtained. These paired vectors constitute the basic dataset reflecting cross-modal correlations under healthy conditions, denoted as... ,in The total number of historical healthy samples, superscript Used for indexing the historical dataset One sample pair.

[0035] Based on the aforementioned historical battery health data, visible light characteristics were fitted using a multivariate statistical analysis method. To infrared thermal imaging features mapping relationship And determine its normal fluctuation range. Thus, a cross-modal consistency constraint model is constructed. This model can be expressed as: for a new, local diagnostic unit to be diagnosed... Features Calculate its residual vector Mahalanobis distance . This is the cross-modal consistency constraint, which should be less than a threshold determined based on the distribution of health data. .

[0036] Among them, the superscript is from arrive The transformation has a clear physical meaning. This represents an index of historical health data samples used to build the model; while This represents an index representing the currently diagnosed object (i.e., a specific local diagnostic unit). This shift marks the transition of analysis from the model training phase to the practical diagnostic application phase.

[0037] The residual vector and the Mahalanobis distance are calculated as follows: ; ; in, It utilizes historical health data The learned prediction function; It is all residual vectors on the historical health dataset. The mean vector; It is its covariance matrix; It is based on health data residuals The judgment threshold, determined by the statistical distribution, is set as a percentile (e.g., the 99th percentile) of the Mahalanobis distance of the residuals of all historical healthy samples, to control the false alarm rate within an allowable range of normal fluctuations. The purpose of the formula is to construct a quantitative statistical criterion for measuring the current diagnosed unit. The degree of consistency between the bimodal feature pairs and the feature association patterns learned from historical health states. If the bimodal characteristics of the unit are met, it is considered that the unit meets the health consistency constraint; otherwise, it is considered a cross-modal mismatch, indicating a potential anomaly.

[0038] It should be noted that the prediction function It is a regression model trained from historical health data through supervised learning. Its mathematical essence is learning a stable mapping relationship from the visible light feature space to the infrared thermal imaging feature space. Specifically, it utilizes a large-scale historical health dataset. With visible light feature vectors As input, the corresponding infrared thermal imaging feature vector The model is trained with the target output as the objective. The goal is to improve the model's predictions. As close as possible to the real infrared thermal imaging feature vector .

[0039] The output of this step includes each current local diagnostic unit. Dual-modal eigenvectors and a quantifiable cross-modal consistency constraint corresponding to the unit. This provides the core data foundation and judgment criteria for the next step of failure correlation analysis.

[0040] Step S3: Based on the visible light feature vector, infrared thermal imaging feature vector and cross-modal consistency constraint, calculate the bimodal correlation degree of each local diagnostic unit and determine the failure mode of each local diagnostic unit.

[0041] This step builds upon the bimodal feature vectors and cross-modal consistency constraints of each local diagnostic unit output from step S2. Its core task is to quantify the correlation between visible light and infrared thermal imaging feature anomalies within each local diagnostic unit and determine the unit-level failure mode accordingly. By introducing the isolated forest algorithm, the distribution boundaries of each feature in a healthy state can be effectively characterized, thereby achieving unsupervised learning and quantification of anomaly correlations.

[0042] For each local diagnostic unit Its bimodal correlation The calculation is performed through the following process: First, a baseline model of visible light and infrared features is trained based on historical health data; second, the anomaly score of the current local diagnostic unit's features relative to the health baseline is calculated; finally, weighted fusion is performed by combining cross-modal consistency.

[0043] From historical data, dual-modal image samples of batteries in a healthy state were selected, and the visible light feature vector sets of each local diagnostic unit were extracted. and infrared thermal imaging feature vector set The isolated forest algorithm was used to train visible light feature anomaly detection models. Infrared feature anomaly detection model .

[0044] Isolation forests isolate samples by constructing multiple binary trees. For samples containing... The training set consists of healthy samples. The goal of model training is to build a tree so that abnormal samples can be isolated more quickly (i.e., the path length from the root node to the leaf node is [length]). (Shorter). Anomaly score of the sample. Determined by the expected value of its path length: ; in, It is a sample The average path length across all trees in an isolated forest; Given a number of samples The path length is standardized to adjust the result range. For healthy samples, the path length is usually large, hence the abnormal score... Close to 0; for anomalous samples, the path length is shorter. Close to 1.

[0045] The purpose of training this model is to learn the spatial distribution of normal features from health data and to establish a detection model that can quantify the degree to which new samples deviate from the health baseline.

[0046] For the local diagnostic unit to be diagnosed Its visible light feature vector and infrared thermal imaging feature vectors Input them into the trained model respectively. and Calculate its abnormal score: ; ; in, Represents a local diagnostic unit The visible light single-mode anomaly score, the higher the score, the greater the probability of an anomaly; Represents a local diagnostic unit The infrared single-mode anomaly score is calculated. The purpose of this calculation is to transform the anomalies in the high-dimensional feature vector into scalar, comparable anomaly scores.

[0047] The single-modal anomaly score is compared with the cross-modal consistency constraint obtained in step S2. Weighted fusion is performed to obtain the final bimodal correlation degree. : ; in, Represents a local diagnostic unit The bimodal correlation degree has a range of [0,1]; It is the Sigmoid function, used to smoothly map input values ​​to the [0,1] interval; These are the weights of the visible light single-mode anomaly scores after standardization using the Sigmoid function; The weights are the infrared single-mode anomaly scores after standardization using the Sigmoid function. The weights of the cross-modal consistency constraints are standardized using the Sigmoid function and satisfy the following conditions: Based on prior knowledge from domain experts, the reliability or importance of the indicator is assessed and set. The purpose of the formula is to synthesize the performance of the local diagnostic unit in two dimensions—single-modal feature anomaly and cross-modal feature consistency—to form a unified and quantifiable index, which is used to characterize the overall confidence level of the unit's actual failure.

[0048] In obtaining bimodal correlation Then, the failure mode of the local diagnostic unit is determined according to preset threshold rules. High-confidence failure threshold. and low confidence failure threshold The settings are based on statistical analysis of historical data (such as ROC curve analysis) to ensure a balance between false positive and false negative rates. The judgment rules are as follows: If bimodal correlation ≥ High-confidence failure threshold Then determine the local diagnostic unit. The failure mode is "high-confidence failure". This mode indicates that the unit exhibits significant anomalies in multiple physical characteristics, and the anomalies are consistent, suggesting a very high probability of a real health problem.

[0049] If the low reliability failure threshold ≤ Bimodal correlation High-Confidence Failure Threshold Then determine the local diagnostic unit. The failure mode is "low confidence failure". This mode indicates that the unit is exhibiting some abnormal signs, but the evidence is insufficient. It may be due to occasional interference or early weak signs and requires continuous monitoring.

[0050] If bimodal correlation Low-confidence failure threshold Then determine the local diagnostic unit. The failure mode is "normal".

[0051] Based on this, all local diagnostic units are classified as: a set of high-confidence failure units. Low-reliability failure unit set And the normal unit set.

[0052] Where the threshold satisfies , can be set , The purpose of this determination is to transform continuous correlation values ​​into discrete failure mode labels with clear engineering significance, providing clear input for subsequent spatial expansion analysis.

[0053] Step S4: When the failure mode of the local diagnostic unit is a high-confidence failure, the spatial expansion trend of the high-confidence failure is obtained by analyzing it using a spatial clustering algorithm based on graph connectivity. This step targets the local diagnostic units identified as "high-confidence failures" in step S3. By introducing a bimodal correlation-weighted graph connectivity spatial clustering algorithm, it analyzes the spatial distribution characteristics of these failure units on the battery surface and assesses whether the failures have a risk of spreading. The specific process includes three sub-steps: weighted spatial adjacency graph construction, spatial clustering based on weighted connected components, and spatial expansion trend assessment based on the weighted clustering scale.

[0054] For constructing a weighted spatial adjacency graph, first construct a weighted undirected graph model. ,in: This represents the set of nodes corresponding to all high-confidence failure units. The set of edges represents the spatial adjacency relationship between units; Represents the set of node weights. For nodes The weights; Represents the set of edge weights. For the edge The weight.

[0055] For any two high-confidence failure units and When the spatial adjacency condition is met (boundary adjacency or geometric center distance less than a threshold), When ), establish edges between the corresponding nodes. The purpose of this step is to establish a weighted graph model that includes both topological connectivity and failure confidence information.

[0056] For spatial clustering based on weighted connected components, an improved weighted depth-first search algorithm is used for connected component identification. The execution process of this algorithm is as follows: First, initialize an array of visit markers called `visited`, whose length is the total number of high-confidence failed units. Then, all elements are set to the logical value "false", indicating that all nodes have not been visited. At the same time, the cluster counter count is initialized to zero.

[0057] Then, traverse all nodes. For each node in the graph... Check its access flag. If the flag is "false", create a new empty cluster. Next, using this node... and clustering Starting from this point, a weighted depth-first search process is invoked. After this process is completed, Add to clustering result set In, and cluster counter The value increases by one ( That is, in the final result ).

[0058] The core of the weighted depth-first search process is to recursively explore neighboring nodes connected to the current node that satisfy specific conditions. Specifically, when considering a node... and clustering When executing this process, first set the node Add to cluster In the middle, update its access flag to "true". Then, traverse the nodes. All neighboring nodes For each neighbor node Only if its access flag is "false" and the connected node and edge weight Greater than or equal to the preset edge weight threshold Only then, based on nodes and clustering The weighted depth-first search process is recursively called with the parameter .

[0059] The visited array is used to record the visit status of each node to prevent duplicate processing. This represents the count-th cluster discovered by the algorithm. Represents nodes The set of spatially adjacent nodes. Edge weight threshold. It is a preset parameter used to ensure that only adjacent units with high correlation are classified into the same cluster, thereby achieving spatial clustering partitioning that takes into account the connection strength.

[0060] For assessing the spatial expansion trend based on weighted cluster size, after obtaining the clustering results... Then, calculate the weighted density and size of each cluster: ; ; ; in, Representative clustering The weighted density; Representative clustering The number of nodes; Represents the largest size among all clusters; represents the maximum weighted density; k is the cluster index. The purpose of the formula is to quantitatively evaluate the size and quality characteristics of the clusters.

[0061] Determine the spatial expansion trend based on the calculation results: when When, it is determined to be an isolated trend; when or When, it is determined to be a local diffusion trend; when At that time, it was determined to be a widespread expansion trend. Among them, This represents the threshold for cluster size; This represents the weighted density threshold. The purpose of this determination is to accurately identify the risk level of failure propagation based on clustering characteristics.

[0062] This step enables a refined analysis of the spatial distribution characteristics of high-confidence failure units, providing a reliable spatial dimension basis for the final health status assessment. The introduction of a weighting mechanism significantly enhances the ability of cluster analysis to identify early risk focus areas.

[0063] Step S5: Combine the failure modes of each local diagnostic unit with the spatial expansion trend of high-confidence failures to generate battery health diagnostic results.

[0064] This step integrates the unit-level failure mode determination results from step S3 and the failure space expansion trend evaluation results from step S4. Through preset multi-level decision rules, it classifies and determines the overall health status of the battery and generates a battery health diagnosis result with detailed evidence.

[0065] The failure modes and spatial expansion trends of high-confidence failures in each local diagnostic unit are quantitatively integrated to form three key judgment indicators, including the proportion of high-confidence failure units. Proportion of low-reliability failure units and spatial expansion trend indicators ; The proportion of high-confidence failure units The calculation formula is: ; The proportion of low-confidence failure units The calculation formula is: ; in, This indicates the number of units that were determined to be high-confidence failures. This indicates the number of units that were judged as low-confidence failures. This represents the total number of units. The purpose of this quantification is to unify health evidence from different dimensions into comparable numerical indicators, providing an objective basis for grading.

[0066] At the same time, the spatial expansion trend will be quantified into a spatial expansion trend indicator. Isolation trend is assigned a value of 1, local diffusion trend is assigned a value of 2, and widespread expansion trend is assigned a value of 3.

[0067] Overall battery health status Based on the proportion of high-reliability failure units as a quantitative indicator Proportion of low-reliability failure units Spatial Expansion Trend Indicator Determined by rules: Regarding health status, when and When established, the corresponding engineering meaning is that the battery shows no obvious abnormal signs and is in a stable operating state; For a state of attention, if and only if and When this condition is established, it corresponds to the existence of isolated anomalies in the project, requiring enhanced monitoring.

[0068] For abnormal states, when or When this condition is established, the corresponding engineering meaning is that a significant anomaly has occurred, requiring planned maintenance.

[0069] In dangerous situations, when or When it is established, the corresponding engineering meaning is that the risk of abnormal spread is high and it needs to be dealt with immediately.

[0070] The percentage thresholds (5%, 10%, 20%) in the judgment criteria can be adjusted according to the sensitivity requirements of specific application scenarios. The purpose of this grading rule is to transform complex multi-source information into a concise and clear health level, making it easier for maintenance personnel to quickly understand the battery status and take appropriate measures.

[0071] Based on the above judgment results, a battery health diagnosis result is generated, including the following: Overall battery health status (Health / Attention / Abnormal / Danger); Local diagnostic unit of high-confidence failure (the location of this unit is marked on the battery visible light image); Proportion of high-confidence failure units Proportion of low-reliability failure units Spatial Expansion Trend Indicator The specific value.

[0072] This step completes the step-by-step reasoning from the failure modes and spatial trends of local diagnostic units to the overall health status, and finally outputs diagnostic conclusions that combine quantitative accuracy and engineering interpretability, providing direct decision support for predictive maintenance of batteries.

[0073] This invention proposes a battery health diagnosis method based on multimodal image fusion, aiming to solve the problems of existing technologies such as insensitivity to early and localized battery health problems, high false positive rates, and poor interpretability of diagnostic results. The core idea of ​​this method is to abandon the traditional holistic, single-dimensional analysis mode and construct a progressively refined diagnostic framework that progresses from local features to cross-modal correlations, then to spatial trends, and finally to health diagnosis.

[0074] The method is implemented through five logically rigorous steps. First, based on prior knowledge of the battery, structural mapping is performed to divide the battery surface into local diagnostic units with clear physical meaning, laying a spatial foundation for subsequent analysis. Second, within each unit, the appearance texture features of visible light images and the temperature distribution features of infrared thermal images are extracted simultaneously, and a pre-constructed cross-modal consistency constraint model is invoked to generate cross-modal consistency constraints, establishing the inherent correlation rules between different physical representations under healthy conditions. The key innovation lies in the subsequent steps: by introducing the isolated forest algorithm to calculate the multimodal correlation degree of each unit, a comprehensive confidence assessment of anomalies is achieved, avoiding misjudgment based on a single mode; furthermore, for high-confidence failure units, a spatial clustering algorithm based on graph connectivity is innovatively applied, and a correlation degree weighting mechanism is introduced to achieve a leap analysis from "point anomaly" to "area risk," enabling accurate identification of the failure propagation trend; finally, by integrating unit-level failure modes and spatial expansion trends, the overall health status is output according to clear hierarchical rules, making the diagnostic results both quantitatively accurate and engineering interpretable.

[0075] The main advantages of this invention are as follows: First, through collaborative analysis and consistency constraints of multimodal images, the impact of environmental interference and occasional anomalies is significantly reduced, improving the reliability of identifying early and hidden failures. Second, through localized analysis and spatial trend judgment, the diagnostic logic is made more consistent with the physical law of battery failure originating locally and expanding to the surrounding areas. This not only assesses the current state but also has the potential for predictive maintenance. Third, the entire method is clear, the diagnostic basis is transparent, and the results are easily understood and trusted by maintenance personnel, providing effective technical support for the safe, stable, and long-term operation of energy storage batteries, and possessing broad engineering application prospects.

[0076] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the statement "including a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0077] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their likenesses.

Claims

1. A battery health diagnosis method based on multimodal image fusion, characterized in that, Includes the following steps: Step S1: Acquire visible light and infrared thermal images of the battery, collect prior battery data, divide the battery based on the prior battery data, and obtain local diagnostic units; Step S2: Using the local diagnostic unit, feature extraction is performed on the visible light image and infrared thermal imaging image of the battery to obtain the visible light feature vector and infrared thermal imaging feature vector of the battery. A pre-built cross-modal consistency constraint model is then invoked to generate cross-modal consistency constraints. The generation of cross-modal consistency constraints by invoking the pre-built cross-modal consistency constraint model includes: To establish the correlation between visible light features and infrared thermal imaging features, a cross-modal consistency constraint model needs to be constructed in advance. This model is based on a fundamental physical principle: under the battery health condition, there is a stable correspondence between appearance and thermal behavior features. The construction of this model depends on the statistical regularities learned from the battery's historical health data. Specifically, the battery historical health data refers to the periodic acquisition of visible light and infrared thermal imaging images of the battery within its known healthy operating cycle. After processing in step S1 and feature extraction, the visible light feature vector and infrared thermal imaging feature vector pairs of each local diagnostic unit are obtained. These paired vectors constitute the basic dataset reflecting cross-modal correlations under healthy conditions, denoted as... ,in The total number of historical healthy samples, superscript Used for indexing the historical dataset One sample pair; Based on the aforementioned historical battery health data, visible light characteristics were fitted using a multivariate statistical analysis method. To infrared thermal imaging features mapping relationship And determine its normal fluctuation range, thereby constructing a cross-modal consistency constraint model, which can be expressed as: for a new, local diagnostic unit to be diagnosed... Features Calculate its residual vector Mahalanobis distance , This is the cross-modal consistency constraint, which should be less than a threshold determined based on the distribution of health data. ; Step S3: Based on the visible light feature vector, infrared thermal imaging feature vector and cross-modal consistency constraint, calculate the bimodal correlation degree of each local diagnostic unit and determine the failure mode of each local diagnostic unit; Step S4: When the failure mode of the local diagnostic unit is a high-confidence failure, the spatial expansion trend of the high-confidence failure is obtained through analysis using a spatial clustering algorithm based on graph connectivity; when the failure mode of the local diagnostic unit is a high-confidence failure, the spatial expansion trend of the high-confidence failure is obtained through analysis using a spatial clustering algorithm based on graph connectivity, including: For local diagnostic units identified as high-confidence failures, a bimodal correlation-weighted graph connectivity spatial clustering algorithm is introduced to analyze the spatial distribution characteristics of these failure units on the battery surface and assess whether the failure has a risk of diffusion. The specific process includes three sub-steps: weighted spatial adjacency graph construction, spatial clustering based on weighted connected components, and spatial expansion trend assessment based on weighted clustering scale. The construction of the weighted spatial adjacency graph includes: Constructing a weighted undirected graph model ,in, This represents the set of nodes corresponding to all high-confidence failure units. The set representing edges; Represents the set of node weights; Represents the set of edge weights; For any two high-confidence failure units and When the spatial adjacency condition is met, an edge is established between the corresponding nodes. ; Step S5: Combine the failure modes of each local diagnostic unit with the spatial expansion trend of high-confidence failures to generate battery health diagnostic results.

2. The battery health diagnosis method based on multimodal image fusion according to claim 1, characterized in that, Acquire visible light and infrared thermal images of the battery, including: Visible light images of the battery under test are simultaneously acquired from the same observation angle using a visible light camera and an infrared thermal imager. and initial infrared thermal imaging images ; Visible light image of the battery and initial infrared thermal imaging images Spatial registration is performed using a feature-point-based registration method. This involves extracting salient feature points from the two images, establishing the correspondence between these feature points, and solving for the optimal affine transformation model to map the initial infrared thermal image to the visible light image coordinate system. The registration process is achieved using the following affine transformation formula: ； in, Represents the initial image of the original infrared thermal imaging. The coordinates of the middle pixel; Represents the initial image of infrared thermal imaging. The corresponding coordinates in the registered image; These are matrix parameters that collectively determine rotation, scaling, and shear transformations; The vector parameters determine the translation transformation; After registration, an infrared thermal image of the battery was obtained for subsequent analysis. .

3. The battery health diagnosis method based on multimodal image fusion according to claim 2, characterized in that, Collect prior battery data, segment the battery based on the prior battery data, and obtain local diagnostic units, including: Collect prior data of the battery under test. It is a data set that includes the three-dimensional structural model of the battery, the precise location of the tabs and solder joints, and information on historically vulnerable areas. After image registration, based on prior data The battery's three-dimensional structural model, the precise locations of the tabs and solder joints, and information on historically degraded areas were used to divide the battery surface. During this division, the boundaries of stress concentration areas were identified. Simultaneously, historically degraded areas were also treated as independent or key units, further dividing the battery surface into... A set of non-overlapping local diagnostic units. ,in, Represents the set of all local diagnostic units; This represents the total number of units.

4. The battery health diagnosis method based on multimodal image fusion according to claim 3, characterized in that, The local diagnostic unit extracts features from the visible light image and infrared thermal imaging image of the battery to obtain the visible light feature vector and infrared thermal imaging feature vector of the battery, including: For each local diagnostic unit obtained in step S1 In the visible light image of the battery Infrared thermal imaging images of batteries The image patch corresponding to the unit region is cropped, and then visible light feature vectors reflecting its appearance integrity and surface texture are extracted from the visible light image patch. Infrared thermal imaging feature vectors reflecting temperature distribution and thermal gradient are extracted from infrared thermal imaging image patches. .

5. A battery health diagnosis method based on multimodal image fusion according to claim 4, characterized in that, Based on the visible light feature vector, infrared thermal imaging feature vector, and cross-modal consistency constraints, the bimodal correlation degree of each local diagnostic unit is calculated, including: Combining unimodal anomaly scores with crossmodal consistency constraints Weighted fusion is performed to obtain the final bimodal correlation degree. : ； in, Represents a local diagnostic unit The bimodal correlation degree has a range of [0,1]; It is the Sigmoid function, used to smoothly map input values ​​to the [0,1] interval; These are the weights of the visible light single-mode anomaly scores after standardization using the Sigmoid function; It is the weight of the single-modal anomaly score in infrared thermal imaging after standardization by the Sigmoid function; It is the weight of the cross-modal consistency constraint after being standardized by the Sigmoid function.

6. The battery health diagnosis method based on multimodal image fusion according to claim 5, characterized in that, Determine the failure modes of each local diagnostic unit, including: In obtaining bimodal correlation Then, the failure mode of the local diagnostic unit is determined according to the preset threshold rules, with a high-confidence failure threshold. and low confidence failure threshold The settings are based on statistical analysis of historical data to ensure a balance between false positive and false negative rates. The judgment rules are as follows: If bimodal correlation High-confidence failure threshold Then determine the local diagnostic unit. The failure mode is "high confidence failure"; If the low reliability failure threshold Bimodal correlation High-confidence failure threshold Then determine the local diagnostic unit. The failure mode is "low confidence failure"; If bimodal correlation Low confidence failure threshold Then determine the local diagnostic unit. The failure mode is "normal".

7. The battery health diagnosis method based on multimodal image fusion according to claim 6, characterized in that, By combining the failure modes of each local diagnostic unit with the spatial expansion trend of high-confidence failures, a battery health diagnostic result is generated, including: The failure modes and spatial expansion trends of high-confidence failures in each local diagnostic unit are quantitatively integrated to form three key judgment indicators, including the proportion of high-confidence failure units. Proportion of low-reliability failure units and spatial expansion trend indicators ; Overall battery health status Based on the proportion of high-reliability failure units as a quantitative indicator Proportion of low-reliability failure units Spatial Expansion Trend Indicator Determined by rules; Generate battery health diagnostic results, including the following: Overall battery health status Local diagnostic units for high-confidence failures; proportion of high-confidence failure units. Proportion of low-reliability failure units Spatial Expansion Trend Indicator The specific value.

Images