Charging pile fault detection method and system based on image recognition

Abstract

The invention relates to the technical field of charging piles, in particular to a charging pile fault detection method and system based on image recognition, and the method comprises the steps: inputting a multi-modal time sequence image into a multi-scale feature extraction network, and obtaining a multi-modal fusion feature; constructing a dynamic thermodynamic diagram of the target charging pile based on the multi-modal fusion features, and performing mask processing on the multi-modal fusion features through the dynamic thermodynamic diagram to obtain fault sensitive area features; inputting the fault sensitive area features into a space-time diagram neural network to obtain a fault evolution feature sequence; and reasoning the fault evolution feature sequence based on a preset fault causal reasoning graph to obtain fault root cause features. According to the method, the fault sensitive area features are constructed through the dynamic thermodynamic diagram, and a potential thermal anomaly area can be effectively identified, so that a fault source is positioned more quickly, and the fault response speed is improved.

Description

Technical Field

[0001] This invention relates to the field of charging pile technology, specifically to a charging pile fault detection method and system based on image recognition. Background Technology

[0002] Charging stations are devices used to provide power to electric vehicles (EVs). They are typically installed in public places, parking lots, residential areas, or commercial buildings to charge the batteries of electric vehicles. Charging stations involve high voltage and high current, and any malfunction can lead to electrical safety hazards such as short circuits, overheating, or fires. Timely detection and handling of malfunctions can effectively prevent accidents and ensure the safety of users and equipment.

[0003] Currently, many traditional methods rely solely on single physical parameters (such as current and voltage) for fault detection, lacking comprehensive analysis of complex fault conditions, which can easily lead to missed or false detections. Furthermore, traditional methods often focus on static data analysis, failing to effectively capture the dynamic changes during the fault evolution process, resulting in insufficient judgment of the fault development trend and an inability to provide early warnings. In addition, in identifying fault-sensitive areas, traditional methods typically lack sufficient spatial resolution, leading to inaccurate fault source location and increased maintenance time and costs.

[0004] Furthermore, traditional methods typically rely on fixed rules and thresholds for fault detection, lacking the ability to learn and adapt, and are unable to self-optimize based on actual operating conditions and environmental changes. In addition, traditional alarm systems usually adopt simple threshold triggering mechanisms, failing to differentiate based on the type and severity of the fault, which may lead to frequent false alarms or missed alarms. Summary of the Invention

[0005] To achieve the above objectives, the present invention provides the following technical solution: a charging pile fault detection method based on image recognition, comprising: Acquire multimodal time-series images of the target charging pile within a preset detection period; input the multimodal time-series images into a multi-scale feature extraction network to obtain multimodal fusion features; A dynamic heat map of the target charging pile is constructed based on multimodal fusion features. The multimodal fusion features are then masked using the dynamic heat map to obtain fault-sensitive area features. The fault-sensitive area features are the areas in the multimodal fusion features where the thermal radiation value is greater than a preset temperature threshold and the texture gradient is greater than a preset gradient threshold. The features of the fault-sensitive region are input into the spatiotemporal graph neural network to obtain the fault evolution feature sequence; the fault evolution feature sequence is used to characterize the changing trend and spatial correlation of the features of the fault-sensitive region in the temporal dimension. The fault evolution feature sequence is inferred based on the pre-set fault causal reasoning graph to obtain the root cause features of the fault; wherein, the fault causal reasoning graph is pre-constructed based on the internal electrical topology of the charging pile and the fault propagation path; The fault-sensitive area features, fault evolution feature sequences, and fault root cause features are fused to obtain comprehensive fault features. The comprehensive fault features are then input into the fault detection head to obtain the target detection results output by the fault detection head. The target detection results include fault type, fault location, fault severity level, and maintenance recommendations.

[0006] Preferably, the multimodal temporal images include visible light image sequences and infrared thermal imaging sequences; the multimodal fusion features include visible light texture features and infrared thermal radiation features; the multi-scale feature extraction network includes a visible light feature extraction branch, an infrared feature extraction branch, and a cross-modal attention fusion module; Multimodal temporal images are input into a multi-scale feature extraction network to obtain multimodal fusion features, including: The visible light image sequence is input into the visible light feature extraction branch to extract the first-level, second-level, and third-level features; the infrared thermal imaging sequence is input into the infrared feature extraction branch to extract the corresponding level of thermal features; Based on the cross-modal attention fusion module, visible light features are used as query vectors and infrared thermal features are used as key and value vectors to calculate the attention weights between visible light features and infrared thermal features. The infrared thermal features are weighted and summed based on attention weights, and the weighted summation result is residually connected with the visible light features to obtain multimodal fusion features.

[0007] Preferably, the multimodal fusion features are masked using dynamic heatmaps to obtain fault-sensitive region features, including: Based on the dynamic heat map, the temperature anomaly focusing area is determined, and a binary mask matrix matching the feature size of the multimodal fusion is generated; Multimodal fusion features are multiplied element-wise based on a binary mask matrix to remove background interference features and obtain fault-sensitive region features.

[0008] Preferably, the spatiotemporal graph neural network includes a node embedding layer and a graph convolutional layer; The features of the fault-sensitive region are input into a spatiotemporal graph neural network to obtain a fault evolution feature sequence, including: The features of the fault-sensitive area are flattened into feature vectors, and the feature vectors are mapped into graph nodes based on the node embedding layer to construct the charging pile fault graph structure. Temporal edges are constructed based on the temporal sequence of fault-sensitive area characteristics, and spatial edges are constructed based on the adjacency relationships of various components in the physical structure of the charging pile. The receptive field of the graph convolution kernel is defined by temporal and spatial edges. The features of the graph node at the current time and the historical graph node in the spatiotemporal neighborhood are aggregated and updated to aggregate the fault features of adjacent time and adjacent regions, so as to obtain the fault evolution feature sequence.

[0009] Preferably, the fault evolution feature sequence is inferred based on a preset fault causal reasoning graph to obtain fault root cause features, including: The fault evolution feature sequence is input as evidence node into the fault causal reasoning graph; Based on the conditional probability table in the fault causal reasoning graph, calculate the posterior probability of each potential fault root cause node. The feature vector corresponding to the potential root cause node with the highest posterior probability is selected as the root cause feature.

[0010] Preferably, the fault detection head includes a classification branch, a localization branch, and a regression branch; The comprehensive fault characteristics are input into the fault detection head to obtain the target detection results output by the fault detection head, including: The comprehensive fault features are input into the classification branch, the localization branch, and the regression branch, respectively. The probability distribution of fault type is output based on the classification branch, the coordinate offset of fault location is output based on the localization branch, and the continuous value of fault severity level is output based on the regression branch. The fault type with the highest confidence is selected based on the probability distribution of fault type, and the fault location is determined by combining the coordinate offset. Pre-set maintenance suggestions are matched according to the continuous values ​​of severity level to obtain the target detection result.

[0011] Preferably, the method further includes a model training step: A charging pile fault sample library is constructed, which includes normal samples, single fault samples, and compound fault samples. The samples in the charging pile fault sample library are processed to obtain the predicted detection results; The classification loss, localization loss, and severity regression loss are calculated based on the difference between the predicted detection results and the true labels. End-to-end backpropagation optimization is performed on the multi-scale feature extraction network, spatiotemporal graph neural network, and fault detection head using the total loss function.

[0012] Preferably, the process of constructing the total loss function includes: Obtain the classification loss function, localization loss function, and severity regression loss function; A fault causal consistency regularization term is introduced to constrain the logical consistency between the fault evolution feature sequence and the fault root cause features; The classification loss function, localization loss function, severity level regression loss function, and fault causality consistency regularization term are weighted and summed to construct the total loss function.

[0013] Preferably, after inputting the comprehensive fault features into the fault detection head and obtaining the target detection result output by the fault detection head, the method further includes: Obtain the fault type and fault severity level from the target detection results; If the fault type is insulation damage and the severity level is greater than the preset level threshold, a level one alarm signal will be generated and the charging pile charging function will be locked. If the fault type is loose charging head and the severity level is less than the preset level threshold, a level 2 prompt signal will be generated and charging will continue but the charging power will be reduced. The alarm signal or prompt signal is associated with the target detection result and stored, and then uploaded to the cloud management platform.

[0014] A charging pile fault detection system based on image recognition, applicable to the aforementioned charging pile fault detection method based on image recognition, includes: The image acquisition unit is used to acquire multimodal time-series images of the target charging pile within a preset detection period; the multimodal time-series images are input into a multi-scale feature extraction network to obtain multimodal fusion features; The fault feature unit is used to construct a dynamic heat map of the target charging pile based on multimodal fusion features. The multimodal fusion features are masked by the dynamic heat map to obtain fault-sensitive area features. The fault-sensitive area features are the area features in the multimodal fusion features where the thermal radiation value is greater than a preset temperature threshold and the texture gradient is greater than a preset gradient threshold. The fault evolution unit is used to input the features of the fault-sensitive region into the spatiotemporal graph neural network to obtain the fault evolution feature sequence; wherein, the fault evolution feature sequence is used to characterize the changing trend and spatial correlation of the features of the fault-sensitive region in the temporal dimension; The fault causal unit is used to reason about the fault evolution feature sequence based on a preset fault causal reasoning graph to obtain the root cause features of the fault; wherein, the fault causal reasoning graph is pre-constructed based on the internal electrical topology of the charging pile and the fault propagation path; The result detection unit is used to fuse the fault-sensitive area features, fault evolution feature sequence, and fault root cause features to obtain comprehensive fault features; the comprehensive fault features are input to the fault detection head to obtain the target detection results output by the fault detection head, wherein the target detection results include fault type, fault location, fault severity level, and maintenance suggestions.

[0015] Compared with the prior art, the beneficial effects of the present invention are: This invention constructs fault-sensitive area features using dynamic heat maps, effectively identifying potential thermal anomaly areas, thereby locating the fault source more quickly and improving fault response speed. Furthermore, the application of spatiotemporal neural networks ensures that fault features not only consider spatial correlations but also reflect trends over time, enhancing understanding of the fault evolution process and aiding in early fault prediction. Finally, the fault causal reasoning graph allows for inference about fault causes, enabling in-depth analysis of complex fault situations, providing more scientific root cause judgments, and facilitating the development of targeted maintenance solutions. This invention, through the classification, localization, and regression branch design of the fault detection head, enables the system to evaluate faults from multiple aspects, ensuring the comprehensiveness and reliability of the detection results. Furthermore, it automatically generates alarm signals based on the detection results and takes different handling measures according to the fault type and severity level, effectively ensuring the safe use of charging piles and preventing accidents. Moreover, by constructing a charging pile fault sample library and training the model, the system's self-learning ability and adaptability are improved, and the detection performance is continuously optimized as the amount of data increases. Attached Figure Description

[0016] Figure 1 This is a schematic flowchart of the overall method in one embodiment of the present invention; Figure 2 This is a schematic diagram of the overall system architecture in one embodiment of the present invention.

[0017] In the figure: 1. Image acquisition unit; 2. Fault feature unit; 3. Fault evolution unit; 4. Fault cause and effect unit; 5. Result detection unit. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1, please refer to Figure 1 This invention provides a technical solution: a charging pile fault detection method based on image recognition, comprising: S1. Acquire multimodal time-series images of the target charging pile within a preset detection period; input the multimodal time-series images into a multi-scale feature extraction network to obtain multimodal fusion features; S2. Construct a dynamic heat map of the target charging pile based on multimodal fusion features. Mask the multimodal fusion features through the dynamic heat map to obtain fault-sensitive area features. Among them, the fault-sensitive area features are the area features in the multimodal fusion features where the thermal radiation value is greater than a preset temperature threshold and the texture gradient is greater than a preset gradient threshold. S3. Input the features of the fault-sensitive area into the spatiotemporal graph neural network to obtain the fault evolution feature sequence; wherein, the fault evolution feature sequence is used to characterize the changing trend and spatial correlation of the features of the fault-sensitive area in the temporal dimension; S4. Based on the preset fault causal reasoning graph, reason about the fault evolution feature sequence to obtain the fault root cause features; wherein, the fault causal reasoning graph is pre-constructed based on the internal electrical topology of the charging pile and the fault propagation path; S5. The fault-sensitive area features, fault evolution feature sequences, and fault root cause features are fused to obtain comprehensive fault features. The comprehensive fault features are input into the fault detection head to obtain the target detection results output by the fault detection head. The target detection results include fault type, fault location, fault severity level, and maintenance suggestions.

[0020] It should be noted that within the preset detection period, the system will collect multimodal time-series images of the charging pile; these images may include visible light images, infrared thermal images, and other sensor data; for example, at different times of day, the system may capture images of the charging pile under normal operating conditions and thermal images under high temperature environments or excessive loads; inputting these multimodal images into a multi-scale feature extraction network can extract important features from the images; these features may be the temperature distribution, color changes, or other visual features on the surface of the charging pile. By fusing features extracted from different modalities, the system generates a dynamic heatmap that displays the status of charging piles at different points in time. For example, if the temperature of a charging pile exceeds the normal range at a certain time, the heatmap will clearly identify this abnormal area. Based on the dynamic heatmap, the system performs masking processing on the multimodal fusion features to extract fault-sensitive area features. These features typically refer to areas where the thermal radiation value is higher than a preset threshold (e.g., the temperature exceeds 70 degrees Celsius) and the texture changes significantly. These fault-sensitive area features are input into a spatiotemporal graph neural network to obtain a fault evolution feature sequence. This sequence describes the changes of these sensitive area features over time and their spatial correlation. For example, if the temperature of a certain area has gradually increased over the past few hours, this indicates a potential fault. Based on a pre-defined fault causal reasoning graph, which is constructed based on the electrical structure of the charging pile and the fault propagation path, the system uses the fault evolution feature sequence to infer the root cause of the fault. For example, the system may infer that the high temperature is caused by an internal circuit short circuit. The fault-sensitive area features, fault evolution feature sequences, and fault root cause features are fused to form a comprehensive fault feature. This step integrates all relevant information and provides a comprehensive understanding of the fault. Finally, these comprehensive fault features are input into the fault detection head to generate target detection results, including fault type (such as circuit short circuit), fault location (such as specific cable or connector), fault severity level (such as high, medium, low), and maintenance suggestions (such as component replacement or emergency shutdown). For example, suppose that during the monitoring of a charging pile, the system detects that the thermal imaging image shows that the temperature in a certain area is rising continuously over a certain period of time. Combined with the analysis of the spatiotemporal neural network, it is finally determined that this is due to a short circuit caused by cable aging. The system will output the fault type as "cable short circuit", the fault location as "the left cable of the charging pile", the severity level as "high", and recommend that maintenance measures be taken immediately to prevent equipment damage or safety accidents.

[0021] In an optional embodiment, the multimodal temporal image includes a visible light image sequence and an infrared thermal imaging sequence; the multimodal fusion features include visible light texture features and infrared thermal radiation features; the multi-scale feature extraction network includes a visible light feature extraction branch, an infrared feature extraction branch, and a cross-modal attention fusion module; Multimodal temporal images are input into a multi-scale feature extraction network to obtain multimodal fusion features, including: The visible light image sequence is input into the visible light feature extraction branch to extract the first-level, second-level, and third-level features; the infrared thermal imaging sequence is input into the infrared feature extraction branch to extract the corresponding level of thermal features; Based on the cross-modal attention fusion module, visible light features are used as query vectors and infrared thermal features are used as key and value vectors to calculate the attention weights between visible light features and infrared thermal features. The infrared thermal features are weighted and summed based on attention weights, and the weighted summation result is residually connected with the visible light features to obtain multimodal fusion features.

[0022] It should be noted that, firstly, the system receives a series of visible light images (e.g., the appearance of the charging pile in different environments or operating states) and infrared thermal images (showing the temperature distribution of different parts of the charging pile); after processing, these sequences are input into two feature extraction branches respectively; the visible light feature extraction branch extracts three levels of features from the visible light images; the first level of features may include basic visual information such as edges and color distribution; the second level of features may involve more complex information such as shape and texture; the third level of features may capture higher-level concepts, such as the overall structure of the charging pile or certain specific components; The system extracts corresponding levels of thermal features from infrared thermal images. The first level of features may represent the basic pattern of temperature distribution; the second level features may reflect areas of concentrated heat; and the third level features may reveal trends or anomalies in heat sources. The system uses a cross-modal attention mechanism to fuse visible light features with infrared thermal features. Visible light features are used as query vectors, meaning the system will focus on the parts that stand out in the visible light image. Infrared thermal features are used as key and value vectors, allowing the system to weight infrared thermal features according to their importance in the visible light image. This approach enables the model to adaptively focus on areas that appear important in the visible light image while considering their corresponding thermal features in the thermal image. After obtaining the attention weights, the system performs a weighted summation of the infrared thermal features; this means that if a region is considered important in visible light, then the thermal features associated with it will be given higher weights; then, the system performs a residual connection between this weighted summation and the visible light features; the residual connection allows information to flow more smoothly during the feature extraction process, so that the fused features retain the complementary information of visible light and infrared thermal imaging. For example, suppose a visible light image sequence of a charging pile under normal operating conditions shows no obvious damage to its surface, but an abnormally high temperature area appears in an infrared thermal image of a certain component. Through the above process: the visible light feature extraction branch extracts features related to the appearance of the charging pile (such as the shape and color of the casing); the infrared feature extraction branch identifies the thermal characteristics of a certain component, showing that the temperature is higher than normal; the cross-modal attention module focuses on areas with potential problems in the visible light image (such as around the charging interface) and calculates the relevant thermal feature weights by combining the information provided by the infrared thermal image; finally, through weighted summation and residual connection, the system generates a fused feature that simultaneously contains information from the visible light image and temperature data from the infrared thermal image, thus providing a more comprehensive perspective for subsequent fault detection.

[0023] In an optional embodiment, the multimodal fusion features are masked using a dynamic heatmap to obtain fault-sensitive region features, including: Based on the dynamic heat map, the temperature anomaly focusing area is determined, and a binary mask matrix matching the feature size of the multimodal fusion is generated; Multimodal fusion features are multiplied element-wise based on a binary mask matrix to remove background interference features and obtain fault-sensitive region features.

[0024] It's important to note that the dynamic heatmap is generated by analyzing the temperature distribution of the charging station at different points in time. This image clearly shows which areas experience abnormally high temperatures. For example, when a charging station malfunctions, the temperature of some components may be significantly higher than under normal operating conditions. At this stage, the system uses the dynamic heatmap to identify these areas of abnormal temperature. These areas represent potential fault locations; for instance, if the temperature of a connector is consistently higher than that of surrounding components, the system will mark it as an abnormal focus area. Once the abnormal temperature focus areas are identified, the system then generates a binary mask matrix. The size of this mask matrix matches the previously obtained multimodal fusion features, ensuring that each location corresponds to a specific element in the fusion features.

[0025] In this binary mask matrix, the location of abnormal regions is marked with 1, while normal or background regions are marked with 0. This method allows the system to explicitly identify regions of interest for subsequent processing. Next, the system uses this binary mask matrix to perform element-wise multiplication of the multimodal fusion features. This process essentially applies a mask to the fusion features, preserving only features in abnormally focused regions while discarding background interference features. For example, if some regions in the multimodal fusion features have high temperature values ​​(indicating anomalies), while other parts (such as the background) have normal temperature values, then after element-wise multiplication, only the high-temperature values ​​within the abnormally focused regions will be retained, while the values ​​in other background regions will become 0.

[0026] For example, suppose a dynamic heatmap of a charging station shows an abnormally high temperature in the area of ​​its left connector, while the surrounding temperature is normal. In this case: through analysis, the system determines that the left connector is an abnormal focusing area. The system generates a binary mask matrix of the same size as the multimodal fusion feature. In this matrix, the element corresponding to the left connector location is marked as 1, and the elements in other normal areas are marked as 0. This binary mask matrix is ​​then multiplied element-wise with the multimodal fusion feature. The result is that only the fusion feature value of the left connector area is retained, while other background feature values ​​are set to 0.

[0027] In one alternative embodiment, the spatiotemporal graph neural network includes a node embedding layer and a graph convolutional layer; The features of the fault-sensitive region are input into a spatiotemporal graph neural network to obtain a fault evolution feature sequence, including: The features of the fault-sensitive area are flattened into feature vectors, and the feature vectors are mapped into graph nodes based on the node embedding layer to construct the charging pile fault graph structure. Temporal edges are constructed based on the temporal sequence of fault-sensitive area characteristics, and spatial edges are constructed based on the adjacency relationships of various components in the physical structure of the charging pile. The receptive field of the graph convolution kernel is defined by temporal and spatial edges. The features of the graph node at the current time and the historical graph node in the spatiotemporal neighborhood are aggregated and updated to aggregate the fault features of adjacent time and adjacent regions, so as to obtain the fault evolution feature sequence.

[0028] It should be noted that in the previous steps, a matrix containing only the features of the fault-sensitive area was obtained. Next, this matrix needs to be flattened into a feature vector. The flattening process involves arranging each row of the matrix sequentially into a one-dimensional array. This feature vector contains all the important information of the area, preparing for the subsequent graph structure construction. The feature vector is mapped to graph nodes through a node embedding layer. Each graph node represents a fault-sensitive area of ​​the charging pile, and the feature of the node is the previously flattened feature vector. Using a graph structure can effectively capture the relationships between various fault areas, allowing the use of graph theory methods in subsequent analysis. For example, if the flattened feature vector of the charging pile is [0.5, 0.8, 0.3], then these values ​​will be transformed into a graph node, representing a specific fault area. Fault characteristics change over time, so it is necessary to construct time edges based on the temporal sequence of the characteristics of fault-sensitive areas. For example, if the characteristics of a fault area at time t are related to the characteristics at time t+1, then a time edge is established between these two nodes, which can reflect how the fault characteristics evolve over time. In addition to time edges, the spatial relationships between different components of the charging pile must also be considered. If two fault areas are physically adjacent, such as belonging to the same module of the charging pile, a spatial edge will be formed between them. This spatial adjacency relationship can help understand how the fault propagates in space. The receptive field refers to the range of nodes considered by the model when performing feature aggregation. By combining temporal and spatial edges, graph convolutional kernels can aggregate features of the current node and its historical nodes in its spatiotemporal neighborhood. This effectively integrates fault features from adjacent time points and adjacent regions. Graph convolutional networks aggregate feature values ​​from the current graph node and its neighboring nodes connected by temporal and spatial edges to form an updated feature representation. This aggregation considers not only the state of the current node but also the states before and after in time and the states of adjacent regions in space. For example, if the fault feature of a connector becomes significant at a certain moment, the state of adjacent circuit boards or power modules may also affect the manifestation of this fault feature. In this way, a more comprehensive fault evolution feature sequence can be obtained. For example: Suppose that during the monitoring of a charging pile, the temperature of the left connector is found to be abnormal in the past few tests, and recorded as follows: Time t: the temperature feature vector of the left connector is [0.7, 0.9, 0.2]; Time t+1: the temperature feature vector of the left connector is [0.8, 0.95, 0.25]; Time t+2: the temperature feature vector of the left connector is [0.9, 0.85, 0.3]. These feature vectors are flattened to form the corresponding node features; a graph structure is created, and time edges are established between the nodes of the left connector and times t, t+1, and t+2; if there are other components (such as power modules) next to the left connector, spatial edges are established between these nodes; through graph convolution kernels, the features of the current node and its spatiotemporal neighborhood are aggregated, and the node features are updated to form a fault evolution feature sequence.

[0029] In an optional embodiment, the fault evolution feature sequence is inferred based on a preset fault causal reasoning graph to obtain fault root cause features, including: The fault evolution feature sequence is input as evidence node into the fault causal reasoning graph; Based on the conditional probability table in the fault causal reasoning graph, calculate the posterior probability of each potential fault root cause node. The feature vector corresponding to the potential root cause node with the highest posterior probability is selected as the root cause feature.

[0030] It should be noted that the fault evolution feature sequence contains the state information of each part of the charging pile at different points in time. This information is input into the fault causal reasoning graph as "evidence nodes." The fault causal reasoning graph is a graph structure that can represent the causal relationship between faults. For example, if the temperature, differential pressure, and current of a charging pile change abnormally at multiple times during monitoring, these data can be used as input data to help deduce possible causes of the fault. In the fault causal reasoning graph, each node represents a potential cause of the fault, and there are causal relationships between the nodes. Associated with each node is a conditional probability table, which describes the probability of a certain node occurring given the states of other nodes. By using the input evidence nodes (fault evolution feature sequences), Bayes' theorem can be used to update the posterior probabilities of these potential root cause nodes. For example, suppose there are three potential root cause nodes: power supply failure, connector failure, and control system failure. The conditional probability table provides the probability of these failures occurring under different conditions. By calculation, the posterior probability of each root cause node can be obtained, thereby assessing its likelihood of occurrence. Once the posterior probabilities of all potential root cause nodes have been calculated, it is necessary to determine which node has the highest posterior probability. The feature vector corresponding to this node is the root cause feature, representing the most likely cause of the failure. Continuing with the example above, suppose that after calculation, the posterior probability of power supply failure is 0.6, connector failure is 0.3, and control system failure is 0.1; then, power supply failure is the most likely root cause of failure; and feature vectors related to power supply failure (such as power supply current, operating temperature, etc.) will be extracted as root cause features of failure. For example: Suppose that when monitoring the fault evolution feature sequence of a charging pile, the following feature changes are obtained: The feature sequence shows that in the past few monitoring sessions, the current of the charging pile gradually increased, the temperature became abnormal, and the connection status of the connector also became unstable. These features (increased current, abnormal temperature, unstable connector) are used as evidence nodes and input into the fault causal reasoning graph. In the fault causal reasoning graph, suppose there are the following nodes: power supply failure, connector failure, and control system failure. Based on the current feature input and the conditional probability table, the posterior probability of power supply failure is calculated as follows: 0.7; posterior probability of connector failure: 0.2; posterior probability of control system failure: 0.1. Since the posterior probability of power supply failure is the highest (0.7), it is confirmed that power supply failure is the most likely cause of failure for this charging pile. Next, feature vectors related to the power supply, such as the voltage and load capacity of the power supply, are extracted as root cause features of the failure for further analysis and decision-making.

[0031] In one alternative embodiment, the fault detection head includes a classification branch, a localization branch, and a regression branch; The comprehensive fault characteristics are input into the fault detection head to obtain the target detection results output by the fault detection head, including: The comprehensive fault features are input into the classification branch, the localization branch, and the regression branch, respectively. The probability distribution of fault type is output based on the classification branch, the coordinate offset of fault location is output based on the localization branch, and the continuous value of fault severity level is output based on the regression branch. The fault type with the highest confidence is selected based on the probability distribution of fault type, and the fault location is determined by combining the coordinate offset. Pre-set maintenance suggestions are matched according to the continuous values ​​of severity level to obtain the target detection result.

[0032] It should be noted that the system collects and extracts comprehensive fault characteristics of the charging pile, such as current, voltage, temperature, and connection status. These characteristics are then fed into three different branches: a classification branch identifies the fault type, such as power supply failure, control system failure, or connector failure; a location branch determines the location of the fault, such as which part of the charging pile (e.g., power module, control panel, or connector); and a regression branch assesses the severity of the fault, outputting a continuous value representing the severity level of the fault, from minor to severe. The classification branch generates a probability distribution based on the input fault characteristics, representing the likelihood of each fault type. For example, after analysis, the possible fault types and their probabilities are: power supply fault: 70%; control system fault: 20%; connector fault: 10%. The positioning branch outputs the coordinate offset of the fault location. This is usually an offset relative to the charging pile reference point, used to indicate the specific location where the fault occurred. For example, the positioning branch might output an offset of (0.5, 1.2), indicating that the fault occurred at a specific location on the charging pile, 0.5 meters to the right and 1.2 meters upward relative to the reference point. The regression branch outputs a continuous value representing the severity of the fault, such as a score from 0 to 10. Assuming an output of 7.5, the fault is relatively severe and requires immediate attention. Based on the probability distribution of the classification branch, the fault type with the highest confidence is selected; in this example, the highest confidence is for a power supply fault (70%). The location of the fault is determined using the coordinate offset provided by the positioning branch. Combining the output coordinate offset (0.5, 1.2), the specific location of the fault can be found. Based on the severity level (7.5) output by the regression branch, a preset maintenance recommendation is matched; for example, if the fault severity level is above 7, the preset recommendation might be to immediately arrange for professional personnel to inspect and repair it. For example: Suppose the following combined fault characteristics of a charging pile are detected: increased current, abnormal temperature, and poor connection status; input these characteristics into the classification branch, location branch, and regression branch; the classification branch outputs the following probability distribution: power supply failure: 70%; control system failure: 20%; connector failure: 10%; the location branch outputs the fault location offset (0.5, 1.2), indicating that the fault location is 0.5 meters to the right and 1.2 meters above the reference point of the charging pile; the regression branch outputs a fault severity level of 7.5; according to the classification branch, the most likely fault type is power supply failure; combined with the location information (0.5, 1.2), the specific location of the fault is determined; based on the severity level of 7.5, the matched maintenance suggestion is "immediately arrange for technicians to conduct on-site inspection and maintenance".

[0033] In an optional embodiment, the method further includes a model training step: A charging pile fault sample library is constructed, which includes normal samples, single fault samples, and compound fault samples. The samples in the charging pile fault sample library are processed to obtain the predicted detection results; The classification loss, localization loss, and severity regression loss are calculated based on the difference between the predicted detection results and the true labels. End-to-end backpropagation optimization is performed on the multi-scale feature extraction network, spatiotemporal graph neural network, and fault detection head using the total loss function.

[0034] It should be noted that the charging pile fault sample library is a dataset containing various types of samples. It mainly includes the following categories: normal samples represent the operating data of a charging pile under normal working conditions; for example, the current, voltage, and temperature data recorded when a charging pile is charging normally; single fault samples represent the manifestation of a specific fault; for example, if a charging pile experiences a power supply failure, the sample may show an abnormally low or zero current, while the voltage may remain within the normal range; composite fault samples represent situations where multiple faults occur simultaneously; for example, a charging pile may simultaneously experience a power supply failure and a control system failure, leading to abnormal current and temperature. The samples in the sample library are preprocessed, including data cleaning, feature extraction, and standardization. These processes help improve the accuracy of subsequent predictions. The processed samples will be used to train the model or input into an already trained model to obtain prediction results. For example, suppose we have the following samples: normal sample: current 10A, voltage 220V, temperature 25°C; single fault sample (power supply fault): current 0A, voltage 220V, temperature 30°C; combined fault sample (power supply fault + control system fault): current 5A, voltage 180V, temperature 50°C. After processing, the features of these samples may be organized into a format that can be used by the model. The processed samples are input into the trained fault detection model, which outputs corresponding prediction results. These results include predictions of fault type, fault location (coordinate offset), and fault severity. For example, for the composite fault sample above, the model might output: Predicted fault type: power supply fault (80% confidence), control system fault (20% confidence); Predicted fault location: coordinate offset (0.3, 1.0); Predicted severity level: 8.0. Next, the difference between the prediction results and the true labels needs to be calculated to evaluate the model's performance. These differences will be represented by a loss function, mainly including: Classification Loss: Compares the predicted fault type with the actual fault type; for example, if the actual label is "power failure" and the model predicts "power failure", the classification loss is low; if the prediction is "control system failure", the classification loss is high. Location Loss: Compares the distance between the predicted fault location and the actual location; for example, if the actual offset is (0.0, 1.0) and the predicted offset is (0.3, 1.0), the location loss is calculated based on the Euclidean distance between them. Severity Regression Loss: Compares the predicted severity level (e.g., 8.0) with the actual severity level (e.g., 9.0); this loss is typically calculated based on the absolute difference or the difference of squares between the two values. The calculated total loss is used as a feedback signal for end-to-end backpropagation optimization. This means that the model's parameters are adjusted according to the gradient of the loss to reduce errors in future predictions. The networks involved include multi-scale feature extraction networks, spatiotemporal graph neural networks, and fault detection heads. During training, these networks learn how to better extract features, capture spatiotemporal relationships, and perform fault detection. For example, if the classification loss is found to be large, the model will adjust its classification weights to improve its ability to identify specific fault types. If the localization loss is large, it may be necessary to redesign the localization network or adjust the feature extraction method to improve localization accuracy.

[0035] In an optional embodiment, the process of constructing the total loss function includes: Obtain the classification loss function, localization loss function, and severity regression loss function; A fault causal consistency regularization term is introduced to constrain the logical consistency between the fault evolution feature sequence and the fault root cause features; The classification loss function, localization loss function, severity level regression loss function, and fault causality consistency regularization term are weighted and summed to construct the total loss function.

[0036] It should be noted that the classification loss function is used to measure the accuracy of the model in predicting fault types. Assuming there are several possible fault types (such as power failure, control system failure, etc.), the classification loss function calculates the difference between the fault type predicted by the model and the actual fault type. For example, if the actual label is "power failure" while the model predicts "control system failure", the classification loss will compare the difference between the two, resulting in a higher loss value. This indicates that the model has a problem in identifying fault types and needs further optimization. The location loss function evaluates the accuracy of the model's prediction of the fault location. The model needs to output an offset or coordinates representing the specific location of the fault. The location loss calculates the difference between the fault location predicted by the model and the actual fault location. For example, suppose the actual fault location is (0.0, 1.0), while the model predicts the location as (0.3, 1.0). In this case, the location loss will measure the degree of error based on the distance between these two locations. If the distance is short, the location loss is low; if the distance is long, the loss will increase. The severity rating regression loss function is used to evaluate the accuracy of the model's prediction of the severity of a fault. The model needs to output a numerical value to represent the severity of the fault, such as a rating from 1 to 10. The regression loss function compares this predicted value with the actual value. For example, suppose the actual fault severity is 8, while the model predicts 6. In this case, the severity rating regression loss will calculate the difference between these two values ​​to reflect the accuracy of the model's judgment of severity. The introduction of a fault causal consistency regularization term aims to ensure the logical consistency between the fault evolution feature sequence learned by the model and the fault root cause features. In other words, the regularization term constrains the model to consider the causal relationship of the fault occurrence when making predictions, ensuring that the fault performance is consistent with its root cause. For example, if the model predicts a power failure in a certain state, but its fault evolution feature sequence shows a problem with the control system, this is an inconsistency. By introducing a causal consistency regularization term, this mismatch can be penalized, thereby prompting the model to better understand and capture the true causal relationship of the fault. The total loss function is constructed by weighted summation of all the aforementioned loss terms to form a comprehensive loss metric. In this way, the model can simultaneously consider classification accuracy, localization precision, severity judgment, and causal consistency during training, thereby achieving more comprehensive performance optimization. For example, assuming the classification loss, localization loss, and severity regression loss are 0.2, 0.3, and 0.4 respectively, and the fault causal consistency regularization term is 0.1, the total loss function can be expressed as a combination of these losses after weighted summation, thus providing comprehensive feedback for training. This means that when the model updates its parameters, it will be influenced by each aspect, encouraging it to achieve a balanced performance across all tasks.

[0037] In an optional embodiment, after inputting the comprehensive fault features into the fault detection head and obtaining the target detection result output by the fault detection head, the method further includes: Obtain the fault type and fault severity level from the target detection results; If the fault type is insulation damage and the severity level is greater than the preset level threshold, a level one alarm signal will be generated and the charging pile charging function will be locked. If the fault type is loose charging head and the severity level is less than the preset level threshold, a level 2 prompt signal will be generated and charging will continue but the charging power will be reduced. The alarm signal or prompt signal is associated with the target detection result and stored, and then uploaded to the cloud management platform.

[0038] It should be noted that when a charging station performs fault detection, the system analyzes the equipment status and identifies possible fault types. For example, the system may detect problems such as "insulation damage" or "loose charging nozzle." Each fault is also assigned a severity level, typically a value calculated using sensor data and a machine learning model to reflect the severity of the fault. For example, assuming the charging station detects "insulation damage" with a severity level of 9 (assuming a range of 1 to 10), it may also detect another fault, "loose charging nozzle," with a severity level of 3. Based on the fault type and severity level, the system will trigger different levels of alarm signals. If the fault type is "insulation failure" and its severity level is greater than the preset threshold (for example, the threshold is set to 7), the system will generate a level 1 alarm signal. This means that the fault is very serious and may pose a threat to user safety. Therefore, the system will immediately lock the charging function of the charging pile to prevent further safety hazards. For example, in the above example, since the severity level of "insulation failure" is 9, which exceeds the set threshold of 7, the system will send a level 1 alarm signal, lock the charging pile, stop the charging operation, and prompt maintenance personnel to handle it as soon as possible. If the fault type is "loose charging nozzle" and its severity level is less than the set threshold (e.g., the set threshold is 5), the system will generate a secondary warning signal. In this case, although a fault exists, charging is allowed to continue due to its low severity, but the charging power will be reduced to mitigate potential risks. For example, in another scenario, if "loose charging nozzle" is detected with a severity level of 3, which is lower than the set threshold of 5, the system will generate a secondary warning signal to inform the user that there is a problem with the charging station, but charging is still allowed, only with the charging power reduced to 50% to ensure safety. All generated alarm signals and detection results need to be associated with the target detection results for subsequent fault analysis and maintenance decisions. At the same time, this information will be uploaded to the cloud management platform for real-time monitoring and management. For example, in the case of "insulation damage", the system will record the fault type, severity level, alarm signal (level 1), and the charging pile's locked status; this information will then be uploaded to the cloud for the technical team to further analyze and record. In the case of "loose charging head", the system will also record fault information, including fault type, severity level, level 2 warning signal, and current charging power; this data will also be uploaded to the cloud for future maintenance personnel to view.

[0039] Example 2, please refer to Figure 2 This invention provides a technical solution: a charging pile fault detection system based on image recognition, which is applicable to the aforementioned charging pile fault detection method based on image recognition, comprising: Image acquisition unit 1 is used to acquire multimodal time-series images of the target charging pile within a preset detection period; the multimodal time-series images are input into a multi-scale feature extraction network to obtain multimodal fusion features; Fault feature unit 2 is used to construct a dynamic heat map of the target charging pile based on multimodal fusion features. The multimodal fusion features are masked by the dynamic heat map to obtain fault-sensitive area features. The fault-sensitive area features are the area features in the multimodal fusion features where the thermal radiation value is greater than a preset temperature threshold and the texture gradient is greater than a preset gradient threshold. Fault evolution unit 3 is used to input the features of the fault-sensitive area into the spatiotemporal graph neural network to obtain the fault evolution feature sequence; wherein, the fault evolution feature sequence is used to characterize the changing trend and spatial correlation of the features of the fault-sensitive area in the temporal dimension; Fault causal unit 4 is used to reason about the fault evolution feature sequence based on a preset fault causal reasoning graph to obtain the root cause features of the fault; wherein, the fault causal reasoning graph is pre-constructed based on the internal electrical topology of the charging pile and the fault propagation path; The result detection unit 5 is used to fuse the fault-sensitive area features, fault evolution feature sequence and fault root cause features to obtain comprehensive fault features; the comprehensive fault features are input to the fault detection head to obtain the target detection results output by the fault detection head, wherein the target detection results include fault type, fault location, fault severity level and maintenance suggestions.

[0040] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited thereto. Various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of the present invention.

Claims

1. A charging pile fault detection method based on image recognition, characterized in that, include: Acquire multimodal time-series images of the target charging pile within a preset detection period; Multimodal temporal images are input into a multi-scale feature extraction network to obtain multimodal fusion features; A dynamic heat map of the target charging pile is constructed based on multimodal fusion features. The multimodal fusion features are then masked using the dynamic heat map to obtain fault-sensitive area features. The fault-sensitive area features are the areas in the multimodal fusion features where the thermal radiation value is greater than a preset temperature threshold and the texture gradient is greater than a preset gradient threshold. The features of the fault-sensitive region are input into the spatiotemporal graph neural network to obtain the fault evolution feature sequence; the fault evolution feature sequence is used to characterize the changing trend and spatial correlation of the features of the fault-sensitive region in the temporal dimension. The fault evolution feature sequence is inferred based on the pre-set fault causal reasoning graph to obtain the root cause features of the fault; wherein, the fault causal reasoning graph is pre-constructed based on the internal electrical topology of the charging pile and the fault propagation path; The fault-sensitive area features, fault evolution feature sequences, and fault root cause features are fused to obtain comprehensive fault features. The comprehensive fault features are then input into the fault detection head to obtain the target detection results output by the fault detection head. The target detection results include fault type, fault location, fault severity level, and maintenance recommendations.

2. The charging pile fault detection method based on image recognition according to claim 1, characterized in that, Multimodal temporal images include visible light image sequences and infrared thermal imaging sequences; multimodal fusion features include visible light texture features and infrared thermal radiation features; The multi-scale feature extraction network includes a visible light feature extraction branch, an infrared feature extraction branch, and a cross-modal attention fusion module; Multimodal temporal images are input into a multi-scale feature extraction network to obtain multimodal fusion features, including: The visible light image sequence is input into the visible light feature extraction branch to extract the first-level, second-level, and third-level features; the infrared thermal imaging sequence is input into the infrared feature extraction branch to extract the corresponding level of thermal features; Based on the cross-modal attention fusion module, visible light features are used as query vectors and infrared thermal features are used as key and value vectors to calculate the attention weights between visible light features and infrared thermal features. The infrared thermal features are weighted and summed based on attention weights, and the weighted summation result is residually connected with the visible light features to obtain multimodal fusion features.

3. The charging pile fault detection method based on image recognition according to claim 2, characterized in that, By masking the multimodal fusion features using dynamic heatmaps, fault-sensitive region features are obtained, including: Based on the dynamic heat map, the temperature anomaly focusing area is determined, and a binary mask matrix matching the feature size of the multimodal fusion is generated; Multimodal fusion features are multiplied element-wise using a binary mask matrix to remove background interference features and obtain fault-sensitive region features.

4. The charging pile fault detection method based on image recognition according to claim 3, characterized in that, Spatiotemporal graph neural networks consist of node embedding layers and graph convolutional layers; The features of the fault-sensitive region are input into a spatiotemporal graph neural network to obtain a fault evolution feature sequence, including: The features of the fault-sensitive area are flattened into feature vectors, and the feature vectors are mapped into graph nodes based on the node embedding layer to construct the charging pile fault graph structure. Temporal edges are constructed based on the temporal sequence of fault-sensitive area characteristics, and spatial edges are constructed based on the adjacency relationships of various components in the physical structure of the charging pile. The receptive field of the graph convolution kernel is defined by temporal and spatial edges. The features of the graph node at the current time and the historical graph node in the spatiotemporal neighborhood are aggregated and updated to aggregate the fault features of adjacent time and adjacent regions, so as to obtain the fault evolution feature sequence.

5. The charging pile fault detection method based on image recognition according to claim 4, characterized in that, Based on a pre-defined fault causal reasoning graph, the fault evolution feature sequence is inferred to obtain the root cause features of the fault, including: The fault evolution feature sequence is input as evidence node into the fault causal reasoning graph; Based on the conditional probability table in the fault causal reasoning graph, calculate the posterior probability of each potential fault root cause node. The feature vector corresponding to the potential root cause node with the highest posterior probability is selected as the root cause feature.

6. The charging pile fault detection method based on image recognition according to claim 5, characterized in that, The fault detection head includes a classification branch, a localization branch, and a regression branch; The comprehensive fault characteristics are input into the fault detection head to obtain the target detection results output by the fault detection head, including: The comprehensive fault features are input into the classification branch, the localization branch, and the regression branch, respectively. The probability distribution of fault type is output based on the classification branch, the coordinate offset of fault location is output based on the localization branch, and the continuous value of fault severity level is output based on the regression branch. The fault type with the highest confidence is selected based on the probability distribution of fault type, and the fault location is determined by combining the coordinate offset. Pre-set maintenance suggestions are matched according to the continuous values ​​of severity level to obtain the target detection result.

7. The charging pile fault detection method based on image recognition according to claim 6, characterized in that, The method also includes a model training step: A charging pile fault sample library is constructed, which includes normal samples, single fault samples, and compound fault samples. The samples in the charging pile fault sample library are processed to obtain the predicted detection results; The classification loss, localization loss, and severity regression loss are calculated based on the difference between the predicted detection results and the true labels. End-to-end backpropagation optimization is performed on the multi-scale feature extraction network, spatiotemporal graph neural network, and fault detection head using the total loss function.

8. The charging pile fault detection method based on image recognition according to claim 7, characterized in that, The process of constructing the total loss function includes: Obtain the classification loss function, localization loss function, and severity regression loss function; A fault causal consistency regularization term is introduced to constrain the logical consistency between the fault evolution feature sequence and the fault root cause features; The classification loss function, localization loss function, severity level regression loss function, and fault causality consistency regularization term are weighted and summed to construct the total loss function.

9. The charging pile fault detection method based on image recognition according to claim 8, characterized in that, After inputting the comprehensive fault characteristics into the fault detection head and obtaining the target detection result output by the fault detection head, the process also includes: Obtain the fault type and fault severity level from the target detection results; If the fault type is insulation damage and the severity level is greater than the preset level threshold, a level one alarm signal will be generated and the charging pile charging function will be locked. If the fault type is loose charging head and the severity level is less than the preset level threshold, a level 2 prompt signal will be generated and charging will continue but the charging power will be reduced. The alarm signal or prompt signal is associated with the target detection result and stored, and then uploaded to the cloud management platform.

10. A charging pile fault detection system based on image recognition, applicable to the charging pile fault detection method based on image recognition as described in any one of claims 1-9, characterized in that, include: The image acquisition unit is used to acquire multimodal time-series images of the target charging pile within a preset detection period; Multimodal temporal images are input into a multi-scale feature extraction network to obtain multimodal fusion features; The fault feature unit is used to construct a dynamic heat map of the target charging pile based on multimodal fusion features. The multimodal fusion features are masked by the dynamic heat map to obtain fault-sensitive area features. The fault-sensitive area features are the area features in the multimodal fusion features where the thermal radiation value is greater than a preset temperature threshold and the texture gradient is greater than a preset gradient threshold. The fault evolution unit is used to input the features of the fault-sensitive region into the spatiotemporal graph neural network to obtain the fault evolution feature sequence; wherein, the fault evolution feature sequence is used to characterize the changing trend and spatial correlation of the features of the fault-sensitive region in the temporal dimension; The fault causal unit is used to reason about the fault evolution feature sequence based on a preset fault causal reasoning graph to obtain the root cause features of the fault; wherein, the fault causal reasoning graph is pre-constructed based on the internal electrical topology of the charging pile and the fault propagation path; The result detection unit is used to fuse the fault-sensitive area features, fault evolution feature sequence, and fault root cause features to obtain comprehensive fault features; the comprehensive fault features are input to the fault detection head to obtain the target detection results output by the fault detection head, wherein the target detection results include fault type, fault location, fault severity level, and maintenance suggestions.

Images