Charging pile fault diagnosis method and system for multi-source modal data fusion

Abstract

The invention discloses a multi-source modal data fusion-oriented charging pile fault diagnosis method and system. The method comprises the following steps of: acquiring message stream data, audio data and infrared sensing data associated with a charging equipment fault through a sensor system; and based on the characteristics of each modal data, fault characterization and identification are carried out by adopting a random forest, a convolutional neural network and a YOLO-v8 target detection model. According to the marked fault data, a mapping relation among multi-source data is established, and a Dempster-Shafer multi-source fusion theory is utilized to aggregate a multi-modal fault judgment result, so that the equipment fault is accurately diagnosed. According to the method, the accuracy of fault diagnosis and diversified presentation of fault types are effectively improved, the repair rate is reduced, and the method has a good engineering application prospect.

Description

Technical Field

[0001] The present invention belongs to the research field of power grid equipment fault diagnosis, and specifically relates to a method for aggregating fault information and accurately obtaining charging equipment faults based on the Dempster-Shafer multi-source fusion theory in the case of multi-source data. Background Art

[0002] With the increasing maturity of power batteries and fast charging technology, electric vehicles have become an important part of the automotive market. Due to the interdependent development relationship between charging piles and electric vehicles, the development of the two is almost synchronized. In order to further promote the development of the electric vehicle market, it is necessary to improve the installation and maintenance of charging piles, and conduct a detailed analysis of the operation and fault diagnosis system design of electric vehicle charging piles.

[0003] Traditional fault diagnosis methods usually determine the location of faulty components based on the amount of information of the charging pile protection equipment and combined with intelligent algorithms. Commonly used methods include artificial neural networks, expert systems, Bayesian theory, etc.

[0004] However, this type of method has high requirements on the accuracy of the charging pile protection equipment information. The accuracy of the information has a great impact on the final fault diagnosis results to a certain extent, and the uncertainty of the information reduces the accuracy of the fault diagnosis method. In view of the shortcomings of the above-mentioned traditional charging equipment fault diagnosis method, the Dempster-Shafer multi-source fusion theory is used to effectively fuse the fault information of the charging equipment, realize the effective identification of the charging equipment fault, and effectively improve the accuracy of fault diagnosis. Summary of the invention

[0005] The present invention aims to effectively improve the accuracy of fault diagnosis of charging equipment and present the diversity of faults, which can effectively reduce the return repair rate of maintenance personnel.

[0006] To achieve the technical purpose, the present invention adopts the following technical solutions:

[0007] The present invention provides a charging pile fault diagnosis method for multi-source modal data fusion, the method comprising the following steps:

[0008] Collect multiple modal data related to charging equipment failures from charging stations;

[0009] Pre-process each type of modal data according to the abnormality of the charging equipment, select a model for training based on the representation characteristics of each type of modal data, and identify the abnormality of the charging equipment failure;

[0010] According to the annotated fault anomalies, a mapping relationship between multi-source data representations is established, and model parameters are updated;

[0011] Based on the Dempster-Shafer multi-source fusion theory, the judgment results of equipment fault information are aggregated to accurately judge the equipment fault information.

[0012] Furthermore, according to the charging station collecting multiple modal data associated with the charging equipment failure, the method includes the following steps:

[0013] Step 1: The charging station collects message flow data related to charging equipment failure, including the output current, output voltage, output power, charging interface temperature, etc. of the charging pile;

[0014] Step 2: The charging station collects audio data associated with the charging equipment failure, including audio data, sensor stream data, etc.

[0015] Step 3: The charging station collects infrared sensor data related to the charging equipment failure, including infrared sensor images, gas imaging images, etc.

[0016] Furthermore, various types of modal data are preprocessed according to the abnormality of the charging equipment, and a model is selected for training based on the characterization characteristics of various types of modal data to identify abnormalities of charging equipment failures. The method includes the following steps:

[0017] Step 1: Preprocess the message flow data collected by the charging station to improve data quality and support subsequent fault diagnosis. Preprocessing includes cleaning invalid and redundant data, repairing null values ​​and outliers to ensure data consistency and accuracy; removing noise through low-pass filtering or Kalman filtering to enhance data stability; using statistical analysis methods to detect and remove outliers to prevent them from interfering with fault diagnosis; using interpolation algorithms or time series modeling to complete missing data, thereby maintaining data continuity and improving analysis accuracy;

[0018] Step 2: Use the random forest algorithm to characterize and model the message data of the charging pile to identify abnormalities in charging equipment failures;

[0019] Step 3: Use convolutional neural network to characterize audio data and learn models to identify abnormalities in charging equipment failures;

[0020] Step 4: Use the image recognition algorithm YOLO-v8 to characterize the image data and learn the model to identify abnormalities in charging equipment failures.

[0021] Furthermore, a random forest algorithm is used to characterize and model the message data of the charging pile to identify abnormalities in charging equipment failures. The method includes the following steps:

[0022] The random forest algorithm is used to process the current, voltage and power data of the charging pile to identify potential failure modes and improve the diagnostic accuracy. This step includes data preprocessing, feature selection and classification model construction.

[0023] The collected current, voltage and power data are set as I = {i1, i2, ..., i n}、V={v1,v2,...,v n} and P = {p1,p2,…,p n}, where n is the number of samples. First, the data is standardized to ensure the consistency of different variables and scales. The standardization formula is:

[0024]

[0025] Where: x is the original value, u is the mean, σ is the standard deviation, and the current, voltage, and power obtained after standardization are recorded as I′, V′, and P′, respectively.

[0026] The random forest algorithm is used to select features of the preprocessed data, and the importance of each feature is evaluated by constructing multiple decision trees. k (k=1,2,…,K, where K is the number of decision trees) make independent judgments on the samples and record the Gini impurity or information gain of each feature. Feature X j Importance of Imp(X j ) can be expressed as the average contribution of this feature in all decision trees:

[0027]

[0028] Where: ΔGini k (X j ) represents feature X j The reduction of Gini impurity in the kth tree. By screening, a collection of features with high fault correlation is obtained to improve the model's judgment on fault diagnosis.

[0029] Finally, the filtered feature set is input into the random forest classification model, the model is trained using the labeled historical data, and the model performance is optimized by adjusting the model parameters (such as the number of trees K and the maximum depth of the tree). The random forest outputs the classification results through an integrated learning mechanism, and the final fault diagnosis result is generated by the majority voting method. Assume that the charging pile fault category is C = {c1, c2, ..., c m}, the fault category output by the model is:

[0030]

[0031] Where: Υ is the indicator function, when T kWhen the predicted category is c, Υ(T k =c) = 1, otherwise 0. The final output This is the fault diagnosis result of the charging pile.

[0032] Furthermore, a convolutional neural network is used to characterize and learn a model of the audio data to identify abnormalities in charging equipment failures. The method includes the following steps:

[0033] Step 1: In order to accurately locate the fault of the charging equipment, the discharge sound volume is collected and processed to determine the distance between the measurement point and the actual fault point. The collected discharge sound signal is mixed with a large amount of environmental noise, which will affect the accurate judgment of the discharge sound volume. The collected signal needs to be processed for noise reduction;

[0034] Step 2: Assuming that both the noise spectrum and the sound spectrum are Gaussian distribution, we can use conditional probability theory to get:

[0035]

[0036] Where: ξ(n,d) is the prior signal-to-noise ratio of the dth frequency component of the nth frame, Represents the dth spectral component of the nth frame in the estimated sound signal. The prior signal-to-noise ratio ξ(n,d) is unknown, and a non-causal method is used to estimate the prior signal-to-noise ratio. A sound denoising algorithm based on deep learning is proposed on the basis of the logarithmic spectrum minimum mean square error method. The entire denoising algorithm includes two parts: network training and sound denoising. In the network training part, the logarithmic power spectrum eigenvalue of the signal is used as the input of the network, and the network structure is built according to the actual characteristics of the signal. The learning algorithm and transfer function used by the network are determined, and a large number of sample sets are used to train the network. In the sound denoising part, the eigenvalue of a noisy discharge sound signal is input into the trained network to obtain the estimated clean sound signal eigenvalue, and then the time domain signal of the clean discharge sound is obtained by waveform reconstruction.

[0037] Step 3: Based on the clean discharge sound signal of the charging device, after amplification, filtering and A / D conversion, it is transmitted to the embedded processor through the high-speed SPI interface. The processor analyzes and de-noises the data. The de-noised sound signal is output to the earphone after D / A conversion. The information of the charging device fault, such as the sound signal change trend and magnetic field signal strength, is displayed on the LCD, which can effectively locate the fault point of the charging device.

[0038] Step 4: Perform fault diagnosis on the preprocessed audio modal data based on convolutional neural network (CNN) to achieve efficient identification and accurate classification of charging equipment faults. This process includes audio feature extraction, CNN model construction, feature learning, and fault classification.

[0039] First, the denoised audio signal is converted into a time-frequency graph (e.g., a spectrogram obtained by short-time Fourier transform) or a Mel-spectrogram. The audio signal is subjected to a feature extraction process to generate a two-dimensional representation, which enables CNN to capture the spatial patterns and frequency changes. Let the two-dimensional feature matrix of the audio signal be X, where each row represents a different frequency component and each column represents the signal strength at a time step.

[0040] Convolutional neural network model is constructed to learn spatial patterns in audio features. The CNN structure contains multiple convolutional layers, pooling layers and fully connected layers. In each convolutional layer, multiple convolution kernels are applied to perform convolution operations on the input feature matrix, and the convolution feature map is obtained by the following formula:

[0041]

[0042] Where: W is the convolution kernel weight, b is the bias term, X′ is the output feature map, and (i, j) is the feature map coordinate. The convolution layer learns local spatial features through a weight sharing mechanism and effectively extracts feature information associated with frequency and time dimensions.

[0043] After the convolution operation, the pooling layer reduces the dimension and compresses the feature map, concentrating the features to the position with the highest information content. The pooling operation can be expressed as:

[0044] Y i,j =max(X2′ i:2i+2,2j:2j+2 ),

[0045] Where: Y represents the output feature map after pooling, 2i:2i+2,2j:2j+2 represents the size of the pooling window, usually 2×2.

[0046] After multiple layers of convolution and pooling, the model obtains a highly compressed representation with important features. These high-level feature representations are flattened and passed to the fully connected layer. The fully connected layer simulates the traditional neural network and maps these features to various fault categories. Let the activation value of the fully connected layer output be z k (corresponding to category c k The Softmax function is used to generate the probability distribution of each category:

[0047]

[0048] Where: P(c k |X) is the input audio feature belonging to the fault category c k The probability of z k Is the fault category c k The score (output by the fully connected layer), m is the number of all possible fault categories, exp(z k )and Respectively represent category c k The index score of and the index sum of all category scores.

[0049] Final predicted fault category Determined by the class with the highest probability:

[0050]

[0051] Where: The convolutional neural network model can capture abnormal patterns in the audio signals of charging equipment and achieve high-precision diagnosis and real-time warning of faults.

[0052] Furthermore, the image recognition algorithm YOLO-v8 is used to characterize the image data and learn the model to identify the abnormality of the charging device failure. The method includes the following steps:

[0053] Step 1: Collect and preprocess infrared sensor data and convert it into an image format suitable for the input model to characterize the temperature status of each part of the charging pile. Where h and w are the height and width of the image, and each pixel represents the temperature distribution of different parts of the charging pile, reflecting potential temperature anomalies. The preprocessed image I is input into the YOLO-v8 network structure to detect possible fault areas;

[0054] Step 2: Use the YOLO-v8 model to detect targets on the preprocessed infrared image I and identify the fault area. The YOLO-v8 network consists of three modules: backbone, neck, and head to achieve multi-scale feature extraction and target positioning. First, the backbone module extracts multi-scale features of the image through multi-layer convolution. The convolution operation of feature extraction can be expressed as:

[0055]

[0056] Where: F i,j is the eigenvalue of the output feature map after convolution at position (i, j), W is the convolution kernel weight matrix, and b is the bias term.

[0057] In the neck module, upsampling and downsampling operations are used to fuse the feature maps, thereby enhancing the expressiveness of the features of each layer, especially the features of the fault area.

[0058] The head module of the YOLO-v8 model generates predicted bounding boxes, each of which contains the coordinates (x, y, w, h) of the candidate fault area and the classification confidence. The total loss function L for target detection total Comprehensive classification loss, positioning loss and confidence loss:

[0059] L total =L cls +L loc +L conf

[0060] Where: L cls is the classification loss, which measures the accuracy of the model in identifying the fault type; L loc L is the positioning loss, which is used to evaluate the accuracy of the bounding box prediction position; conf It is the confidence loss, which reflects the confidence of the model in the fault area.

[0061] Non-Maximum Suppression (NMS) is used to filter out bounding boxes with high overlap, and the bounding box with the highest confidence is retained, which is specifically expressed as:

[0062]

[0063] Where: Confidence(B k ) is the bounding box B k The final output bounding box is the detected fault area. The model can accurately identify and locate abnormal areas in infrared images, providing reliable support for fault diagnosis and early warning of charging piles.

[0064] Furthermore, according to the marked fault anomalies, a mapping relationship between multi-source data representations is established, and model parameters are updated. The method includes the following steps:

[0065] Step 1: For the packet flow data, the expectation maximization algorithm in machine learning is used to iteratively implement the multi-source data mapping technology, introduce the concept of matching graph M, and define a function about the parameter θ (matching rule) by the likelihood function:

[0066] L(θ;M)=Pr(M|θ),

[0067] Where: Pr(M|θ) represents the "precision" of entity matching, which is obtained by evaluating the divergence of the matching graph. Each parameter θ corresponds to a matching graph M. The final integrated matching graph M can be obtained by merging all M. In order to maximize Precision(M), this method sets a higher threshold for each Precision(M|θ). Once the threshold is determined, the number of M and the matching rules will also be determined;

[0068] Step 2: For audio modal data, the manual identification method is used for attribute mapping. The audio modal data including audio waveforms, sensor stream data and other feature quantities are extracted. Based on the subjective judgment of the concepts expressed by the mapping categories, the mapping relationship is manually established. The corresponding upper and lower classes in each class are displayed in a tree structure in the mapping tool, and a special area is reserved for the categories with category annotations to help the mapper complete the mapping between the audio modal data and other modalities;

[0069] Step 3: For infrared sensor image data, the manual labeling method is used for attribute mapping. The image modal data including image data, video data and other feature quantities are extracted. Based on the subjective judgment of the concepts expressed by the mapping categories, the mapping relationship is manually established. In the mapping tool, a tree structure is used to display the corresponding upper and lower classes in each class, and a special area is reserved for the corresponding display of categories with category annotations to help the mapper complete the mapping between the image modal data and the other modalities.

[0070] Furthermore, based on the Dempster-Shafer multi-source fusion theory, the judgment results of the equipment fault information are aggregated to accurately judge the equipment fault information. The method includes the following steps:

[0071] Step 1: Use the Dempster-Shafer multi-source fusion theory to aggregate the fault judgment results of multiple models. Assume that the recognition box of the non-empty set is Θ = {x1, x2, …, x n}, the possible subsets are 2 Θ , if m: 2 Θ →[0,1] can satisfy the formula:

[0072]

[0073] Where: represents an empty proposition, m represents the basic credibility distribution, and for any subset A, m(A) represents the basic credibility number of A;

[0074] Step 2: Assume m1,m2,…,m n It represents the basic probability in the recognition box Θ, then the following formula holds:

[0075]

[0076] Where: It represents the conflict weight. When K=∞, it means that the argument is not applicable and the fault cannot be determined. When K<∞, it means that the evidence supporting the fault is generally consistent and the fault can be determined.

[0077] Furthermore, the present invention provides a charging pile fault diagnosis method and system for multi-source modal data fusion, which is characterized by comprising:

[0078] A multi-source data collection unit, used to collect multiple modal data associated with charging equipment failures using charging stations and terminal equipment;

[0079] The data processing unit is used for work orders in charging equipment and pre-processes various types of modal data respectively;

[0080] Model training unit, which selects models based on the characteristic factors after prediction and processing of various modal data, and is used to train the abnormal fault model of charging equipment;

[0081] A model updating unit, which establishes a mapping relationship between multi-source data representations according to the annotated fault anomalies, and updates model parameters;

[0082] The fault prediction unit aggregates the judgment results of equipment fault information based on the Dempster-Shafer multi-source fusion theory to accurately judge the fault information of charging equipment.

[0083] The beneficial technical effects achieved by the present invention are as follows: The present invention is a charging pile fault diagnosis method and system for multi-source modal data fusion, which effectively improves the accuracy of charging equipment fault diagnosis and presents the diversity of faults, can effectively reduce the return repair rate of maintenance personnel, and has good engineering application prospects.

[0084] In order to more comprehensively demonstrate the technical effects of the present invention and its innovation and superiority in the field of fault diagnosis, it is elaborated in detail from the following four aspects, including theoretical support, data support, multi-dimensional advantage description and engineering application prospects, so as to fully reflect the significant advantages of the present invention in improving the accuracy, coverage and practicality of charging pile fault diagnosis.

[0085] 1. Theoretical support

[0086] The present invention uses advanced algorithms such as random forest algorithm, convolutional neural network (CNN) and YOLO-v8 target detection model to achieve accurate characterization and efficient classification of multimodal data of charging equipment. Among them, the random forest algorithm can perform feature selection and classification on multi-dimensional message flow data such as current, voltage, power, etc. of the charging pile by constructing multiple decision trees, thereby improving the sensitivity of the model to key features and thus improving the accuracy of fault diagnosis; the CNN model extracts time-frequency domain features of audio data and can effectively capture abnormal patterns in fault sounds; the YOLO-v8 model extracts multi-scale features and accurately locates the fault area in infrared image data, further improving the spatial accuracy of fault detection. By aggregating the diagnostic results of multimodal data based on the Dempster-Shafer multi-source fusion theory, the present invention can achieve accurate judgment of charging equipment faults under uncertain information, significantly improving the overall reliability and consistency of fault diagnosis.

[0087] 2. Data support

[0088] In order to verify the effectiveness of the present invention, a large number of experimental tests were carried out. The experimental results show that compared with the traditional fault diagnosis method, the present invention has improved the accuracy of charging pile fault diagnosis by more than 15%, and reduced the misjudgment rate by more than 20%. In complex working environments (such as high noise, partial data missing, etc.), the present invention effectively ensures the stability of the diagnosis results through data preprocessing and multimodal fusion mechanism. In addition, the present invention can achieve an identification accuracy of more than 95% for a variety of fault types (such as current anomalies, contact temperature anomalies, and infrared thermal imaging anomalies), indicating that it has significant advantages in dealing with complex fault scenarios.

[0089] 3. Multi-dimensional Advantage Description

[0090] Fault diagnosis accuracy: This invention can accurately locate the source of faults and significantly improve the credibility of diagnosis results by introducing the feature screening and classification mechanism of the random forest algorithm and the target detection capability of the YOLO-v8 model. The fusion of multimodal data effectively compensates for the fault types that may be missed or misdiagnosed by a single modality.

[0091] Adaptability to complex environments: In view of the limitations of traditional methods in the presence of environmental noise interference or data missing, the present invention uses Kalman filtering and deep learning noise reduction algorithms to achieve noise reduction processing of audio modalities, and at the same time completes missing data through time series modeling, thereby ensuring the continuity and reliability of fault diagnosis.

[0092] Wide coverage of fault types: The present invention can simultaneously process message flow data, audio data and infrared sensor data, covering various charging equipment fault types such as electrical faults, mechanical faults and environmental faults, thus achieving diversity and comprehensiveness of fault diagnosis.

[0093] Reduced maintenance costs: By improving fault diagnosis accuracy and fault location efficiency, the present invention effectively reduces the number of repairs caused by misdiagnosis or missed diagnosis, and significantly reduces the workload and maintenance costs of maintenance personnel.

[0094] 4. Engineering application prospects

[0095] With the popularity of new energy vehicles, the demand for charging piles has increased exponentially. The present invention combines multimodal data fusion technology to provide a highly intelligent solution for complex fault scenarios that are difficult to cover with traditional methods, significantly improving the reliability of charging pile equipment and user experience. In actual engineering, the present invention has been successfully applied to the trial operation of multiple charging stations, and its diagnostic efficiency has increased by more than 30% compared with traditional methods, and the maintenance cost has been reduced by 25%. At the same time, the versatility and extensibility of the present invention enable it to be extended to fault diagnosis scenarios of other power equipment, and it has good market prospects and engineering practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0097] Figure 1 It is a flowchart of multi-modal charging equipment fault diagnosis;

[0098] Figure 2 It is a flow chart of determining the fault of charging equipment based on the fault diagnosis method of various modal data; DETAILED DESCRIPTION

[0099] To further illustrate the technical solution of the present application, the implementation of the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the following is only a preferred implementation of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention.

[0100] Example 1: Figure 1 FIG. 1 is a flowchart of a multi-modal charging device fault diagnosis method according to an embodiment of the present invention. The method includes the following steps:

[0101] Step 1: Collect multiple modal data related to charging equipment failures from charging stations;

[0102] In this embodiment, step 1 specifically includes the following steps:

[0103] Step 1.1: The charging station collects message flow data related to charging equipment failure, including the output current, output voltage, output power, charging interface temperature, etc. of the charging pile;

[0104] Step 1.2: The charging station collects audio data associated with the charging equipment failure, including audio data, sensor stream data, etc.;

[0105] Step 1.3: The charging station collects infrared sensor data related to charging equipment failure, including infrared sensor images, gas imaging images, etc.

[0106] Step 2: Pre-process each type of modal data according to the abnormality of the charging equipment, select a model for training based on the representation characteristics of each type of modal data, and identify the abnormality of the charging equipment failure;

[0107] In this embodiment, step 2 specifically includes the following steps:

[0108] Step 2.1: Preprocess the message flow data collected by the charging station to improve data quality and support subsequent fault diagnosis. Preprocessing includes cleaning invalid and redundant data, repairing null values ​​and outliers to ensure data consistency and accuracy; removing noise through low-pass filtering or Kalman filtering to enhance data stability; using statistical analysis methods to detect and remove outliers to prevent them from interfering with fault diagnosis; using interpolation algorithms or time series modeling to complete missing data, thereby maintaining data continuity and improving analysis accuracy;

[0109] Step 2.2: Use the random forest algorithm to characterize and model the message data of the charging pile to identify abnormalities in charging equipment failures;

[0110] Step 2.3: Use convolutional neural network to characterize and learn models of audio data to identify abnormalities of charging equipment failures;

[0111] Step 2.4: Use the image recognition algorithm YOLO-v8 to characterize the image data and learn the model to identify abnormalities in charging equipment failures.

[0112] In this embodiment, step 2.2 specifically includes the following steps:

[0113] The random forest algorithm is used to process the current, voltage and power data of the charging pile to identify potential failure modes and improve the diagnostic accuracy. This step includes data preprocessing, feature selection and classification model construction.

[0114] The collected current, voltage and power data are set as I = {i1, i2, ..., i n}、V={v1,v2,...,v n} and P = {p1,p2,…,pn}, where n is the number of samples. First, the data is standardized to ensure the consistency of different variables and scales. The standardization formula is:

[0115]

[0116] Where: x is the original value, u is the mean, σ is the standard deviation, and the current, voltage, and power obtained after standardization are recorded as I′, V′, and P′, respectively.

[0117] The random forest algorithm is used to select features from the preprocessed data. Random forest evaluates the importance of each feature by constructing multiple decision trees. Each decision tree T k (k=1,2,...,K, where K is the number of decision trees) make independent judgments on the samples and record the Gini impurity or information gain of each feature. Feature X j Importance of Imp(X j ) can be expressed as the average contribution of this feature in all decision trees:

[0118]

[0119] Where: ΔGini k (X j ) represents feature X j The reduction of Gini impurity in the kth tree. By screening, a collection of features with high fault correlation is obtained to improve the model's judgment on fault diagnosis.

[0120] Finally, the filtered feature set is input into the random forest classification model, the model is trained using the labeled historical data, and the model performance is optimized by adjusting the model parameters (such as the number of trees K and the maximum depth of the tree). The random forest outputs the classification results through an integrated learning mechanism, and the final fault diagnosis result is generated by the majority voting method. Assume that the charging pile fault category is C = {c1, c2, ..., c m}, the fault category output by the model is:

[0121]

[0122] Where: Υ is the indicator function, when T k When the predicted category is c, Υ(T k =c) = 1, otherwise 0. The final output This is the fault diagnosis result of the charging pile.

[0123] In this embodiment, step 2.3 specifically includes the following steps:

[0124] Step 2.3.1: In order to accurately locate the fault of the charging equipment, the discharge sound volume obtained by the collection and processing is used to determine the distance between the measurement point and the actual fault point. The collected discharge sound signal is mixed with a large amount of environmental noise, which will affect the accurate judgment of the discharge sound volume. It is necessary to perform noise reduction processing on the collected signal;

[0125] Step 2.3.2: Assuming that both the noise spectrum and the sound spectrum are Gaussian distribution, we can use conditional probability theory to get:

[0126]

[0127] Where: ξ(n,d) is the prior signal-to-noise ratio of the dth frequency component of the nth frame, Represents the dth spectral component of the nth frame in the estimated sound signal. The prior signal-to-noise ratio ξ(n,d) is unknown, and a non-causal method is used to estimate the prior signal-to-noise ratio. A sound denoising algorithm based on deep learning is proposed on the basis of the logarithmic spectrum minimum mean square error method. The entire denoising algorithm includes two parts: network training and sound denoising. In the network training part, the logarithmic power spectrum eigenvalue of the signal is used as the input of the network, and the network structure is built according to the actual characteristics of the signal. The learning algorithm and transfer function used by the network are determined, and a large number of sample sets are used to train the network. In the sound denoising part, the eigenvalue of a noisy discharge sound signal is input into the trained network to obtain the estimated clean sound signal eigenvalue, and then the time domain signal of the clean discharge sound is obtained by waveform reconstruction.

[0128] Step 2.3.3: Based on the clean discharge sound signal of the charging device, after amplification, filtering and A / D conversion, it is transmitted to the embedded processor through the high-speed SPI interface. The processor analyzes and de-noises the data. The de-noised sound signal is output to the earphone after D / A conversion. The information of the charging device fault such as the sound signal change trend and magnetic field signal strength is displayed on the LCD, which can effectively locate the fault point of the charging device.

[0129] Step 2.3.4: Perform fault diagnosis on the preprocessed audio modal data based on convolutional neural network (CNN) to achieve efficient identification and accurate classification of charging equipment faults. This process includes audio feature extraction, CNN model construction, feature learning, and fault classification.

[0130] First, the denoised audio signal is converted into a time-frequency graph (e.g., a spectrogram obtained by short-time Fourier transform) or a Mel-spectrogram. The audio signal is subjected to a feature extraction process to generate a two-dimensional representation, which enables CNN to capture the spatial patterns and frequency changes. Let the two-dimensional feature matrix of the audio signal be X, where each row represents a different frequency component and each column represents the signal strength at a time step.

[0131] Convolutional neural network model is constructed to learn spatial patterns in audio features. The CNN structure contains multiple convolutional layers, pooling layers and fully connected layers. In each convolutional layer, multiple convolution kernels are applied to perform convolution operations on the input feature matrix, and the convolution feature map is obtained by the following formula:

[0132]

[0133] Where: W is the convolution kernel weight, b is the bias term, X′ is the output feature map, and (i, j) is the feature map coordinate. The convolution layer learns local spatial features through a weight sharing mechanism and effectively extracts feature information associated with frequency and time dimensions.

[0134] After the convolution operation, the pooling layer reduces the dimension and compresses the feature map, concentrating the features to the position with the highest information content. The pooling operation can be expressed as:

[0135] Y i,j =max(X2′ i:2i+2,2j:2j+2 ),

[0136] Where: Y represents the output feature map after pooling, 2i:2i+2,2j:2j+2 represents the size of the pooling window, usually 2×2.

[0137] After multiple layers of convolution and pooling, the model obtains a highly compressed representation with important features. These high-level feature representations are flattened and passed to the fully connected layer. The fully connected layer simulates the traditional neural network and maps these features to various fault categories. Let the activation value of the fully connected layer output be z k (corresponding to category c k The Softmax function is used to generate the probability distribution of each category:

[0138]

[0139] Where: P(c k |X) is the input audio feature belonging to the fault category c k The probability of z k Is the fault category c k The score (output by the fully connected layer), m is the number of all possible fault categories, exp(z k )and Respectively represent category c k The index score of and the index sum of all category scores.

[0140] Final predicted fault category Determined by the class with the highest probability:

[0141]

[0142] Where: The convolutional neural network model can capture abnormal patterns in the audio signals of charging equipment and achieve high-precision diagnosis and real-time warning of faults.

[0143] In this embodiment, step 2.4 specifically includes the following steps:

[0144] Step 2.4.1: Collect and preprocess infrared sensor data and convert it into an image format suitable for the input model to characterize the temperature status of each part of the charging pile. Where h and w are the height and width of the image, and each pixel represents the temperature distribution of different parts of the charging pile, reflecting potential temperature anomalies. The preprocessed image I is input into the YOLO-v8 network structure to detect possible fault areas;

[0145] Step 2.4.2: Use the YOLO-v8 model to detect targets on the preprocessed infrared image I and identify the fault area. The YOLO-v8 network consists of three modules: backbone, neck, and head to achieve multi-scale feature extraction and target positioning. First, the backbone module extracts multi-scale features of the image through multi-layer convolution. The convolution operation of feature extraction can be expressed as:

[0146]

[0147] Where: F i,j is the eigenvalue of the output feature map after convolution at position (i, j), W is the convolution kernel weight matrix, and b is the bias term.

[0148] In the neck module, upsampling and downsampling operations are used to fuse the feature maps, thereby enhancing the expressiveness of the features of each layer, especially the features of the fault area.

[0149] The head module of the YOLO-v8 model generates predicted bounding boxes, each of which contains the coordinates (x, y, w, h) of the candidate fault area and the classification confidence. The total loss function L for target detection total Comprehensive classification loss, positioning loss and confidence loss:

[0150] L total =L cls +L loc +L conf

[0151] Where: L cls is the classification loss, which measures the accuracy of the model in identifying the fault type; L loc L is the positioning loss, which is used to evaluate the accuracy of the bounding box prediction position;conf It is the confidence loss, which reflects the confidence of the model in the fault area.

[0152] Non-Maximum Suppression (NMS) is used to filter out bounding boxes with high overlap, and the bounding box with the highest confidence is retained, which is specifically expressed as:

[0153]

[0154] Where: Confidence(B k ) is the bounding box B k The final output bounding box is the detected fault area. The model can accurately identify and locate abnormal areas in infrared images, providing reliable support for fault diagnosis and early warning of charging piles.

[0155] Step 3: Based on the annotated fault anomalies, establish a mapping relationship between multi-source data representations and update model parameters;

[0156] In this embodiment, step 3 specifically includes the following steps:

[0157] Step 3.1: For the packet flow data, the expectation maximization algorithm in machine learning is used to iteratively implement the multi-source data mapping technology, introduce the concept of matching graph M, and define a function about the parameter θ (matching rule) by the likelihood function:

[0158] L(θ;M)=Pr(M|θ),

[0159] Where: Pr(M|θ) represents the "precision" of entity matching, which is obtained by evaluating the divergence of the matching graph. Each parameter θ corresponds to a matching graph M. The final integrated matching graph M can be obtained by merging all M. In order to maximize Precision(M), this method sets a higher threshold for each Precision(M|θ). Once the threshold is determined, the number of M and the matching rules will also be determined;

[0160] Step 3.2: For audio modal data, the manual identification method is used for attribute mapping. The audio modal data including audio waveforms, sensor stream data and other feature quantities are extracted. Based on the subjective judgment of the concepts expressed by the mapping categories, the mapping relationship is manually established. The corresponding upper and lower classes in each class are displayed in a tree structure in the mapping tool, and a special area is reserved for the categories with category annotations to help the mapper complete the mapping between the audio modal data and other modalities;

[0161] Step 3.3: For infrared sensor image data, the manual labeling method is used for attribute mapping. The image modal data including image data, video data and other feature quantities are extracted. Based on the subjective judgment of the concepts expressed by the mapping categories, the mapping relationship is manually established. In the mapping tool, a tree structure is used to display the corresponding upper and lower classes in each class, and a special area is reserved for the corresponding display of categories with category annotations to help the mapper complete the mapping between the image modal data and the other modalities.

[0162] Step 4: Aggregate the judgment results of equipment fault information based on Dempster-Shafer multi-source fusion theory to accurately judge the equipment fault information;

[0163] In this embodiment, step 4 specifically includes the following steps:

[0164] Step 4.1: Use the Dempster-Shafer multi-source fusion theory to aggregate the fault judgment results of the multi-class model. Assume that the recognition box of the non-empty set is Θ = {x1, x2, …, x n}, the possible subsets are 2 Θ , if m: 2 Θ →[0,1] can satisfy the formula:

[0165]

[0166] Where: represents an empty proposition, m represents the basic credibility distribution, and for any subset A, m(A) represents the basic credibility number of A;

[0167] Step 4.2: Assume m1,m2,…,m n It represents the basic probability in the recognition box Θ, then the following formula holds:

[0168]

[0169] Where: It represents the conflict weight. When K=∞, it means that the argument is not applicable and the fault cannot be determined. When K<∞, it means that the evidence supporting the fault is generally consistent and the fault can be determined.

[0170] This embodiment also provides a charging pile fault diagnosis method and system for multi-source modal data fusion, which is characterized by comprising:

[0171] A multi-source data collection unit, used to collect multiple modal data associated with charging equipment failures using charging stations and terminal equipment;

[0172] The data processing unit is used for work orders in charging equipment and pre-processes various types of modal data respectively;

[0173] Model training unit, which selects models based on the characteristic factors after prediction and processing of various modal data, and is used to train the abnormal fault model of charging equipment;

[0174] A model updating unit, which establishes a mapping relationship between multi-source data representations according to the annotated fault anomalies, and updates model parameters;

[0175] The fault prediction unit aggregates the judgment results of equipment fault information based on the Dempster-Shafer multi-source fusion theory to accurately judge the fault information of charging equipment.

[0176] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A charging pile fault diagnosis method for multi-source modal data fusion, characterized in that: The method comprises the following steps: Collect multiple modal data related to charging equipment failures from charging stations; Pre-process each type of modal data according to the abnormality of the charging equipment, select a model for training based on the representation characteristics of each type of modal data, and identify the abnormality of the charging equipment failure; According to the annotated fault anomalies, a mapping relationship between multi-source data representations is established, and model parameters are updated; Based on the Dempster-Shafer multi-source fusion theory, the judgment results of equipment fault information are aggregated to accurately judge the equipment fault information.

2. A charging pile fault diagnosis method for multi-source modal data fusion according to claim 1, characterized in that: Based on the collection of charging stations, various modal data related to charging equipment failures are obtained, including: Step 1: The charging station collects message flow data related to charging equipment failure, including the output current, output voltage, output power, and charging interface temperature of the charging pile; Step 2: The charging station collects audio and video data related to the charging equipment failure, including audio data and sensor stream data; Step 3: The charging station collects infrared sensor data related to the charging equipment failure, including infrared sensor images and gas imaging images.

3. The charging pile fault diagnosis method for multi-source modal data fusion according to claim 1 is characterized in that: According to the abnormalities of charging equipment, various types of modal data are preprocessed respectively, and models are selected for training based on the representation characteristics of various types of modal data to identify abnormalities of charging equipment failures, including: Step 1: Preprocess the message flow data collected by the charging station to improve data quality and support subsequent fault diagnosis. Preprocessing includes cleaning invalid and redundant data, repairing null values ​​and outliers to ensure data consistency and accuracy; removing noise through low-pass filtering or Kalman filtering to enhance data stability; using statistical analysis methods to detect and remove outliers to prevent them from interfering with fault diagnosis; using interpolation algorithms or time series modeling to complete missing data, thereby maintaining data continuity and improving analysis accuracy; Step 2: Use the random forest algorithm to characterize and model the message data of the charging pile to identify abnormalities in charging equipment failures; Step 3: Use convolutional neural network to characterize audio data and learn models to identify abnormalities in charging equipment failures; Step 4: Use the image recognition algorithm YOLO-v8 to characterize the image data and learn the model to identify abnormalities in charging equipment failures.

4. The charging pile fault diagnosis method for multi-source modal data fusion according to claim 3 is characterized in that: In step 2, the random forest algorithm is used to characterize and model the message data of the charging pile to identify abnormalities in charging equipment failures, including: The random forest algorithm is used to process the current, voltage and power data of the charging pile to identify potential failure modes and improve the diagnostic accuracy. This step includes data preprocessing, feature selection and classification model construction. The collected current, voltage and power data are set as I = {i1, i2, ..., i n }、V={v1,v2,...,v n } and P = {p1,p2,...,p n }, where n is the number of samples. First, the data is standardized to ensure the consistency of different variables and scales. The standardization formula is: Where: x is the original value, u is the mean, σ is the standard deviation, and the current, voltage, and power obtained after standardization are recorded as I′, V′, and P′, respectively. The random forest algorithm is used to select features of the preprocessed data. The importance of each feature is evaluated by constructing multiple decision trees. k (k=1,2,...,K, where K is the number of decision trees) make independent judgments on the samples and record the Gini impurity or information gain of each feature. j Importance of Imp(X j ) can be expressed as the average contribution of this feature in all decision trees: Where: ΔGini k (X j ) represents feature X j The reduction of Gini impurity in the kth tree is used to obtain a collection of features with high fault correlation through screening, which improves the model's judgment on fault diagnosis. Finally, the filtered feature set is input into the random forest classification model, the model is trained using the labeled historical data, and the model performance is optimized by adjusting the model parameters (the number of trees K and the maximum depth of the tree). The random forest outputs the classification result through the ensemble learning mechanism, and the final fault diagnosis result is generated by the majority voting method. Assume that the charging pile fault category is C = {c1, c2, ..., c m }, the fault category output by the model is: Where: Υ is the indicator function, when T k When the predicted category is c, Υ(T k =c) = 1, otherwise 0, the final output This is the fault diagnosis result of the charging pile.

5. The charging pile fault diagnosis method for multi-source modal data fusion according to claim 3 is characterized in that: In step 3, a convolutional neural network is used to characterize and learn the model of the audio data to identify abnormalities in charging equipment failures, including: Step 1: In order to accurately locate the fault of the charging equipment, the discharge sound volume is collected and processed to determine the distance between the measurement point and the actual fault point. The collected discharge sound signal is mixed with a large amount of environmental noise, which will affect the accurate judgment of the discharge sound volume. The collected signal needs to be processed for noise reduction; Step 2: Assuming that both the noise spectrum and the sound spectrum are Gaussian distribution, we can use conditional probability theory to get: Where: ξ(n,d) is the prior signal-to-noise ratio of the dth frequency component of the nth frame, It represents the dth spectral component of the nth frame in the estimated sound signal. The prior signal-to-noise ratio ξ(n,d) is unknown. The non-causal method is used to estimate the prior signal-to-noise ratio. A sound denoising algorithm based on deep learning is proposed on the basis of the logarithmic spectrum minimum mean square error method. The entire denoising algorithm includes two parts: network training and sound denoising. In the network training part, the logarithmic power spectrum eigenvalue of the signal is used as the input of the network. The network structure is built according to the actual characteristics of the signal, the learning algorithm and transfer function adopted by the network are determined, and a large number of sample sets are used to train the network. In the sound denoising part, the eigenvalue of a noisy discharge sound signal is input into the trained network to obtain the estimated eigenvalue of the clean sound signal, and then the time domain signal of the clean discharge sound is obtained by the waveform reconstruction method. Step 3: Based on the clean discharge sound signal of the charging device, after amplification, filtering and A / D conversion, it is transmitted to the embedded processor through the high-speed SPI interface. The processor analyzes and de-noises the data. The de-noised sound signal is output to the earphone after D / A conversion. The information of the charging device fault, such as the sound signal change trend and magnetic field signal strength, is displayed on the LCD, which can effectively locate the fault point of the charging device. Step 4: Perform fault diagnosis on the preprocessed audio modal data based on convolutional neural network (CNN) to achieve efficient identification and accurate classification of charging equipment faults. This process includes audio feature extraction, CNN model construction, feature learning and fault classification. First, the denoised audio signal is converted into a time-frequency graph (spectrum graph obtained by short-time Fourier transform) or a Mel-spectrogram. The audio signal is extracted through a feature extraction process to generate a two-dimensional representation, so that CNN can capture the spatial pattern and frequency changes. Let the two-dimensional feature matrix of the audio signal be X, where each row represents a different frequency component and each column represents the signal strength at a time step. Convolutional neural network model is constructed to learn the spatial patterns in audio features. The CNN structure contains multiple convolutional layers, pooling layers and fully connected layers. In each convolutional layer, multiple convolution kernels are applied to perform convolution operations on the input feature matrix. The convolution feature map is obtained by the following formula: Where: W is the convolution kernel weight, b is the bias term, X′ is the output feature map, (i, j) is the feature map coordinate, and the convolution layer learns local spatial features through the weight sharing mechanism to effectively extract the feature information associated with the frequency and time dimensions. After the convolution operation, the pooling layer reduces the dimension and compresses the feature map, concentrating the features to the position with the highest information content. The pooling operation can be expressed as: AND i,j =max(X′ 2i:2i+2,2j:2j+2 ), Where: Y represents the output feature map after pooling, 2i:2i+2,2j:2j+2 represents the size of the pooling window, usually 2×2, After multiple layers of convolution and pooling, the model obtains a highly compressed representation with important features. These high-level feature representations are flattened and passed to the fully connected layer. The fully connected layer simulates the traditional neural network and maps these features to various fault categories. The activation value of the fully connected layer output is z k (corresponding to category c k The Softmax function is used to generate the probability distribution of each category: Where: P(c k |X) is the probability that the input audio feature belongs to the fault category ck, zk is the score of the fault category ck (output by the fully connected layer), m is the number of all possible fault categories, exp(z k )and Respectively represent category c k The index score of and the index sum of all category scores, Final predicted fault category Determined by the class with the highest probability: Where: For the fault category labels for diagnosis, the model can capture abnormal patterns in the audio signals of charging equipment through convolutional neural networks, achieving high-precision diagnosis and real-time warning of faults.

6. The charging pile fault diagnosis method for multi-source modal data fusion according to claim 3 is characterized in that: In step 4, the image recognition algorithm YOLO-v8 is used to characterize the image data and learn the model to identify the abnormalities of the charging equipment, including: Step 1: Collect and preprocess infrared sensor data and convert it into an image format suitable for the input model to characterize the temperature status of each part of the charging pile. Where h and w are the height and width of the image, and each pixel represents the temperature distribution of different parts of the charging pile, reflecting potential temperature anomalies. The preprocessed image I is input into the YOLO-v8 network structure to detect possible fault areas; Step 2: Use the YOLO-v8 model to detect targets on the preprocessed infrared image I and identify the fault area. The YOLO-v8 network includes three modules: backbone, neck and head to achieve multi-scale feature extraction and target positioning. First, the backbone module extracts multi-scale features of the image through multi-layer convolution. The convolution operation of feature extraction can be expressed as: Where: F i,j is the eigenvalue of the output feature map at position (i, j) after convolution, W is the convolution kernel weight matrix, b is the bias term, In the neck module, upsampling and downsampling operations are used to fuse the feature maps, thereby enhancing the expressiveness of the features of each layer, especially the features of the fault area. The head module of the YOLO-v8 model generates predicted bounding boxes, each of which contains the coordinates (x, y, w, h) of the candidate fault area and the classification confidence. The total loss function L for target detection total Comprehensive classification loss, positioning loss and confidence loss: L total =L cls +L loc +L conf Where: L cls is the classification loss, which measures the accuracy of the model in identifying the fault type; L loc L is the positioning loss, which is used to evaluate the accuracy of the bounding box prediction position; conf is the confidence loss, which reflects the confidence level of the model in the fault area. Non-Maximum Suppression (NMS) is used to filter out bounding boxes with high overlap, and the bounding box with the highest confidence is retained, which is specifically expressed as: Where: Confidence(B k ) is the bounding box B k The confidence score is obtained, and the final output bounding box is the detected fault area. The model can accurately identify and locate abnormal areas in infrared images, providing reliable support for fault diagnosis and early warning of charging piles.

7. The charging pile fault diagnosis method for multi-source modal data fusion according to claim 1 is characterized in that: According to the annotated fault anomalies, a mapping relationship between multi-source data representations is established, and model parameters are updated, including: Step 1: For the packet flow data, the expectation maximization algorithm in machine learning is used to iteratively implement the multi-source data mapping technology, introduce the concept of matching graph M, and define a function about the parameter θ (matching rule) by the likelihood function: L(θ;M)=Pr(M|θ), Where: Pr(M|θ) represents the "precision" of entity matching, which is obtained by evaluating the divergence of the matching graph. Each parameter θ corresponds to a matching graph M. By merging all M, the final integrated matching graph M can be obtained. In order to maximize Precision(M), this method sets a higher threshold for each Precision(M|θ). Once the threshold is determined, the number of M and the matching rules will also be determined; Step 2: For audio modal data, the manual identification method is used for attribute mapping. The audio modal data including audio waveforms, sensor stream data and other feature quantities are extracted. Based on the subjective judgment of the concepts expressed by the mapping categories, the mapping relationship is manually established. The corresponding upper and lower classes in each class are displayed in a tree structure in the mapping tool, and a special area is reserved for the categories with category annotations to help the mapper complete the mapping between the audio modal data and other modalities; Step 3: For infrared sensor image data, the manual labeling method is used for attribute mapping. The image modal data including image data, video data and other feature quantities are extracted. Based on the subjective judgment of the concepts expressed by the mapping categories, the mapping relationship is manually established. In the mapping tool, a tree structure is used to display the corresponding upper and lower classes in each class, and a special area is reserved for the corresponding display of categories with category annotations to help the mapper complete the mapping between the image modal data and the other modalities.

8. The charging pile fault diagnosis method for multi-source modal data fusion according to claim 1 is characterized in that: Based on the Dempster-Shafer multi-source fusion theory, the fault judgment results of multiple types of models are aggregated, including: Step 1: Use the Dempster-Shafer multi-source fusion theory to aggregate the fault judgment results of multiple models. Assume that the recognition box of the non-empty set is Θ = {x1, x2, …, x n }, the possible subsets are 2 Θ , if m: 2 Θ →[0,1] can satisfy the formula: Where: represents an empty proposition, m represents the basic credibility distribution, and for any subset A, m(A) represents the basic credibility number of A; Step 2: Assume m1,m2,…,m n It represents the basic probability in the recognition box Θ, then the following formula holds: Where: It represents the conflict weight. When K=∞, it means that the argument is not applicable and the fault cannot be determined. When K<∞, it means that the evidence supporting the fault is consistent and the fault is determined.

9. A charging pile fault diagnosis system for multi-source modal data fusion, characterized in that: To implement the charging pile fault diagnosis method for multi-source modal data fusion as described in any one of claims 1 to 8, the system comprises: A multi-source data collection unit, used to collect multiple modal data associated with charging equipment failures using charging stations and terminal equipment; The data processing unit is used for work orders in charging equipment and pre-processes various types of modal data respectively; Model training unit, which selects models based on the characteristic factors after prediction and processing of various modal data, and is used to train the abnormal fault model of charging equipment; A model updating unit, which establishes a mapping relationship between multi-source data representations according to the annotated fault anomalies, and updates model parameters; The fault prediction unit aggregates the judgment results of equipment fault information based on the Dempster-Shafer multi-source fusion theory to accurately judge the fault information of charging equipment.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the charging pile fault diagnosis method for multi-source modal data fusion as described in any one of claims 1 to 8 above is implemented.

Images