A charging pile fault diagnosis method and system for multi-source modal data fusion

Abstract

The application discloses a charging pile fault diagnosis method and system for multi-source modal data fusion, which collects message flow data, audio data and infrared sensing data related to charging equipment faults through a sensor system. Based on the characteristics of each modal data, random forest, convolutional neural network and YOLO-v8 target detection model are used for fault characterization and identification. According to the labeled fault data, the mapping relationship between multi-source data is established, and the Dempster-Shafer multi-source fusion theory is used to aggregate the multi-modal fault judgment results to accurately diagnose the equipment fault. The application effectively improves the accuracy of fault diagnosis and the diversification of fault types, reduces the repair rate, and has good engineering application prospect.

Description

Technical Field

[0001] The present invention belongs to the research field of power grid equipment fault diagnosis, and specifically relates to a method for aggregating fault information and accurately obtaining charging equipment faults based on the Dempster-Shafer multi-source fusion theory in the case of multi-source data. Background Art

[0002] With the increasing maturity of power batteries and fast-charging technologies, electric vehicles have become a vital component of the automotive market. Due to the interdependent development of charging piles and electric vehicles, their development is almost synchronized. To further promote the development of the electric vehicle market, it is necessary to improve the installation and maintenance of charging piles and conduct a detailed analysis of the operation and fault diagnosis system design of electric vehicle charging piles.

[0003] Traditional fault diagnosis methods usually determine the location of faulty components based on the amount of information from charging pile protection equipment and combined with intelligent algorithms. Commonly used methods include artificial neural networks, expert systems, Bayesian theory, etc.

[0004] However, this approach places high demands on the accuracy of charging pile protection device information. This information accuracy significantly impacts the final fault diagnosis results, and the uncertainty of this information reduces the accuracy of the fault diagnosis method. To address the shortcomings of these traditional charging equipment fault diagnosis methods, the Dempster-Shafer multi-source fusion theory is employed to effectively fuse the fault information of charging equipment, enabling efficient identification of charging equipment faults and significantly improving the accuracy of fault diagnosis. Summary of the Invention

[0005] The present invention aims to effectively improve the accuracy of fault diagnosis of charging equipment, present the diversity of faults, and effectively reduce the return repair rate of maintenance personnel.

[0006] To achieve this technical purpose, the present invention adopts the following technical solutions:

[0007] The present invention provides a charging pile fault diagnosis method for multi-source modal data fusion, the method comprising the following steps:

[0008] Collect multiple modal data related to charging equipment failures from charging stations;

[0009] Pre-process each type of modal data based on charging equipment anomalies, select a model for training based on the characterization characteristics of each type of modal data, and identify charging equipment failure anomalies;

[0010] Based on the annotated fault anomalies, a mapping relationship between multi-source data representations is established and model parameters are updated;

[0011] The determination results of the equipment fault information are aggregated based on a Dempster-Shafer multi-source fusion theory, and the equipment fault information is accurately determined.

[0012] Further, various modal data associated with charging equipment faults are collected by the charging field station, and the method includes the following steps:

[0013] Step 1: Collecting charging equipment fault associated message stream data by the charging field station, including output current, output voltage, output power, charging interface temperature, etc. of the charging pile;

[0014] Step 2: Collecting charging equipment fault associated audio data by the charging field station, including audio data, sensor stream data, etc. ;

[0015] Step 3: Collecting charging equipment fault associated infrared sensing data by the charging field station, including infrared sensing pictures, gas imaging pictures, etc.

[0016] Further, various modal data associated with charging equipment faults are collected by the charging field station, and the method includes the following steps:

[0017] Step 1: Preprocessing the message stream data collected by the charging field station to improve data quality and support subsequent fault diagnosis. The preprocessing includes cleaning invalid and redundant data, repairing null and abnormal values to ensure data consistency and accuracy; removing noise through low-pass filtering or Kalman filtering to enhance data stability; detecting and removing outliers using statistical analysis methods to prevent interference with fault diagnosis; using interpolation algorithm or time series modeling to complete missing data, thereby maintaining data continuity and improving analysis accuracy;

[0018] Step 2: Using random forest algorithm to represent and model learning of the message data of the charging pile, to identify charging equipment fault anomalies;

[0019] Step 3: Using convolutional neural network to represent and model learning of the audio data, to identify charging equipment fault anomalies;

[0020] Step 4: Using image recognition algorithm YOLO-v8 to represent and model learning of the image data, to identify charging equipment fault anomalies.

[0021] Further, random forest algorithm is used to represent and model learning of the message data of the charging pile, to identify charging equipment fault anomalies, and the method includes the following steps:

[0022] A random forest algorithm is used to process the current, voltage, and power data of charging piles to identify potential failure modes and improve diagnostic accuracy. This step includes data preprocessing, feature selection, and classification model construction.

[0023] The collected current, voltage and power data are set as I={i1,i2,...,i n}、V={v1,v2,...,v n} and P={p1,p2,…,p n}, where n is the number of samples. First, the data is standardized to ensure consistency across variables and scales. The standardization formula is:

[0024]

[0025] Where x is the original value, u is the mean, σ is the standard deviation, and the standardized current, voltage, and power are denoted as I′, V′, and P′, respectively.

[0026] The random forest algorithm is used to select features of the preprocessed data and the importance of each feature is evaluated by constructing multiple decision trees. k (k=1,2,…,K, where K is the number of decision trees) make independent judgments on the samples and record the Gini impurity or information gain of each feature. Feature X j Importance of Imp(X j ) can be expressed as the average contribution of the feature in all decision trees:

[0027]

[0028] Where: ΔGini k (X j ) represents feature X j The reduction in Gini impurity in the kth tree. By filtering, a collection of features with high fault relevance is obtained, which improves the model's judgment on fault diagnosis.

[0029] Finally, the filtered feature set is input into the random forest classification model, the model is trained using labeled historical data, and the model performance is optimized by adjusting model parameters (such as the number of trees K and the maximum depth of the tree). The random forest outputs the classification results through an ensemble learning mechanism, and the final fault diagnosis result is generated by the majority voting method. Assume that the charging pile fault category is C = {c1, c2, ..., c m}, the fault category output by the model is:

[0030]

[0031] Where: Υ is the indicator function, when T kthe predicted class of T is c when Y(T k = c) = 1, otherwise 0. The final output of the network is is the fault diagnosis result of the charging pile.

[0032] Further, the audio data is characterized and model learned by using a convolutional neural network to identify the charging device fault anomaly, which includes the following steps:

[0033] Step 1: In order to accurately locate the charging device fault, the distance between the discharge sound size judgment measurement point and the actual fault point is obtained by collecting and processing. A large amount of environmental noise is mixed in the collected discharge sound signal, which will affect the accurate judgment of the discharge sound size, so the collected signal needs to be denoised.

[0034] Step 2: Assuming that the noise spectrum and the sound spectrum are Gaussian distribution, the conditional probability theory can be used to obtain:

[0035]

[0036] In the formula: ξ(n,d) is the prior signal-to-noise ratio of the dth frequency component of the nth frame, represents the estimated dth spectrum component of the nth frame in the sound signal. The prior signal-to-noise ratio ξ(n,d) is unknown, and a non-causal method is used to estimate the prior signal-to-noise ratio. Based on the least mean square error method of the logarithmic spectrum, a sound denoising algorithm based on deep learning is proposed. The entire denoising algorithm includes network training and sound denoising two parts: in the network training part, the logarithmic power spectrum eigenvalue of the signal is used as the input of the network, the network structure is built according to the actual characteristics of the signal, the learning algorithm and transfer function used by the network are determined, and a large number of sample sets are used to train the network; in the sound denoising part, the eigenvalue of a segment of noisy discharge sound signal is input into the trained network to obtain the estimated clean sound signal eigenvalue, and then the time domain signal of the clean discharge sound is obtained through the waveform reconstruction method;

[0037] Step 3: Based on the clean discharge sound signal of the charging device, after amplification, filtering and A / D conversion, it is transmitted to the embedded processor through the high-speed SPI interface. The processor analyzes and denoises the data. The sound signal after denoising is output to the earphone after D / A conversion. The information of the charging device fault such as sound signal transformation trend and magnetic field signal strength is displayed through the LCD, which can effectively locate the fault point of the charging device.

[0038] Step 4: Based on the convolutional neural network (CNN), the preprocessed audio modal data is diagnosed to realize efficient identification and accurate classification of the charging device fault. This process includes steps such as audio feature extraction, CNN model construction, feature learning and fault classification.

[0039] First, the noise-reduced audio signal is converted into a time-frequency plot (e.g., a spectrogram) or a mel-spectrogram. The audio signal is processed by a feature extraction procedure to generate a two-dimensional representation, which enables the CNN to capture the spatial patterns and frequency variations therein. Let the two-dimensional feature matrix of the audio signal be X, where each row represents a different frequency component and each column represents the signal intensity at a time step.

[0040] A convolutional neural network model is constructed to learn the spatial patterns in the audio features, which comprises multiple convolutional layers, pooling layers, and fully connected layers. In each convolutional layer, multiple convolution kernels are applied to convolve the input feature matrix, and the convolutional feature map is obtained by the following equation:

[0041]

[0042] where W is the convolution kernel weight, b is the bias term, X' is the output feature map, and (i, j) is the feature map coordinate. The convolutional layer learns the local spatial features through the weight sharing mechanism, effectively extracting the correlated feature information in the frequency and time dimensions.

[0043] After the convolution operation, the pooling layer reduces the dimension and compresses the feature map, focusing on the positions with the highest information content. The pooling operation can be represented as:

[0044] Y i,j = max(X2 i:2i+2,2j:2j+2 ),

[0045] where Y represents the output feature map after pooling, and 2i:2i+2, 2j:2j+2 represent the size of the pooling window, which is usually 2x2.

[0046] After multiple layers of convolution and pooling, the model obtains a highly compressed representation with important features. These high-level feature representations are passed to the fully connected layer through a flattening operation. Let the activation value of the fully connected layer be z k (corresponding to the score of class c k ), and the Softmax function is used to generate the probability distribution of each class:

[0047]

[0048] where P(c k | X) is the probability that the input audio feature belongs to fault class c k , z k is the score of fault class c k (output by the fully connected layer), m is the number of all possible fault classes, and exp(z k ) and Represents category c k The index score of and the index sum of all category scores.

[0049] Final predicted fault category Determined by the class with the highest probability:

[0050]

[0051] Where: The CNN model is used to detect abnormal patterns in the audio signals of charging equipment, enabling high-precision fault diagnosis and real-time warning.

[0052] Furthermore, the image recognition algorithm YOLO-v8 is used to characterize the image data and learn a model to identify charging equipment fault anomalies. The method includes the following steps:

[0053] Step 1: Collect and pre-process infrared sensor data and convert it into an image format suitable for the input model to represent the temperature status of each part of the charging pile. Where h and w are the height and width of the image, and each pixel represents the temperature distribution of different parts of the charging pile, reflecting potential temperature anomalies. The preprocessed image I is input into the YOLO-v8 network structure to detect possible fault areas;

[0054] Step 2: Use the YOLO-v8 model to perform target detection on the preprocessed infrared image I and identify the fault area. The YOLO-v8 network consists of three modules: backbone, neck, and head, to achieve multi-scale feature extraction and target localization. First, the backbone module extracts multi-scale features of the image through multiple layers of convolution. The convolution operation for feature extraction can be expressed as:

[0055]

[0056] Where: F i,j is the eigenvalue of the output feature map after convolution at position (i, j), W is the convolution kernel weight matrix, and b is the bias term.

[0057] In the neck module, upsampling and downsampling operations are used to fuse the feature maps, thereby enhancing the expressiveness of features at each layer, especially the features of the fault area.

[0058] The head module of the YOLO-v8 model generates predicted bounding boxes, each of which contains the coordinates (x, y, w, h) of the candidate fault area and the classification confidence. The total loss function L for target detection is total Comprehensive classification loss, positioning loss and confidence loss:

[0059] L total =L cls +L loc +L conf

[0060] wherein: L cls is a classification loss, measuring the accuracy of the model in identifying the fault type; L loc is a localization loss, used to evaluate the accuracy of the predicted position of the bounding box; L conf is a confidence loss, reflecting the degree of confidence of the model in the fault area.

[0061] Through Non-Maximum Suppression (NMS), the bounding boxes with high overlap are filtered, and the bounding box B = (x, y, w, h) with the highest confidence is retained, which is specifically represented as:

[0062]

[0063] wherein: Confidence(B k ) is the confidence score of the bounding box B k . The final output bounding box is the detected fault area, and the model can accurately identify and locate the abnormal area in the infrared image, providing reliable support for fault diagnosis and early warning of charging piles.

[0064] Further, according to the labeled fault anomaly, a mapping relationship between multi-source data representations is established, and the model parameters are updated, which includes the following steps:

[0065] Step 1: For message stream data, a multi-source data mapping technology is realized by using the Expectation-Maximization algorithm in machine learning, and the concept of matching graph M is introduced, which is defined as a function of parameter θ (matching rule) by a likelihood function:

[0066] L(θ; M) = Pr(M | θ),

[0067] wherein: Pr(M | θ) represents the "precision" of entity matching, which is obtained by evaluating the divergence of the matching graph, and each parameter θ corresponds to a matching graph M. By combining all M, the final integrated matching graph M can be obtained. In order to maximize Precision(M), a higher threshold is set for each Precision(M | θ) in this method. Once the threshold is determined, the number of M and the matching rule will also be determined;

[0068] Step 2: For audio modality data, attribute mapping is performed by artificial marking method, and characteristic quantities such as audio waveform and sensor stream data are extracted, and based on subjective judgment of the concept expressed by the mapping category, the mapping relationship is artificially established, and in the mapping tool, the upper and lower categories corresponding to each category are displayed in a tree structure, and a special area is left for the display of the category with category annotations, helping the mapper to complete the mapping between the audio modality data and the remaining modalities.

[0069] Step 3: For infrared sensor image data, attribute mapping is performed by artificial marking method, and characteristic quantities such as image data and video data are extracted, and based on subjective judgment of the concept expressed by the mapping category, the mapping relationship is artificially established, and in the mapping tool, the upper and lower categories corresponding to each category are displayed in a tree structure, and a special area is left for the display of the category with category annotations, helping the mapper to complete the mapping between the image modality data and the remaining modalities.

[0070] Further, the determination results of the equipment fault information are aggregated based on the Dempster-Shafer multi-source fusion theory to accurately determine the equipment fault information, and the method comprises the following steps:

[0071] Step 1: The fault determination results of the multi-class model are aggregated by using the Dempster-Shafer multi-source fusion theory, and the recognition frame of the non-empty set is assumed to be Θ={x1,x2,…,x n} and the possible subsets are 2 Θ If m: 2 Θ →[0,1] can satisfy the formula:

[0072]

[0073] In the formula: is an empty proposition, m represents a basic confidence distribution, and for any subset A, m(A) represents the basic confidence number of A;

[0074] Step 2: Assuming that m1,m2,…,m n represent the basic probability in the recognition frame Θ, the following formula is established:

[0075]

[0076] In the formula: is the conflict weight, when K=∞, it means that the argument is not applicable and the fault cannot be determined; when K<∞, it means that the evidence supporting the fault is generally consistent and the fault can be determined.

[0077] Further, the application provides a charging pile fault diagnosis method and system for multi-source modal data fusion, characterized in that the method comprises the following steps:

[0078] A multi-source data collection unit is configured to collect various modal data associated with charging equipment faults by using charging stations and terminal devices;

[0079] A data processing unit is configured to pre-process various modal data for each work order in the charging equipment;

[0080] A model training unit is configured to select a model based on the characteristic factors of the pre-processed various modal data, and train a charging equipment fault anomaly model;

[0081] A model updating unit is configured to establish a mapping relationship between multi-source data representations according to the labeled fault anomalies, and update the model parameters;

[0082] A fault prediction unit is configured to aggregate the determination results of the equipment fault information based on the Dempster-Shafer multi-source fusion theory, and accurately determine the charging equipment fault information.

[0083] The application has the following beneficial technical effects: The application is a charging pile fault diagnosis method and system for multi-source modal data fusion, which effectively improves the accuracy of charging equipment fault diagnosis and presents the diversity of faults, effectively reduces the repair rate of maintenance personnel, and has good engineering application prospects.

[0084] In order to more fully demonstrate the technical effects of the application and its innovation and superiority in the field of fault diagnosis, the following four aspects are described in detail, including theoretical support, data support, multi-dimensional advantage description and engineering application prospect, so as to fully reflect the significant advantages of the application in improving the accuracy, coverage and practicability of charging pile fault diagnosis.

[0085] 1. Theoretical support

[0086] The application realizes accurate representation and efficient classification of multi-modal data of charging equipment by adopting advanced algorithms such as random forest algorithm, convolutional neural network (CNN) and YOLO-v8 target detection model. The random forest algorithm can select and classify multi-dimensional message stream data such as current, voltage and power of the charging pile by constructing multiple decision trees, improve the sensitivity of the model to key features, and thus improve the accuracy of fault diagnosis; the CNN model extracts time-frequency domain features from audio data, which can effectively capture abnormal patterns in fault sound; the YOLO-v8 model extracts multi-scale features and accurately locates the fault area in the infrared image data, further improving the spatial accuracy of fault detection. By aggregating the diagnosis results of multi-modal data based on Dempster-Shafer multi-source fusion theory, the application can accurately judge the fault of the charging equipment under uncertain information, significantly improving the overall reliability and consistency of fault diagnosis.

[0087] 2. Data support

[0088] To verify the effectiveness of the application, a large number of experimental tests were conducted. The experimental results show that, compared with traditional fault diagnosis methods, the application improves the charging pile fault diagnosis accuracy by more than 15%, while the misjudgment rate is reduced by more than 20%. In complex working environments (such as high noise, partial data missing, etc.), the application effectively ensures the stability of the diagnosis results through data preprocessing and multi-modal fusion mechanism. In addition, the application can achieve more than 95% recognition accuracy for various fault types (such as current anomaly, contact temperature anomaly, infrared thermal image anomaly), indicating that it has a significant advantage in handling complex fault scenarios.

[0089] 3. Multi-dimensional advantage description

[0090] Fault diagnosis accuracy: The application introduces the feature selection and classification mechanism of the random forest algorithm and the target detection ability of the YOLO-v8 model, which can accurately locate the fault source and significantly improve the reliability of the diagnosis results. The fusion of multi-modal data makes up for the fault types that may be missed or misdiagnosed by a single modality.

[0091] Complex environment adaptability: In view of the limitations of traditional methods in the presence of environmental noise interference or data missing, the application realizes noise reduction processing of audio modality through Kalman filtering and deep learning noise reduction algorithm, and completes the missing data through time series modeling, thereby ensuring the continuity and reliability of fault diagnosis.

[0092] Wide coverage of fault types: The application can handle message stream data, audio data and infrared sensing data at the same time, covering various charging equipment fault types such as electrical faults, mechanical faults and environmental faults, realizing the diversity and comprehensiveness of fault diagnosis.

[0093] Maintenance cost reduction: By improving fault diagnosis accuracy and fault location efficiency, the invention effectively reduces the repair situation caused by misdiagnosis or missed diagnosis, greatly reducing the workload and maintenance cost of maintenance personnel.

[0094] 4. Engineering application prospect

[0095] With the popularization of new energy vehicles, the demand for charging piles industry is growing exponentially. The invention combines multi-modal data fusion technology to provide a highly intelligent solution for complex fault scenarios that traditional methods cannot cover, significantly improving the reliability and user experience of charging pile equipment. In actual engineering, the invention has been successfully applied to the trial operation of multiple charging sites, with a diagnosis efficiency improved by more than 30% compared to traditional methods, and a maintenance cost reduced by 25%. At the same time, the universality and scalability of the invention make it applicable to other power equipment fault diagnosis scenarios, with good market prospects and engineering practical value. BRIEF DESCRIPTION OF DRAWINGS

[0096] The invention will be further described below in conjunction with the drawings and examples, wherein:

[0097] Figure 1 is a flowchart of multi-modal based charging equipment fault diagnosis;

[0098] Figure 2 is a flowchart of fault diagnosis method based on various modal data for determining charging equipment faults; DETAILED DESCRIPTION

[0099] To further illustrate the technical solutions of the present application, the embodiments of the present application will be further described in detail below in conjunction with the drawings and specific embodiments. It should be noted that the following is only the preferred embodiment of the present application, and for those skilled in the art, without departing from the technical principles of the present application, a number of improvements and modifications can be made, which should be considered as the protection scope of the present application.

[0100] Example 1: As shown in Figure 1 is a flowchart of multi-modal based charging equipment fault diagnosis provided by the embodiment of the present application, which includes the following steps:

[0101] Step 1: Collect various modal data associated with charging equipment faults from charging stations;

[0102] Step 1 in this embodiment specifically includes the following steps:

[0103] Step 1.1: The charging station collects message flow data related to charging equipment failure, including the output current, output voltage, output power, and charging interface temperature of the charging pile;

[0104] Step 1.2: The charging station collects audio data related to the charging equipment failure, including audio data, sensor stream data, etc.

[0105] Step 1.3: The charging station collects infrared sensor data related to the charging equipment failure, including infrared sensor images, gas imaging images, etc.

[0106] Step 2: Preprocess each type of modal data based on the charging equipment anomaly, select a model for training based on the characterization characteristics of each type of modal data, and identify charging equipment failure anomalies;

[0107] In this embodiment, step 2 specifically includes the following steps:

[0108] Step 2.1: Preprocess the message flow data collected by the charging station to improve data quality and support subsequent fault diagnosis. Preprocessing includes cleaning invalid and redundant data and repairing null values ​​and outliers to ensure data consistency and accuracy; removing noise through low-pass filtering or Kalman filtering to enhance data stability; using statistical analysis methods to detect and remove outliers to prevent them from interfering with fault diagnosis; and using interpolation algorithms or time series modeling to fill in missing data to maintain data continuity and improve analysis accuracy.

[0109] Step 2.2: Use the random forest algorithm to characterize and model the charging pile message data to identify charging equipment failure anomalies;

[0110] Step 2.3: Use a convolutional neural network to characterize and model the audio data to identify charging equipment failure anomalies.

[0111] Step 2.4: Use the image recognition algorithm YOLO-v8 to characterize the image data and learn the model to identify charging equipment fault anomalies.

[0112] In this embodiment, step 2.2 specifically includes the following steps:

[0113] A random forest algorithm is used to process the current, voltage, and power data of charging piles to identify potential failure modes and improve diagnostic accuracy. This step includes data preprocessing, feature selection, and classification model construction.

[0114] The collected current, voltage and power data are set as I={i1,i2,...,i n}、V={v1,v2,...,v n} and P={p1,p2,…,pn}, where n is the number of samples. First, the data is standardized to ensure consistency across different variables and scales. The standardization formula is:

[0115]

[0116] where x is the original value, u is the mean, and σ is the standard deviation. The normalized current, voltage, and power are denoted as I', V', and P', respectively.

[0117] The pre-processed data is then used to select features using the random forest algorithm. Random forest evaluates the importance of each feature by constructing multiple decision trees. Each decision tree T k (k = 1, 2,..., K, where K is the number of decision trees) independently judges the samples and records the Gini impurity or information gain of each feature. The importance of feature X j , denoted as Imp(X j ), can be represented as the average contribution of this feature across all decision trees:

[0118]

[0119] where ΔGini k (X j ) represents the reduction in Gini impurity of feature X j in the kth tree. By screening, a high-fault-related feature set is obtained, which improves the model's judgment of fault diagnosis.

[0120] Finally, the selected feature set is input into the random forest classification model, which is trained using the labeled historical data. The model performance is optimized by adjusting the model parameters, such as the number of trees K and the maximum depth of the tree. The random forest outputs the classification result through the ensemble learning mechanism. The final fault diagnosis result is generated by majority voting. Let the charging pile fault categories be C = {c1, c2,..., c m}, then the model output fault category is:

[0121]

[0122] where Y is the indicator function, Y(T k = c) = 1 when the predicted class of T k is c, otherwise 0. The final output is the fault diagnosis result of the charging pile.

[0123] The specific steps of step 2.3 in this embodiment are as follows:

[0124] Step 2.3.1: In order to accurately locate the fault of the charging equipment, the distance between the measured discharge sound size and the actual fault point needs to be determined. The collected discharge sound signal contains a lot of environmental noise, which can affect the accuracy of the discharge sound size judgment. Therefore, noise reduction processing is needed for the collected signal.

[0125] Step 2.3.2: Assuming that the noise spectrum and the sound spectrum are both Gaussian distributions, the conditional probability theory can be used to obtain:

[0126]

[0127] In the formula: ξ(n,d) is the prior signal-to-noise ratio of the nth frame of the dth frequency component, represents the estimated dth spectral component of the nth frame in the sound signal. The prior signal-to-noise ratio ξ(n,d) is unknown, and a non-causal method is used to estimate it. Based on the log-spectral minimum mean square error method, a sound noise reduction algorithm based on deep learning is proposed. The entire noise reduction algorithm includes network training and sound noise reduction: In the network training part, the log power spectrum eigenvalue of the signal is used as the input of the network, the network structure is built according to the actual characteristics of the signal, the learning algorithm and transfer function used by the network are determined, and a large number of sample sets are used to train the network. In the sound noise reduction part, the eigenvalues of a segment of noisy discharge sound signal are input into the trained network to obtain the estimated clean sound signal eigenvalues, and then the time domain signal of the clean discharge sound is obtained through waveform reconstruction.

[0128] Step 2.3.3: Based on the clean discharge sound signal of the charging equipment, after amplification, filtering and A / D conversion, the signal is transmitted to the embedded processor through the high-speed SPI interface. The processor analyzes and denoises the data, and the denoised sound signal is output to the earphone after D / A conversion. The information of the charging equipment fault such as sound signal transformation trend and magnetic field signal strength is displayed through the LCD, which can effectively locate the fault point of the charging equipment.

[0129] Step 2.3.4: Based on the convolutional neural network (CNN), the preprocessed audio modal data is used for fault diagnosis to realize efficient recognition and accurate classification of charging equipment faults. This process includes audio feature extraction, CNN model construction, feature learning and fault classification.

[0130] Firstly, the denoised audio signal is converted into a time-frequency graph (such as a frequency spectrum graph obtained by short-time Fourier transform) or a mel-frequency spectrum graph. The audio signal is generated into a two-dimensional representation through the feature extraction process, so that the CNN can capture the spatial patterns and frequency changes in it. Let the two-dimensional feature matrix of the audio signal be X, where each row represents a different frequency component and each column represents the signal intensity at a time step.

[0131] The convolutional neural network model is constructed to learn the spatial patterns in audio features, and the CNN structure contains multiple convolutional layers, pooling layers and fully connected layers. In each convolutional layer, multiple convolution kernels are applied to the input feature matrix for convolution operation, and the convolution feature map is obtained by the following formula:

[0132]

[0133] In the formula, W is the convolution kernel weight, b is the bias term, X' is the output feature map, and (i, j) is the feature map coordinate. The convolutional layer learns the local spatial features through the weight sharing mechanism, effectively extracting the features associated in the frequency and time dimensions.

[0134] After the convolution operation, the pooling layer reduces the dimension and compresses the feature map, focusing on the position with the highest information amount. The pooling operation can be represented as:

[0135] Y i,j =max(X2′ i:2i+2,2j:2j+2 ),

[0136] In the formula, Y represents the output feature map after pooling, and 2i:2i+2, 2j:2j+2 represents the size of the pooling window, usually 2x2.

[0137] After multiple convolution and pooling, the model obtains a highly compressed and important feature representation. These high-level feature representations are passed to the fully connected layer through the flattening operation. The activation value of the fully connected layer output is z k (corresponding to the score of class c k ), and the Softmax function is used to generate the probability distribution of each class:

[0138]

[0139] In the formula, P(c k |X) is the probability that the input audio feature belongs to fault class c k , z k is the score of fault class c k (output by the fully connected layer), m is the number of all possible fault classes, exp(z k ) and represent the exponential score of class c k and the exponential sum of all class scores, respectively.

[0140] The final predicted fault class is determined by the class with the maximum probability:

[0141]

[0142] In the formula: The diagnostic fault category label. Through the convolutional neural network, the model can capture abnormal patterns in the charging device audio signal, realize high-precision diagnosis and real-time warning of faults.

[0143] The specific steps of step 2.4 in this embodiment are as follows:

[0144] Step 2.4.1: Collect and preprocess infrared sensor data, and convert it into an image format suitable for inputting into the model to represent the temperature state of each part of the charging pile. Let the infrared image Where h and w are the height and width of the image, and each pixel represents the temperature distribution of different parts of the charging pile, reflecting potential temperature abnormalities. The preprocessed image I is input into the YOLO-v8 network structure to detect possible fault areas;

[0145] Step 2.4.2: Use the YOLO-v8 model to detect the preprocessed infrared image I to identify fault areas. The YOLO-v8 network includes three modules: backbone, neck, and head, to achieve multi-scale feature extraction and target positioning. First, the backbone module extracts multi-scale features of the image through multiple convolution layers. The convolution operation for feature extraction can be represented as:

[0146]

[0147] In the formula: F i,j is the feature value of the output feature map after convolution at position (i,j), W is the convolution kernel weight matrix, and b is the bias term.

[0148] In the neck module, up-sampling and down-sampling operations are used to fuse features, thereby enhancing the expression ability of features at each layer, especially the features of fault areas.

[0149] The head module of the YOLO-v8 model generates predicted bounding boxes, each containing the coordinates (x,y,w,h) of the candidate fault area and the classification confidence. The total loss function L total The classification loss, positioning loss, and confidence loss are combined:

[0150] L total =L cls +L loc +L conf

[0151] In the formula: L cls is the classification loss, which measures the accuracy of the model in identifying fault types; L loc is the positioning loss, which is used to evaluate the accuracy of the predicted position of the bounding box; Lconf For the confidence loss, the confidence degree of the model to the fault area is reflected.

[0152] Through Non-Maximum Suppression (NMS), the boundary box with high overlap is screened, and the boundary box B=(x, y, w, h) with the highest confidence is reserved, which is specifically represented as:

[0153]

[0154] In the formula, Confidence(B k ) is the confidence score of the boundary box B k . The final output boundary box is the detected fault area, and the model can accurately identify and locate the abnormal area in the infrared image, providing reliable support for fault diagnosis and early warning of the charging pile.

[0155] Step 3: According to the labeled fault anomaly, the mapping relationship between the multi-source data representations is established, and the model parameters are updated;

[0156] In this embodiment, step 3 specifically includes the following steps:

[0157] Step 3.1: For the message stream data, the expectation maximization algorithm in machine learning is used to realize the multi-source data mapping technology, and the concept of matching graph M is introduced, and a function about parameter θ (matching rule) is defined by the likelihood function:

[0158] L(θ;M)=Pr(M|θ),

[0159] In the formula, Pr(M|θ) represents the "precision" of entity matching, which is obtained by evaluating the divergence of the matching graph, and each parameter θ corresponds to a matching graph M. By combining all M, the final integrated matching graph M can be obtained. In order to maximize Precision(M), a higher threshold is set for each Precision(M|θ). Once the threshold is determined, the number of M and the matching rule will also be determined;

[0160] Step 3.2: For audio modal data, attribute mapping is performed by artificial identification method, and the feature quantities of audio modal data including audio waveform, sensor stream data, etc. are extracted. Based on the subjective judgment of the concept expressed by the mapping category, the mapping relationship is artificially established. In the mapping tool, a tree structure is used to display the corresponding upper and lower categories in each category, and a special area is left for the display of the categories with annotations, helping the mapper to complete the mapping between the audio modal data and the remaining modal data.

[0161] Step 3.3: For the infrared sensing image data, attribute mapping is performed by using a manual marking method, feature quantities of image modal data including image data, video data and the like are extracted, a mapping relationship is manually established based on subjective judgment of a concept expressed by a mapping category, a tree structure is used in a mapping tool to display upper and lower categories corresponding to each category, and a special area is left for a category with a category note to display, thereby helping a mapper to complete mapping between image modal data and other modal data.

[0162] Step 4: The determination result of the equipment fault information is aggregated by using the Dempster-Shafer multi-source fusion theory, and the equipment fault information is accurately determined.

[0163] Step 4 in the embodiment specifically includes the following steps:

[0164] Step 4.1: The fault determination results of the multi-class model are aggregated by using the Dempster-Shafer multi-source fusion theory, and it is assumed that the recognition frame of a non-empty set is Θ={x1,x2,…,x n}, and possible subsets thereof are 2 Θ If m: 2 Θ →[0,1] can satisfy the formula:

[0165]

[0166] In the formula: represents an empty proposition, m represents a basic confidence distribution, and for any subset A, m(A) represents a basic confidence number of A;

[0167] Step 4.2: It is assumed that m1,m2,…,m n represent basic probabilities in the recognition frame Θ, and the following formula is established:

[0168]

[0169] In the formula: represents a conflict weight value, when K=∞, it indicates that the argument is not applicable and the fault cannot be determined; and when K<∞, it indicates that the evidence supporting the fault is generally consistent, and the fault can be determined.

[0170] The embodiment also provides a charging pile fault diagnosis method and system facing multi-source modal data fusion, and the method and system are characterized in that they include the following steps:

[0171] A multi-source data collection unit is configured to collect various modal data associated with charging equipment faults by using charging stations and terminal equipment;

[0172] A data processing unit is configured to respectively preprocess various modal data for work orders in the charging equipment;

[0173] a model training unit configured to select a model for representing characteristics of feature factors processed based on various types of modal data, and train a charging device fault anomaly model;

[0174] a model updating unit configured to establish a mapping relationship between representations of multi-source data according to labeled fault anomalies, and update model parameters;

[0175] a fault prediction unit configured to aggregate determination results of device fault information based on Dempster-Shafer multi-source fusion theory, and accurately determine charging device fault information.

[0176] The embodiments of the present application are described above with reference to the drawings; however, the present application is not limited to the specific embodiments described above, which are merely illustrative rather than restrictive, and a person of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims.

Claims

1. A charging pile fault diagnosis method for multi-source modal data fusion, characterized in that: The method comprises the following steps: Step 1: Collect multiple modal data related to charging equipment failures at charging stations; Step 2: Preprocess each type of modal data separately, select a model for training based on the characterization characteristics of each type of modal data, and identify charging equipment fault anomalies; Step 3: Based on the annotated fault anomalies, establish a mapping relationship between multi-source data representations and update the model parameters; Step 4: Aggregate the judgment results of equipment fault information based on the Dempster-Shafer multi-source fusion theory to accurately judge the equipment fault information; Step 1 is as follows: Step 1.1: The charging station collects message flow data related to charging equipment failure, including the output current, output voltage, output power, and charging interface temperature of the charging pile; Step 1.2: The charging station collects audio and video data related to the charging equipment failure, including audio data and sensor stream data; Step 1.3: The charging station collects infrared sensor data related to the charging equipment failure, including infrared sensor images and gas imaging images; Step 2 is as follows: Step 2.1: Preprocess the message flow data collected by the charging station to improve data quality and support subsequent fault diagnosis. Preprocessing includes cleaning invalid and redundant data and repairing null values ​​and outliers to ensure data consistency and accuracy. Low-pass filtering or Kalman filtering is used to remove noise and enhance data stability. Statistical analysis methods are used to detect and remove outliers to prevent them from interfering with fault diagnosis. Interpolation algorithms or time series modeling are used to fill in missing data to maintain data continuity and improve analysis accuracy. Step 2.2: Use the random forest algorithm to characterize and model the charging pile message data to identify charging equipment failure anomalies; Step 2.3: Use a convolutional neural network to characterize and model the audio data to identify charging equipment failure anomalies. Step 2.4: Use the image recognition algorithm YOLO-v8 to characterize the image data and learn the model to identify charging equipment fault anomalies; Step 3 is as follows: Step 3.1: For the packet flow data, the expectation maximization algorithm in machine learning is used to iteratively implement the multi-source data mapping technology. The concept of matching graph M is introduced, and a function with respect to the parameter θ is defined by the likelihood function: L(θ;M)=Pr(M|θ), Where: Pr(M|θ) represents the "precision" of entity matching, which is obtained by evaluating the divergence of the matching graph. Each parameter θ corresponds to a matching graph M. By merging all M, we can obtain the final integrated matching graph M. To maximize Precision(M), a threshold is set for each Precision(M|θ). Once the threshold is determined, the number of M and the matching rules will also be determined. Step 3.2: For audio modal data, attribute mapping is performed using a manual labeling method. The audio modal data, including audio waveforms and sensor stream data features, is extracted. Based on subjective judgment of the concepts expressed by the mapping categories, a mapping relationship is manually established. The mapping tool uses a tree structure to display the corresponding superordinate and subordinate categories for each category. Categories with existing category annotations are displayed in a dedicated area to help mappers complete the mapping between the audio modal data and other modalities. Step 3.3: For infrared sensor image data, attribute mapping is performed using a manual labeling method. The image modal data, including image data and video data feature quantities, are extracted. Based on subjective judgment of the concepts expressed by the mapping categories, a mapping relationship is manually established. A tree structure is used in the mapping tool to display the corresponding upper and lower classes in each class, and a special area is reserved for the corresponding display of categories with category annotations to help the mapper complete the mapping between image modal data and other modalities.

2. The charging pile fault diagnosis method for multi-source modal data fusion according to claim 1 is characterized in that: In step 2.2, the random forest algorithm is used to characterize and model the charging pile message data to identify charging equipment failure anomalies, including: The random forest algorithm is used to process the current, voltage and power data of the charging pile to identify potential failure modes and improve diagnostic accuracy. This step includes data preprocessing, feature selection and classification model construction. The collected current, voltage and power data are set as I={i1,i2,...,i n }、V={v1,v2,...,v n } and P={p1,p2,...,p n }, where n is the number of samples. First, the data is standardized to ensure the consistency of different variables and scales. The standardization formula is: Where: x is the original value, u is the mean, σ is the standard deviation, and the current, voltage, and power obtained after standardization are recorded as I′, V′, and P′ respectively. The random forest algorithm is used to select features of the preprocessed data, and the importance of each feature is evaluated by constructing multiple decision trees. k Make independent judgments on the samples, k = 1, 2, ..., K, where K is the number of decision trees, and record the Gini impurity or information gain of each feature, feature X j Importance of Imp(X j ) represents the average contribution of this feature in all decision trees: Where: ΔGini k (X j ) represents feature X j The reduction of Gini impurity in the k-th tree is used to obtain a collection of features with high fault correlation by screening, which improves the model's judgment on fault diagnosis. Finally, the filtered feature set is input into the random forest classification model, and the model is trained using the labeled historical data. The model performance is optimized by adjusting the model parameters, namely the number of trees K and the maximum depth of the tree. The random forest outputs the classification result through the ensemble learning mechanism, and the final fault diagnosis result is generated by the majority voting method. Assume that the charging pile fault category is C = {c1, c2, ..., c m }, the fault category output by the model is: Where: Υ is the indicator function, when T k When the predicted category is c, Υ(T k =c) = 1, otherwise 0, the final output This is the fault diagnosis result of the charging pile.

3. The charging pile fault diagnosis method for multi-source modal data fusion according to claim 2 is characterized in that: In step 2.3, a convolutional neural network is used to characterize and model the audio data to identify charging equipment failure anomalies, including: Step 2.3.1: To accurately locate the charging equipment fault, collect and process the discharge sound volume to determine the distance between the measurement point and the actual fault point. The collected discharge sound signal is mixed with a large amount of environmental noise, which will affect the accurate determination of the discharge sound volume. Therefore, the collected signal needs to be noise-reduced. Step 2.3.2: Assuming that both the noise spectrum and the sound spectrum are Gaussian, we can use conditional probability theory to obtain: Where: ξ(n,d) is the prior signal-to-noise ratio of the dth frequency component of the nth frame, It represents the dth spectral component of the nth frame in the estimated sound signal. The prior signal-to-noise ratio ξ(n,d) is unknown. The non-causal method is used to estimate the prior signal-to-noise ratio. Based on the logarithmic spectrum minimum mean square error method, a sound noise reduction algorithm based on deep learning is proposed. The entire noise reduction algorithm includes two parts: network training and sound noise reduction. In the network training part, the logarithmic power spectrum eigenvalue of the signal is used as the input of the network. The network structure is built according to the actual characteristics of the signal, the learning algorithm and transfer function adopted by the network are determined, and a large number of sample sets are used to train the network. In the sound noise reduction part, the eigenvalue of a noisy discharge sound signal is input into the trained network to obtain the estimated clean sound signal eigenvalue, and then the time domain signal of the clean discharge sound is obtained by waveform reconstruction. Step 2.3.3: The clean discharge sound signal from the charging device is amplified, filtered, and A / D converted before being transmitted to the embedded processor via a high-speed SPI interface. The processor analyzes and de-noises the data. The de-noised sound signal is then D / A converted and output to the headphones. The sound signal change trend and magnetic field signal strength, indicating the charging device fault, are displayed on the LCD, effectively locating the fault point of the charging device. Step 2.3.4: Perform fault diagnosis on the pre-processed audio modal data based on the convolutional neural network (CNN) to achieve efficient identification and accurate classification of charging equipment faults. This process includes audio feature extraction, CNN model construction, feature learning, and fault classification steps. First, the denoised audio signal is converted into a time-frequency graph or a Mel-spectrogram. The audio signal is extracted through a feature extraction process to generate a two-dimensional representation, which enables CNN to capture the spatial pattern and frequency changes. Let the two-dimensional feature matrix of the audio signal be X, where each row represents a different frequency component and each column represents the signal strength at a time step. Convolutional neural network models are constructed to learn spatial patterns in audio features. The CNN structure contains multiple convolutional layers, pooling layers, and fully connected layers. In each convolutional layer, multiple convolution kernels are applied to perform convolution operations on the input feature matrix. The convolution feature map is obtained by the following formula: Where: W is the convolution kernel weight, b is the bias term, X′ is the output feature map, (i, j) is the feature map coordinate, the convolution layer learns local spatial features through the weight sharing mechanism, and effectively extracts the feature information associated with the frequency and time dimensions. After the convolution operation, the pooling layer reduces the dimension and compresses the feature map, concentrating the features to the position with the highest information content. The pooling operation can be expressed as: AND i,j =max(X′ 2i:2i+2,2j:2j+2 ), Where: Y represents the output feature map after pooling, 2i:2i+2,2j:2j+2 represents the size of the pooling window, usually 2×2, After multiple layers of convolution and pooling, the model obtains a highly compressed representation with important features. These high-level feature representations are flattened and passed to the fully connected layer. The fully connected layer simulates the traditional neural network and maps these features to various fault categories. The activation value of the fully connected layer output is z k , corresponding to category c k The score of , uses the Softmax function to generate the probability distribution of each category: Where: P(c k |X) is the input audio feature belonging to the fault category c k The probability of z k Is fault category c k The score is output by the fully connected layer, m is the number of all possible fault categories, exp(z k )and Represents category c k The index score of and the index sum of all category scores, Final predicted fault category Determined by the class with the highest probability: Where: For fault category labels for diagnosis, the model can capture abnormal patterns in the audio signals of charging equipment through convolutional neural networks, achieving high-precision diagnosis and real-time warning of faults.

4. The charging pile fault diagnosis method for multi-source modal data fusion according to claim 3 is characterized in that: In step 2.4, the image recognition algorithm YOLO-v8 is used to characterize the image data and learn a model to identify charging equipment fault anomalies, including: Step 2.4.1: Collect and pre-process infrared sensor data and convert it into an image format suitable for the input model to represent the temperature status of each part of the charging pile. Where h and w are the height and width of the image. Each pixel represents the temperature distribution of different parts of the charging pile, reflecting potential temperature anomalies. The preprocessed image I is input into the YOLO-v8 network structure to detect possible fault areas. Step 2.4.2: Use the YOLO-v8 model to detect targets in the preprocessed infrared image I and identify the fault area. The YOLO-v8 network consists of three modules: backbone, neck, and head to achieve multi-scale feature extraction and target positioning. First, the backbone module extracts multi-scale features of the image through multi-layer convolution. The convolution operation of feature extraction can be expressed as: Where: F i,j is the eigenvalue of the output feature map at position (i, j) after convolution, W is the convolution kernel weight matrix, b is the bias term, In the neck module, upsampling and downsampling operations are used to fuse the feature maps, thereby enhancing the expressive power of each layer feature. The head module of the YOLO-v8 model generates predicted bounding boxes. Each bounding box contains the coordinates (x, y, w, h) of the candidate fault area and the classification confidence. The total loss function L for target detection is total Comprehensive classification loss, positioning loss and confidence loss: L total =L cls +L loc +L conf Where: L cls is the classification loss, which measures the accuracy of the model in identifying the fault type; L loc L is the positioning loss, which is used to evaluate the accuracy of the bounding box prediction position; conf is the confidence loss, which reflects the confidence level of the model in the fault area. Non-Maximum Suppression (NMS) is used to filter bounding boxes with high overlap, and the bounding box with the highest confidence is retained. It is specifically expressed as: Where: Confidence(B k ) is the bounding box B k The confidence score is calculated, and the final output bounding box is the detected fault area. The model can accurately identify and locate abnormal areas in infrared images, providing reliable support for fault diagnosis and early warning of charging piles.

5. A charging pile fault diagnosis system for multi-source modal data fusion, characterized by: To implement the charging pile fault diagnosis method for multi-source modal data fusion according to any one of claims 1 to 4, the system includes: A multi-source data collection unit, used to collect multiple modal data related to charging equipment failures using charging stations and terminal equipment; A data processing unit, used to pre-process various types of modal data respectively; Model training unit, which selects a model based on the characteristic factors represented by various modal data after prediction and processing, and is used to train the charging equipment fault anomaly model; A model updating unit establishes a mapping relationship between multi-source data representations based on the annotated fault anomalies and updates the model parameters; The fault prediction unit aggregates the judgment results of equipment fault information based on the Dempster-Shafer multi-source fusion theory to accurately judge the fault information of charging equipment.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the charging pile fault diagnosis method for multi-source modal data fusion as described in any one of claims 1 to 4 above is implemented.

Images