Charging pile fault detection method based on image recognition

Abstract

The invention discloses a charging pile fault detection method based on image recognition, and relates to the technical field of intelligent fault diagnosis, and the method comprises the steps: collecting the multi-modal image data of a charging pile through a multi-modal camera, carrying out the preprocessing, carrying out the feature extraction of a multi-modal image data set through deep learning, and obtaining a multi-modal image data set; fusing through a feature fusion algorithm to obtain a comprehensive feature vector, constructing a fault detection model, inputting the comprehensive feature vector, obtaining a detection result, constructing a fault prediction model by using a long short-term memory (LSTM) network, obtaining a comprehensive risk index, setting a threshold value, and taking corresponding measures according to the comprehensive risk index. Through multi-modal data acquisition, deep learning feature extraction, multilayer neural network detection, time sequence prediction and comprehensive risk assessment, a complete charging pile fault detection and prediction system is formed. The steps are linked with one another, so that the detection accuracy is ensured, the fault trend is effectively predicted, and a technical guarantee is provided for safe operation of the charging pile.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent fault diagnosis, and in particular to a charging pile fault detection method based on image recognition. Background Art

[0002] With the rapid development of new energy vehicles around the world, charging piles, as an important part of the new energy vehicle infrastructure, are expanding in construction scale and application scope. However, charging piles may be affected by many factors in long-term operation. Existing charging pile fault detection technologies mainly rely on traditional sensor monitoring and signal analysis methods. These methods usually collect a single data source of current, voltage, and temperature of the charging pile to determine whether there is an abnormality. They are often limited in complex environments, especially when facing comprehensive diagnosis of multiple fault types. Traditional methods are difficult to effectively identify the characteristics of different faults. In addition, existing technologies lack the ability to predict faults and cannot evaluate potential risks in advance, which may lead to the failure to timely troubleshoot hidden faults, and even cause safety accidents in serious cases. Therefore, how to design a more intelligent, efficient and comprehensive charging pile fault detection and prediction method has become the focus of current technology.

[0003] With the rapid development of artificial intelligence and computer vision technology, fault diagnosis technology based on image recognition has gradually been applied to the monitoring and maintenance of industrial equipment, especially multimodal image fusion technology. By simultaneously collecting visible light, infrared and ultraviolet image data, it can effectively make up for the limitations of a single data source. Visible light images can capture the appearance structure and surface damage of the equipment, infrared images can reflect the temperature distribution characteristics of the equipment, and ultraviolet images can identify the partial discharge signals of electrical equipment. However, most of the fault detection technologies based on image recognition are still insufficient in practical applications. The existing methods mostly use simple weighting or splicing methods for the fusion processing of multimodal image data, and fail to fully explore the complementarity between different modal data, resulting in insufficient expression of the comprehensive feature vector. The existing fault detection methods mostly focus on static detection, lack of deep modeling of time series information, and cannot predict fault trends. The model has poor adaptability and often needs to be retrained when facing new types of faults, which makes it difficult to meet the requirements of real-time and scalability in actual scenarios. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a charging pile fault detection method based on image recognition to solve the problem that existing fault detection methods mostly focus on static detection, lack of deep modeling of time series information, and cannot predict fault trends.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a charging pile fault detection method based on image recognition, which comprises:

[0008] Use visible light cameras, infrared cameras, and ultraviolet cameras to collect multimodal image data of charging piles and perform preprocessing;

[0009] Based on the preprocessed multimodal image dataset, deep learning is used to extract features from the multimodal image dataset, and a comprehensive feature vector is obtained by fusion through a feature fusion algorithm;

[0010] Construct a fault detection model based on the comprehensive feature vector, input the comprehensive feature vector, and obtain the detection result;

[0011] Based on the detection results, a fault prediction model is constructed using the long short-term memory network LSTM to obtain a comprehensive risk index;

[0012] Set thresholds and take corresponding measures based on the comprehensive risk index.

[0013] As a preferred solution of the charging pile fault detection method based on image recognition described in the present invention, wherein: the multimodal image data of the charging pile is collected by using a visible light camera, an infrared camera and an ultraviolet camera, and preprocessed, and the specific steps are:

[0014] Deploy visible light cameras, infrared cameras and ultraviolet cameras around the charging piles to collect visible light image data, infrared image data and ultraviolet image data of the charging piles;

[0015] Preprocessing visible light image data, infrared image data and ultraviolet image data;

[0016] The preprocessed multimodal image data is integrated into the multimodal image dataset x, and the expression is:

[0017] x={x1,x2,x3};

[0018] Among them, x1 represents the preprocessed visible light image data, x2 represents the preprocessed infrared image data, and x3 represents the preprocessed ultraviolet image data.

[0019] As a preferred solution of the charging pile fault detection method based on image recognition described in the present invention, wherein: based on the preprocessed multimodal image data set, deep learning is used to extract features from the multimodal image data set, and a comprehensive feature vector is obtained by fusion through a feature fusion algorithm, and the specific steps are as follows:

[0020] The ResNet-50 network is used to extract the visible light image data x1 to obtain the visible light feature vector f1;

[0021] The EfficientNet-B0 network is used to extract the infrared image data x2 and obtain the infrared feature vector f2;

[0022] The DenseNet-121 network is used to extract the ultraviolet image data x3 to obtain the ultraviolet feature vector f3;

[0023] The weight generation function based on the attention mechanism is used to obtain the weight α of each feature vector i ;

[0024] The three feature vectors are weighted fused to obtain the fused comprehensive feature vector F, which is expressed as:

[0025]

[0026] Among them, i represents the index variable of the feature vector, f i represents the i-th eigenvector, α i represents the weight of the i-th eigenvector.

[0027] As a preferred solution of the charging pile fault detection method based on image recognition of the present invention, wherein: the fault detection model is constructed based on the comprehensive feature vector, the comprehensive feature vector is input, and the detection result is obtained. The specific steps are:

[0028] The fault detection model is constructed using a deep neural network DNN. The comprehensive feature vector is input and the expression is:

[0029]

[0030] Among them, s1 represents the output of the first layer of neural network, represents the Swish activation function, W1 represents the weight matrix of the first layer of the neural network, and b1 represents the bias vector of the first layer of the neural network;

[0031] The output h1 of the first layer of the neural network is input into the second layer to continue the nonlinear mapping.

[0032] As a preferred solution of the charging pile fault detection method based on image recognition of the present invention, wherein: the output h1 of the first layer of neural network is input into the second layer to continue nonlinear mapping, and the expression is:

[0033]

[0034] Among them, s2 represents the output of the second layer neural network, W2 represents the weight matrix of the second layer neural network, and b2 represents the bias vector of the second layer neural network;

[0035] The final output layer expression is:

[0036] p = Softmax(W o ·s2+b o );

[0037] Among them, Softmax represents the probability distribution generating function, W o represents the weight matrix of the output layer, b o represents the bias vector of the output layer, and p represents the output of the fault detection model;

[0038] The final output detection result p={p1,p2,…,p k}, is the predicted probability of each fault category, where k represents the number of fault categories.

[0039] As a preferred solution of the charging pile fault detection method based on image recognition described in the present invention, wherein: based on the detection results, a long short-term memory network LSTM is used to build a fault prediction model to obtain a comprehensive risk index, and the specific steps are as follows:

[0040] The fault detection result at each time point is For any of them, it represents the probability of the jth type of failure at time point t, and j represents the index variable of the number of failure categories;

[0041] Construct a historical time window T to form a time series matrix V = [p 1 ,p 2 ,…,p T ], where V is a T×j matrix, representing the change in the failure probability of j categories at the past T time points;

[0042] Input the time series matrix V into the LSTM model to capture the long-term dependencies of the time series through the forget gate, input gate, and output gate;

[0043] The forget gate determines the historical information that needs to be discarded at the current moment. The time series matrix V is input into the forget gate to obtain the output vector g of the forget gate. t ;

[0044] The input gate determines the new information that needs to be added at the current moment. The time series matrix V is input into the input gate to obtain the output vector q of the input gate. t ;

[0045] At the same time, based on the time series matrix V, the hyperbolic tangent function is used to calculate the candidate memory unit state

[0046] Use the output vector q of the input gate t Multiply by the memory cell before the update and add the output vector q using the input gate t Multiply by the candidate memory cell state Get the updated memory unit C t ;

[0047] The output gate determines the output of the hidden state at the current moment, and the expression is:

[0048] l t =σ(W l ·[h t―1 ,V t ]+b l );

[0049] Among them, l t represents the output vector of the output gate, W l represents the weight matrix of the output gate, b l represents the bias vector of the output gate;

[0050] According to l t Update hidden state h t .

[0051] As a preferred solution of the charging pile fault detection method based on image recognition according to the present invention, wherein: t Update hidden state h t , the expression is:

[0052] h t = l t tanh(C t );

[0053] Through the hidden state h t , predict the failure probability distribution at future time points, the expression is:

[0054] a j (t) = σ(W a ·h t +b a );

[0055] Among them, a j (t) represents the probability of the jth fault occurring at time point t, W a represents the weight matrix of the output layer, b a Represents the bias vector of the output layer;

[0056] According to the probability a of the jth fault occurring at time point t j (t), calculate the comprehensive risk index, the expression is:

[0057]

[0058] Among them, R t represents the comprehensive risk index at time point t, w j represents the risk weight of the jth type of failure.

[0059] As a preferred solution of the charging pile fault detection method based on image recognition of the present invention, wherein: the threshold is set and corresponding measures are taken according to the comprehensive risk index, and the specific steps are:

[0060] Set the thresholds θ2 and θ2, when R t When ≤θ1, it means there is no risk and no measures are needed;

[0061] When θ2 <R t When <θ2, it indicates a small risk and the sampling and testing frequency should be increased;

[0062] When θ2≤R t When it is detected, it indicates a high risk, an alarm is sent to the user, and a professional maintenance team is contacted for on-site inspection.

[0063] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the charging pile fault detection method based on image recognition as described in the first aspect of the present invention is implemented.

[0064] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the charging pile fault detection method based on image recognition as described in the first aspect of the present invention is implemented.

[0065] The beneficial effects of the present invention are as follows: through multimodal data acquisition, deep learning feature extraction, multi-layer neural network detection, LSTM time series prediction and comprehensive risk assessment, a complete charging pile fault detection and prediction system is formed. Each step is closely linked to each other, which not only ensures the accuracy of detection, but also realizes the effective prediction of fault trends, providing technical guarantee for the safe and stable operation of charging piles. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0067] Figure 1This is a flow chart of the charging pile fault detection method based on image recognition in Example 1.

[0068] Figure 2 This is a schematic diagram of the fault prediction model in Example 1. DETAILED DESCRIPTION

[0069] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0070] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0071] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0072] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, and provides a charging pile fault detection method based on image recognition, comprising the following steps:

[0073] S1 uses visible light camera, infrared camera and ultraviolet camera to collect multimodal image data of charging piles and perform preprocessing;

[0074] Deploy visible light cameras, infrared cameras and ultraviolet cameras around the charging piles to collect visible light image data, infrared image data and ultraviolet image data of the charging piles;

[0075] The single-mode monitoring method in the existing technology has information blind spots. For example, it is impossible to detect appearance damage using only infrared data, and it is difficult to detect temperature anomalies using only visible light data.

[0076] The introduction of trimodal data is the core innovation of this invention that distinguishes it from the prior art, especially the use of ultraviolet data, which can capture special fault signals such as arc discharge, and combine visible light and infrared data to make up for the problem of information blind spots in single-modal detection;

[0077] Visible light image data captures the appearance of the charging pile, such as physical damage and stains;

[0078] Infrared image data reflects temperature anomalies, such as overheating and poor connection characteristics;

[0079] The UV image data captured the discharge signals, such as arc discharge and insulation fault details;

[0080] Preprocessing visible light image data, infrared image data and ultraviolet image data;

[0081] The timestamps of multimodal image data were synchronized through the PTP protocol, the spatial geometric calibration of multimodal image data was performed using the Zhang Zhengyou calibration method, the visible light image data was denoised using the edge-preserving denoising method based on Gaussian filtering, the infrared image data was denoised using a smoothing algorithm based on median filtering, the ultraviolet image data was denoised using a bilateral filtering method to enhance the details of the discharge signal, and the multimodal image data was normalized;

[0082] The time consistency of multimodal data is ensured through timestamp synchronization, and spatial and signal interference is eliminated through geometric calibration and denoising processing to make the features clearer;

[0083] Through preprocessing, the temporal and spatial consistency of multimodal data is ensured, providing high-quality input for subsequent feature extraction;

[0084] The preprocessed multimodal image data is integrated into the multimodal image dataset x, and the expression is:

[0085] x={x1,x2,x3};

[0086] Wherein, x1 represents the preprocessed visible light image data, x2 represents the preprocessed infrared image data, and x3 represents the preprocessed ultraviolet image data;

[0087] x represents the overall multimodal image dataset after preprocessing, which is the collection of all modal data. The rationality lies in its unified expression, which facilitates the standardization of subsequent calculations and model inputs.

[0088] x1, x2, and x3 represent the preprocessing results of visible light, infrared, and ultraviolet image data, respectively. The specific sources of different modal data are clarified, and each modal data has its unique role in fault detection, such as visible light detection of appearance damage, infrared detection of temperature anomalies, and ultraviolet capture of discharge signals. The rationality lies in distinguishing modal characteristics for subsequent feature extraction;

[0089] Multimodal data collection combines the characteristics of different modes, using visible light to capture appearance anomalies, infrared to detect temperature anomalies, and ultraviolet to capture electrical discharges, thus achieving a comprehensive perception of the operating status of the charging pile and avoiding the problem of insufficient information in single-modal data. Preprocessing steps, such as denoising and normalization, improve data quality and reduce noise interference, enabling subsequent deep learning models to extract effective features more efficiently.

[0090] Provide high-quality multimodal data input for deep learning algorithms, ensuring that subsequent feature extraction and fusion steps can be analyzed based on accurate and reliable data;

[0091] Ultimately, through the collection and preprocessing of multimodal data, we achieved all-round perception of the operating status of the charging pile, significantly improving the comprehensiveness and accuracy of fault detection.

[0092] S2 uses deep learning to extract features from the preprocessed multimodal image dataset and obtains a comprehensive feature vector through feature fusion algorithm;

[0093] The ResNet-50 network is used to extract the visible light image data x1 to obtain the visible light feature vector f1;

[0094] The EfficientNet-B0 network is used to extract the infrared image data x2 and obtain the infrared feature vector f2;

[0095] The DenseNet-121 network is used to extract the ultraviolet image data x3 to obtain the ultraviolet feature vector f3;

[0096] The features extracted from different modal data are modality-specific and can more accurately reflect the key information of each modality, such as infrared features reflecting temperature changes and ultraviolet features reflecting electrical anomalies;

[0097] Separate extraction of feature vectors helps to use the optimal deep learning model for different modal data and improve the accuracy and efficiency of single-modal feature extraction;

[0098] The ResNet-50 network extracts high-level semantic features from visible light image data, such as abnormal appearance of charging piles;

[0099] The EfficientNet-B0 network extracts temperature distribution features from infrared image data and identifies overheated areas;

[0100] The DenseNet-121 network extracts discharge signal features from UV image data and captures electrical fault information;

[0101] The weight generation function based on the attention mechanism is used to obtain the weight α of each feature vector i ;

[0102] The introduction of the attention mechanism is a key innovation in feature fusion, which can dynamically evaluate the importance of features of different modalities, avoid the dilution of key information by mean fusion, and improve the discriminative ability of the comprehensive feature vector F;

[0103] Assign weights α through the attention mechanism i , highlighting key modal features and the most important data sources in the current situation, and reducing the impact of redundant information;

[0104] Dynamically generate weight α i , highlighting the most important modal features in the current situation, such as increasing the weight of infrared features in high temperature environments and improving the fault discrimination ability of the comprehensive feature vector;

[0105] The three feature vectors are weighted fused to obtain the fused comprehensive feature vector F, which is expressed as:

[0106]

[0107] Among them, i represents the index variable of the feature vector, f i represents the i-th eigenvector, α i represents the weight of the i-th eigenvector;

[0108] In the existing technology, multimodal data is usually simply concatenated or averaged, ignoring the importance differences of different modal features, resulting in the dilution of key information;

[0109] F is the input of the subsequent fault detection model, and its quality directly determines the accuracy of the detection results. The design of the fusion process reflects the in-depth use of multimodal data in the invention;

[0110] The features of multimodal data are deeply integrated to obtain a comprehensive feature vector F that can fully reflect the status of the charging pile, providing high-dimensional, information-intensive input for the fault detection model;

[0111] Finally, through multimodal feature extraction and fusion, the multi-dimensional feature characterization of charging pile failures was achieved, which improved the effectiveness of feature input and the robustness of the detection model.

[0112] S3 builds a fault detection model based on the comprehensive feature vector, inputs the comprehensive feature vector, and obtains the detection result;

[0113] The fault detection model is constructed using a deep neural network DNN. The comprehensive feature vector is input and the expression is:

[0114]

[0115] Among them, s1 represents the output of the first layer of neural network, represents the Swish activation function, W1 represents the weight matrix of the first layer of the neural network, and b1 represents the bias vector of the first layer of the neural network;

[0116] The output h1 of the first layer of the neural network is input into the second layer to continue the nonlinear mapping.

[0117] 5. The charging pile fault detection method based on image recognition according to claim 4 is characterized in that: the output h1 of the first layer of neural network is input into the second layer to continue nonlinear mapping, and the expression is:

[0118]

[0119] Among them, s2 represents the output of the second layer neural network, W2 represents the weight matrix of the second layer neural network, and b2 represents the bias vector of the second layer neural network;

[0120] Through the Swish activation function Realize nonlinear transformation of features, capture complex feature relationships, and extract high-order semantic features layer by layer, so that the model can accurately distinguish different fault categories;

[0121] The final output layer expression is:

[0122] p = Softmax(W o ·s2+b o );

[0123] Among them, Softmax represents the probability distribution generating function, W o represents the weight matrix of the output layer, b o represents the bias vector of the output layer, and p represents the output of the fault detection model;

[0124] Traditional fault detection methods rely on manual feature extraction and rule matching, which makes it difficult to deal with complex and diverse charging pile faults, and the detection accuracy is limited by the design capabilities of manual rules;

[0125] The deep neural network model DNN captures complex feature relationships through multi-layer nonlinear mapping, such as the Swish activation function, and can automatically learn high-order features of fault modes, effectively improving the accuracy and generalization ability of fault detection. At the same time, the fault category probability distribution p generated by the Softmax output layer quantifies the possibility of fault occurrence, providing a reliable foundation for subsequent time series modeling;

[0126] Through the Softmax output probability distribution p, accurate classification of multiple fault categories such as overheating and arc discharge can be achieved;

[0127] The final output detection result p={p1,p2,…,p k}, is the predicted probability of each fault category, where k represents the number of fault categories;

[0128] The output in the form of probability distribution facilitates the subsequent risk prediction model to use historical probability data to construct time series, reflecting the seamless connection between detection and prediction;

[0129] Convert the comprehensive feature vector into specific fault category prediction results to provide basic data for subsequent fault trend analysis and risk assessment;

[0130] Ultimately, by building a fault detection model, we achieved efficient identification and classification of charging pile faults, significantly improving the accuracy and real-time performance of fault detection.

[0131] S4 builds a fault prediction model based on the detection results using the long short-term memory network LSTM to obtain a comprehensive risk index;

[0132] Existing technologies usually perform detection based on static data at a single time point and lack the ability to predict fault evolution trends;

[0133] The fault detection result at each time point is For any of them, it represents the probability of the jth type of failure at time point t, and j represents the index variable of the number of failure categories;

[0134] Construct a historical time window T to form a time series matrix V = [p 1 ,p 2 ,…,p T ], where V is a T×j matrix, representing the change in the failure probability of j categories at the past T time points;

[0135] Construct a historical time window T, extract the fault probability distribution at multiple time points, form a time series matrix V, and capture the dynamic characteristics of faults changing over time;

[0136] Capturing failure trends through time series matrices solves the problem of existing technologies being unable to dynamically predict risks.

[0137] Input the time series matrix V into the long short-term memory network LSTM model to capture the long-term dependency of the time series through the forget gate, input gate and output gate;

[0138] The long short-term memory network (LSTM) can effectively retain key historical information and ignore irrelevant noise through the design of forget gates, input gates, and output gates, thereby capturing the long-term evolution of faults.

[0139] The forget gate determines the historical information that needs to be discarded at the current moment. The time series matrix V is input into the forget gate to obtain the output vector g of the forget gate. t , the expression is:

[0140] g t =σ(W g·[h t―1 ,V t ]+b g );

[0141] Among them, g t represents the output of the forget gate, σ represents the Sigmoid activation function, W g represents the weight matrix of the forget gate, h t―1 represents the hidden state at the previous moment, P t represents the time series characteristics of the current input, b g Represents the bias vector of the forget gate;

[0142] The input gate determines the new information that needs to be added at the current moment. The time series matrix V is input into the input gate to obtain the output vector q of the input gate. t , the expression is:

[0143] q t =σ(W q ·[h t―1 ,V t ]+b q );

[0144] Among them, q t represents the output of the input gate, W q represents the weight matrix of the input gate, b q represents the bias vector of the input gate;

[0145] At the same time, based on the time series matrix V, the hyperbolic tangent function is used to calculate the candidate memory unit state The expression is:

[0146]

[0147] Among them, tanh represents the hyperbolic tangent function, W C The weight matrix representing the candidate memory cell state, b C A bias vector representing the state of the candidate memory cell;

[0148] Use the output vector q of the input gate t Multiply by the memory cell before the update and add the output vector q using the input gate t Multiply by the candidate memory cell state Get the updated memory unit C t , the expression is:

[0149]

[0150] Among them, C t represents the updated memory unit, C t―1 represents the memory unit before updating;

[0151] The output gate determines the output of the hidden state at the current moment, and the expression is:

[0152] l t =σ(W l ·[h t―1 ,V t ]+b l );

[0153] Among them, l t represents the output vector of the output gate, W l represents the weight matrix of the output gate, b l represents the bias vector of the output gate;

[0154] According to l t Update hidden state h t , the expression is:

[0155] h t = l t tanh(C t );

[0156] The long short-term memory network LSTM model uses memory unit C t and the hidden state h t Preserve critical historical information and ignore irrelevant short-term fluctuations, thereby more accurately predicting future failure probabilities;

[0157] Through time series modeling and long-term dependency learning, possible failures in the future are predicted, providing a reliable basis for risk assessment;

[0158] Finally, we built a fault prediction model through the long short-term memory network (LSTM) model, realized the trend prediction of charging pile fault development, and improved the foresight and scientific nature of fault warning.

[0159] Through the hidden state h t , predict the failure probability distribution at future time points, the expression is:

[0160] a j (t) = σ(W a ·h t +b a );

[0161] Among them, a j (t) represents the probability of the jth fault occurring at time point t, W a represents the weight matrix of the output layer, b a Represents the bias vector of the output layer;

[0162] According to the probability a of the jth fault occurring at time point t j(t), calculate the comprehensive risk index, the expression is:

[0163]

[0164] Among them, R t represents the comprehensive risk index at time point t, w j represents the risk weight of the jth type of failure;

[0165] The long short-term memory network model LSTM captures the long-term dependency of time series through forget gates, input gates, and output gates, and can explore the potential laws of fault changes over time, solving the problem that traditional methods cannot handle dynamic changes in faults.

[0166] Existing fault detection technologies are mostly static analysis, which cannot predict future fault trends, resulting in delayed fault management;

[0167] Comprehensive risk index R t Combined with the historical probability distribution of all fault categories, the overall risk level is quantified, providing a scientific basis for subsequent classification processing;

[0168] By predicting the probability of failure at a future time point, the risk can be quantified in advance, providing a scientific basis for fault warning and graded processing. The risk index quantifies the contribution of different fault categories to the overall risk, and combines the threshold to achieve graded response.

[0169] Comprehensively consider the risk weights of different failure types w j , the impact of multiple failures is quantified as a comprehensive risk index R t , which facilitates intuitive assessment of the overall risk level.

[0170] S5 sets thresholds and takes corresponding measures based on the comprehensive risk index;

[0171] Set the thresholds θ2 and θ2, when R t When ≤θ1, it means there is no risk and no measures are needed;

[0172] When θ2 <R t When <θ2, it indicates a small risk and the sampling and testing frequency should be increased;

[0173] When θ2≤R t When it is detected, it indicates a high risk, an alarm is sent to the user, and a professional maintenance team is contacted for on-site inspection;

[0174] In existing technologies, fault responses are mostly "one-size-fits-all" and lack a hierarchical processing mechanism, which can easily lead to resource waste or delayed response;

[0175] Through the hierarchical response mechanism, unnecessary intervention is avoided when the risk is low, and alarms and maintenance are triggered in time when the risk is high, which optimizes the utilization efficiency of detection resources and improves the initiative and intelligence level of fault management;

[0176] The calculation of comprehensive risk index and classification of response measures significantly improve the refinement and intelligence level of fault management;

[0177] Ultimately, through risk quantification and graded response, we achieved early warning and scientific decision-making for charging pile failures, reducing the impact of failures on users and equipment.

[0178] This embodiment also provides a computer device, which is suitable for the case of a charging pile fault detection method based on image recognition, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the charging pile fault detection method based on image recognition proposed in the above embodiment.

[0179] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0180] The present embodiment also provides a storage medium on which a computer program is stored. When the program is executed by the processor, the charging pile fault detection method based on image recognition as proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, disk or optical disk.

[0181] In summary, the present invention forms a complete charging pile fault detection and prediction system through multimodal data collection, deep learning feature extraction, multi-layer neural network detection, LSTM time series prediction and comprehensive risk assessment. Each step is closely linked to each other, which not only ensures the accuracy of detection, but also realizes the effective prediction of fault trends, providing technical guarantee for the safe and stable operation of charging piles.

[0182] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A charging pile fault detection method based on image recognition, characterized in that: include, Use visible light cameras, infrared cameras, and ultraviolet cameras to collect multimodal image data of charging piles and perform preprocessing; Based on the preprocessed multimodal image dataset, deep learning is used to extract features from the multimodal image dataset, and a comprehensive feature vector is obtained by fusion through a feature fusion algorithm; Construct a fault detection model based on the comprehensive feature vector, input the comprehensive feature vector, and obtain the detection result; Based on the detection results, a fault prediction model is constructed using the long short-term memory network LSTM to obtain a comprehensive risk index; Set thresholds and take corresponding measures based on the comprehensive risk index.

2. The charging pile fault detection method based on image recognition according to claim 1, characterized in that: The method of collecting multimodal image data of the charging pile using a visible light camera, an infrared camera, and an ultraviolet camera and performing preprocessing comprises the following specific steps: Deploy visible light cameras, infrared cameras and ultraviolet cameras around the charging piles to collect visible light image data, infrared image data and ultraviolet image data of the charging piles; Preprocessing visible light image data, infrared image data and ultraviolet image data; The preprocessed multimodal image data is integrated into the multimodal image dataset x, and the expression is: x={x1,x2,x3}; Among them, x1 represents the preprocessed visible light image data, x2 represents the preprocessed infrared image data, and x3 represents the preprocessed ultraviolet image data.

3. The charging pile fault detection method based on image recognition as claimed in claim 2, characterized in that: Based on the preprocessed multimodal image dataset, deep learning is used to extract features from the multimodal image dataset, and a comprehensive feature vector is obtained by fusing the features through a feature fusion algorithm. The specific steps are as follows: The ResNet-50 network is used to extract the visible light image data x1 to obtain the visible light feature vector f1; The EfficientNet-B0 network is used to extract the infrared image data x2 and obtain the infrared feature vector f2; The DenseNet-121 network is used to extract the ultraviolet image data x3 to obtain the ultraviolet feature vector f3; The weight generation function based on the attention mechanism is used to obtain the weight α of each feature vector i ; The three feature vectors are weighted fused to obtain the fused comprehensive feature vector F, which is expressed as: Among them, i represents the index variable of the feature vector, f i represents the i-th eigenvector, α i represents the weight of the i-th eigenvector.

4. The charging pile fault detection method based on image recognition as claimed in claim 3, characterized in that: The fault detection model is constructed based on the comprehensive feature vector, the comprehensive feature vector is input, and the detection result is obtained. The specific steps are as follows: The fault detection model is constructed using a deep neural network DNN. The comprehensive feature vector is input and the expression is: Among them, s1 represents the output of the first layer of neural network, represents the Swish activation function, W1 represents the weight matrix of the first layer of the neural network, and b1 represents the bias vector of the first layer of the neural network; The output h1 of the first layer of the neural network is input into the second layer to continue the nonlinear mapping.

5. The charging pile fault detection method based on image recognition as claimed in claim 4, characterized in that: The output h1 of the first layer of neural network is input into the second layer to continue nonlinear mapping, and the expression is: Among them, s2 represents the output of the second layer neural network, W2 represents the weight matrix of the second layer neural network, and b2 represents the bias vector of the second layer neural network; The final output layer expression is: <h2 style=";text-align:left;direction:ltr">p = Softmax(W<h2 style=";text-align:left;direction:ltr"> o <h2 style=";text-align:left;direction:ltr"> s2+b<h2 style=";text-align:left;direction:ltr"> o <h2 style=";text-align:left;direction:ltr"> ); Among them, Softmax represents the probability distribution generating function, W o represents the weight matrix of the output layer, b o represents the bias vector of the output layer, and p represents the output of the fault detection model; The final output detection result p={p1,p2,…,p k }, is the predicted probability of each fault category, where k represents the number of fault categories.

6. The charging pile fault detection method based on image recognition according to claim 5, characterized in that: Based on the detection results, a fault prediction model is constructed using a long short-term memory network LSTM to obtain a comprehensive risk index. The specific steps are as follows: The fault detection result at each time point is For any of them, it represents the probability of the jth type of failure at time point t, and j represents the index variable of the number of failure categories; Construct a historical time window T to form a time series matrix V = [p 1 ,p 2 ,…,p T ], where V is a T×j matrix, representing the change in the failure probability of j categories at the past T time points; Input the time series matrix V into the LSTM model to capture the long-term dependencies of the time series through the forget gate, input gate, and output gate; The forget gate determines the historical information that needs to be discarded at the current moment. The time series matrix V is input into the forget gate to obtain the output vector g of the forget gate. t ; The input gate determines the new information that needs to be added at the current moment. The time series matrix V is input into the input gate to obtain the output vector q of the input gate. t ; At the same time, based on the time series matrix V, the hyperbolic tangent function is used to calculate the candidate memory unit state Use the output vector q of the input gate t Multiply by the memory cell before the update and add the output vector q using the input gate t Multiply by the candidate memory cell state Get the updated memory unit C t ; The output gate determines the output of the hidden state at the current moment, and the expression is: l t =σ(W1·[h t-1 ,V t ]+b1); Among them, l t represents the output vector of the output gate, W1 represents the weight matrix of the output gate, and b1 represents the bias vector of the output gate; According to l t Update hidden state h t .

7. The charging pile fault detection method based on image recognition according to claim 6, characterized in that: According to the t Update hidden state h t , the expression is: h t =l t ·tanh(C t ); Through the hidden state h t , predict the failure probability distribution at future time points, the expression is: a j (t)=σ(W a ·h t +b a ); Among them, a j (t) represents the probability of the jth fault occurring at time point t, W a represents the weight matrix of the output layer, b a Represents the bias vector of the output layer; According to the probability a of the jth fault occurring at time point t j (t), calculate the comprehensive risk index, the expression is: Among them, R t represents the comprehensive risk index at time point t, w j represents the risk weight of the jth type of failure.

8. The charging pile fault detection method based on image recognition according to claim 7, characterized in that: The threshold is set and corresponding measures are taken according to the comprehensive risk index. The specific steps are as follows: Set the thresholds θ2 and θ2, when R t When ≤θ1, it means there is no risk and no measures are needed; When θ2 <R t When <θ2, it means there is a small risk, and the sampling and detection frequency should be increased; When θ2≤R t When it is detected, it indicates a high risk, an alarm is sent to the user, and a professional maintenance team is contacted for on-site inspection.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the charging pile fault detection method based on image recognition according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the charging pile fault detection method based on image recognition according to any one of claims 1 to 7 are implemented.

Images