A Fault Diagnosis Method for Charging Piles Based on Deep Q-Network

Abstract

The present invention provides a method for diagnosing charging pile faults based on a deep Q-network, comprising the following steps: By adopting a B / S architecture, a charging pile fault diagnosis system is built to achieve real-time data transmission; A charging pile fault diagnosis data model is constructed; The real-time charging parameter differences of the vehicle and the fault diagnosis data are preprocessed to obtain the corresponding data set {N K ,P K}; A loss function is set, and the data set {N K ,P K} is used to update the parameters of the deep Q-network by the stochastic gradient descent method; The learning formula of the deep Q-network is determined to obtain the value of the damage degree of the charging pile, and then the corresponding diagnostic measures are given according to the damage degree value, and they are displayed on the WEB interface. This method is based on Internet technology, builds a charging pile fault diagnosis platform using a B / S architecture, uploads vehicle charging parameters in real time, and uses a deep Q-network to fit the data to form a more accurate diagnostic report to remind charging personnel to take relevant measures.

Description

Technical Field:

[0001] The present invention relates to the technical field of electric vehicles, and particularly to a method for diagnosing faults in charging piles. Background Art:

[0002] With the rapid development of electric vehicle technology, electric vehicles have become an indispensable part of the automotive market. Charging piles have also developed synchronously with the development of electric vehicles, and there is a mutual dependence between the two. However, in the actual application process of charging piles, generally, the driver operates by himself for charging, and faults may occur due to reasons such as poor line contact, overload, and human operation errors. Therefore, there is an urgent need for a diagnostic system that can evaluate the fault condition of the charging pile, cut off the power in time for maintenance, and ensure the safety of people and vehicles.

[0003] As a research hotspot in the field of machine learning, the basic idea of deep learning is to combine the use of multi-layer network structures and non-linear transformations to extract features with lower dimensions and high distinguishability from high-dimensional original features. Therefore, the deep learning method ignores the control process and focuses more on the cognition and expression of things. With the rapid development of artificial intelligence, more and more practical problems need to use deep learning to extract the features of large-scale input data, and based on these features, self-motivated learning is carried out to optimize the strategies for solving problems.

[0004] Reinforcement learning emphasizes a self-learning process from environmental states to action mappings. The intelligent agent interacts with the environment at each moment and selects actions. The environment responds to this action and reaches a new state, and then evaluates the goodness or badness of each state or state-action through a value function, and finally determines the optimal strategy to reach the target state.

[0005] The deep Q-network combines the neural network in deep learning and the Q-learning algorithm for solving the optimal action value function in traditional reinforcement learning. The deep Q-network has the ability to perceive complex inputs and solve optimal strategies at the same time, has a low dependence on specific mathematical models, and is good at learning from data, providing an effective solution for the automatic fault diagnosis of charging piles. It has played a promoting role in the research of optimizing the average user charging time and charging behavior, etc., but has not yet involved the fault diagnosis and analysis of charging piles using technologies such as deep Q-networks. Currently, the existing technology uses deep convolutional neural networks for data learning. Since deep convolutional neural networks can only train models in a limited data set, the accuracy of charging pile fault diagnosis needs to be improved. Therefore, how to use the deep Q-network to realize the fault diagnosis system analysis of charging piles has become an important problem that needs to be solved urgently in this field.

[0006] Technical Content

[0007] The present invention provides a method for diagnosing charging pile faults based on a deep Q-network. This method is based on Internet technology and uses a B / S architecture to build a charging pile fault diagnosis platform, which uploads vehicle charging parameters in real time, uses the deep Q-network to fit the data, forms a more accurate diagnosis report, and reminds charging personnel to take relevant measures.

[0008] The technical solution adopted by the present invention to solve its technical problems is as follows:

[0009] A method for diagnosing charging pile faults based on a deep Q-network, including the steps of: building a charging pile fault diagnosis system by adopting a B / S architecture to realize real-time data transmission; constructing a charging pile fault diagnosis data model; preprocessing the difference between vehicle real-time charging parameters and fault diagnosis data to obtain a corresponding data set {N K , P K}; setting a loss function, and using the data set {N K , P K}, updating the parameters of the deep Q-network by the stochastic gradient descent method; determining the learning formula of the deep Q-network to obtain the value of the degree of damage of the charging pile, and further giving the corresponding diagnostic measures corresponding to the value of the degree of damage, and displaying them on the WEB interface.

[0010] Further, the B / S architecture includes a WEB interface, a WEB server, and a database.

[0011] Further, the WEB interface is developed using the Vue and / or MVC framework to simplify the input / output function and perform data visualization processing; the WEB server is used to execute the algorithm of the deep Q-network and information transmission; the database is MySQL; the data layer uses the abstract factory design pattern for hierarchical management of data and services.

[0012] Further, a charging pile fault diagnosis data model is constructed for vehicle real-time charging parameters and standard charging parameters; the charging pile fault diagnosis data model includes a state space S, an action space A, and a reward function R.

[0013] Further, the preprocessing of the difference between vehicle real-time charging parameters and fault diagnosis data to obtain a corresponding data set {N K , P K} includes the specific steps of: cleaning abnormal diagnosis data, and retaining correct fault diagnosis data according to the difference between vehicle real-time charging parameters; performing corresponding fault diagnosis scheme labels on the charging pile fault diagnosis data to obtain a corresponding data set {N K , P K}.

[0014] Further, the setting of the loss function, using the data set {N K , P K}, updating the parameters of the deep Q-network by the stochastic gradient descent method includes the following specific steps: initializing the deep Q-network to establish a Q(N K ,P K ; θ) framework; inputting the data set {N K ,P K} into Q(N K ,P K ; θ), learning the corresponding reward value through this network, and setting a loss function L(θ) related to the reward value; updating the Q-neural network parameter θ of the loss function L(θ) by the stochastic gradient descent method; continuously iterating the Q-neural network parameter θ until the loss function value is less than 0.001.

[0015] Furthermore, the formula of the loss function L(θ) is:

[0016] L i (θ i ) = E s,a,r [(E s' [r + γmaxQ(N k ,P k ,θ)|s,a] - Q(N k ,P k ,θ)) 2 )

[0017] Furthermore, the learning formula of the deep Q-network is:

[0018] Q(N k ,P k ,θ) = E s' [r + γmaxQ(N k ,P k ,θ)|s,a]

[0019] Furthermore, the state space S selects the difference of all real-time charging parameters of the vehicle as the state feature; the action space A is all fault diagnoses for the charging pile; the reward function R is the reward obtained after various fault diagnoses are correct.

[0020] Furthermore, the difference of the vehicle's real-time charging parameters includes the normalized differences of all real-time charging current values, all real-time charging voltage values, and all real-time charging temperature values of the vehicle from the respective standard times.

[0021] The beneficial effects of the present invention are:

[0022] First, based on the B / S framework, the real-time transmission and visualization processing of vehicle charging parameters are realized, and users can log in to the system on a computer or mobile phone without installing client software to view the historical and real-time vehicle charging parameters on the charging pile, realizing the refined management of charging pile charging.

[0023] Second, compared with fault diagnosis using deep convolutional neural networks and other traditional fault diagnosis methods, the deep Q-network can not only perform real-time fault diagnosis, but also mine more data for deep Q-learning, making the Q-neural network model more and more proficient in fault diagnosis, and thus the accuracy of fault diagnosis is getting higher and higher.

[0024] Third, the deep convolutional neural network can only take corresponding diagnoses for limited faults, while the fault diagnosis based on the deep Q-network can learn and master more fault diagnosis schemes in an environment of unknown faults. Brief Description of the Drawings:

[0025] The present invention will be further described below in conjunction with the drawings and embodiments.

[0026] Figure 1 : Schematic diagram of the B / S three-layer structure.

[0027] Figure 2 : Flowchart of the deep Q-network algorithm. Specific Embodiments:

[0028] The present invention will be further described in detail below in conjunction with the drawings and specific embodiments. The embodiments cannot be elaborated one by one here, but the embodiments of the present invention are not limited to the following embodiments accordingly.

[0029] A method for diagnosing charging pile faults based on a deep Q-network, by adopting a B / S architecture, building a charging pile fault diagnosis system, and realizing real-time data transmission.

[0030] The B / S architecture includes a WEB interface, a WEB server, and a database; the WEB interface is developed using the Vue and / or MVC framework to simplify the input / output function and perform data visualization processing; the WEB server is used to execute the algorithm of the deep Q-network and information transmission; the database is MySQL; the data layer uses the abstract factory design pattern for hierarchical management of data and services.

[0031] The B / S architecture is a network structure model. The B / S architecture includes a WEB interface, a WEB server, and a database. The WEB interface is the presentation layer, which refers to the software interface that can be seen under a browser and is used to complete the interaction between the user and the background and display the input / output results of data; the WEB server is the logic layer, which completes the application logic function of the client and is used to execute the algorithm of the deep Q-network; the database runs SQL or stored procedures and is developed using MySQL. The data layer uses the abstract factory design pattern for hierarchical management of data and services.

[0032] In the B / S architecture mode, the core part of the system function implementation is centralized on the WEB server, simplifying the development, maintenance, and use of the system. The B / S architecture does not require the installation of specialized software and only requires a WEB interface. The WEB interface interacts with the database through the WEB server and can work on different platforms.

[0033] MVC is a design framework based on the WEB interface. The WEB interface includes a model, a view, and a controller. MVC compulsorily separates the input, processing, and output of an application. The model is the data presented by the view. The view is the view of the model and is the interface that users see and interact with. The controller is the logic for handling user interactions, accepting user input, and calling the model and view to meet user requirements.

[0034] Vue is an open-source progressive framework for building WEB interfaces. MySQL is a relational database management system.

[0035] Build a charging pile fault diagnosis data model. For the vehicle's real-time charging parameters and standard charging parameters, build a charging pile fault diagnosis data model. The charging pile fault diagnosis data model includes a state space S, an action space A, and a reward function R.

[0036] S is the set of all environmental states. A is the set of all executable actions. R is the set of reward returns r obtained when a specific state s in the state space S set executes a specific action a in the action space A set. The state space S selects the difference of all the vehicle's real-time charging parameters as the state feature. The difference of the vehicle's real-time charging parameters includes the normalized differences of all the vehicle's real-time charging current values, all the vehicle's real-time charging voltage values, and all the vehicle's real-time charging temperature values from the respective standards. The action space A is for all the fault diagnoses of the charging pile. The reward function R is the reward obtained after various correct fault diagnoses.

[0037] Preprocess the difference of the vehicle's real-time charging parameters and the fault diagnosis data. Clean the abnormal diagnosis data, and retain the correct fault diagnosis data according to the difference of the vehicle's real-time charging parameters. Add corresponding fault diagnosis scheme labels to the charging pile fault diagnosis data to obtain the corresponding data set {N K ,P K}.

[0038] The N K is the set of preprocessed states, that is, the set of the differences of the vehicle's real-time charging parameters after preprocessing. The P K is the set of preprocessed actions, that is, the set of fault diagnosis data after preprocessing.

[0039] Set the loss function and use the data set {N K ,PK}, update the parameters of the deep Q-network by the stochastic gradient descent method. Initialize the deep Q-network, and establish a Q(N K ,P K ; θ) network; input the dataset {N K ,P K} into Q(N K ,P K ; θ), learn the corresponding reward value through this network, and set the loss function L(θ) related to the reward value; the loss function L(θ) updates the Q-neural network parameter θ by the stochastic gradient descent method; continuously iterate the Q-neural network parameter θ until the loss function value is less than 0.001.

[0040] The deep Q-network is a combination of a neural network in deep learning and the Q-learning algorithm for solving the optimal action value function in reinforcement learning. The Q(N K ,P K ; θ) framework is a non-linear function approximation model. The θ is the Q-neural network parameter. The initialization of the deep Q-network is achieved by constructing a neural network architecture, that is, constructing a Q(N K ,P K ; θ) framework, initializing the θ tensor, and the initial value of θ is a tensor composed of random data.

[0041] The dataset {N K ,P K} is a dataset obtained after preprocessing the vehicle real-time charging parameter difference and fault diagnosis data. All states N K ,P K in the dataset {N K after selecting the action P K obtain the reward value corresponding to the reward function R of the charging pile fault diagnosis data model.

[0042] The formula of the loss function is as follows:

[0043] L i (θ i ) = E s,a,r [(E s' [r + γmaxQ(N k ,P k , θ)|s,a] - Q(N k ,P k , θ)) 2

[0044] E represents experience replay; i represents the number of iterations; θ is the Q-neural network parameter; r represents the current state reward value; γ represents the discount factor, and the γ = 0.99; γmaxQ(N K ,P K ​, θ) is the optimal solution of the next state formed after executing action a in the current state s; the E s’ [r + γmaxQ(N K , P K , θ)|s, a] is the true value of the current state; in the loss function formula, Q(N K , P K , θ) is the simulated approximation value of the Q neural network; the loss function is the mean square error between the true value of the current state and the simulated approximation value of the Q neural network; the parameters of the Q neural network are minimized by using the stochastic gradient descent method.

[0045] By continuously iterating the Q neural network parameters θ until the value of the loss function is less than 0.001. At this time, the E s’ [r + γmaxQ(N K , P K , θ)|s, a] of the true value of the current state is very close to the simulated approximation value of Q(N K , P K , θ) of the Q neural network, thus completing the training of the Q neural network parameters θ and finally determining the learning formula of the deep Q network.

[0046] After updating the Q neural network parameters in the deep Q network, finally determine the learning formula of the deep Q network, obtain the value of the damage degree of the charging pile, and then give the corresponding diagnostic measures for the damage degree value and display it on the WEB interface.

[0047] The learning formula of the deep Q network is as follows:

[0048] Q(N k , P k , θ) = E s' [r + γmaxQ(N k , P k , θ)|s, a]

[0049] According to the learning formula of the deep Q network, the value of the damage degree of the charging pile can be obtained.

[0050] The above embodiments of the present invention are merely examples for clearly illustrating the present invention, rather than limitations on the implementation manners of the present invention. For those of ordinary skill in the art in the technical field, other different forms of changes or modifications can be made on the basis of the above description. It is not necessary and impossible to enumerate all implementation manners here. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the claims of the present invention.

Claims

1. A method for diagnosing charging pile faults based on a deep Q-network, characterized in that: It includes the steps: S1: By adopting the B / S architecture, build a charging pile fault diagnosis system to achieve real-time data transmission; S2: Construct a charging pile fault diagnosis data model; S3: Preprocess the difference in real-time vehicle charging parameters and fault diagnosis data to obtain the corresponding dataset {N K ,P K}; S4: Set the loss function and use the dataset {N K , P K} to update the parameters of the deep Q-network by stochastic gradient descent; S5: Determine the learning formula of the deep Q-network, obtain the value of the damage degree of the charging pile, and then give the corresponding diagnostic measures for the damage degree value, and display it on the WEB interface; The specific steps of the said S3 include: S31: Clean abnormal diagnostic data, and retain correct fault diagnostic data according to the real-time charging parameter difference of the vehicle; S32: Perform corresponding fault diagnosis scheme tagging on the charging pile fault diagnosis data to obtain the corresponding dataset {N K , P K}; The specific steps of the said S4 include: S41: Initialize the deep Q-network to establish the Q(N K , P K ; θ) network; S42: Input the data set {N K , P K} into Q(N K , P K ; θ), learn the corresponding reward value through this network, and set the loss function L(θ) related to the reward value; S43: The loss function L(θ) updates the Q-neural network parameter θ by using the stochastic gradient descent method; S44: Continuously iterate the Q-neural network parameter θ until the loss function value is less than 0.

001.

2. A method for diagnosing charging pile faults based on a deep Q-network according to claim 1, characterized in that: The said B / S architecture includes a WEB interface, a WEB server, and a database.

3. A method for diagnosing charging pile faults based on a deep Q-network according to claim 2, characterized in that: The said WEB interface is developed using the Vue and / or MVC framework to simplify the input and output functions and perform data visualization processing; the said WEB server is used to execute the algorithm of the deep Q-network and information transmission; the database is MySQL; The data layer uses the abstract factory design pattern for hierarchical management of data and business.

4. A method for diagnosing charging pile faults based on a deep Q-network according to claim 1, characterized in that: For the real-time charging parameters of the vehicle and the standard charging parameters, construct a charging pile fault diagnosis data model; the said charging pile fault diagnosis data model includes a state space S, an action space A, and a reward function R.

5. A method for diagnosing charging pile faults based on a deep Q-network according to claim 4, characterized in that: The formula of the said loss function L(θ): L i (θ i ) = E s,a,r [(E s' [r + γmaxQ(N k , P k , θ)|s, a] - Q(N k , P k , θ)) 2 .

6. A method for diagnosing charging pile faults based on a deep Q-network according to claim 5, characterized in that: The deep Q-network learning formula: Q(N k ,P k ,θ) = E s' [r + γ max Q(N k ,P k ,θ)|s,a].

7. A method for diagnosing charging pile faults based on a deep Q-network according to claim 4, characterized in that: The said state space S selects the difference of all real-time charging parameters of the vehicle as the state feature; the said action space A is for all fault diagnoses of the charging pile; the said reward function R is the reward obtained after various fault diagnoses are correct.

8. A method for diagnosing charging pile faults based on a deep Q-network according to claim 7, characterized in that: The said difference of the vehicle's real-time charging parameters includes the normalized differences of all real-time charging current values, all real-time charging voltage values, and all real-time charging temperature values of the vehicle from the respective standards.

Images