An electric vehicle charging station operation and maintenance optimization strategy based on deep reinforcement learning

Abstract

This invention discloses an electric vehicle charging station operation and maintenance optimization strategy based on deep reinforcement learning. 1) Based on the risk index values ​​of the charging station and the health status values ​​of the charging piles within it, the priority charging station for operation and maintenance and the order of charging pile operation and maintenance for that station are determined. 2) A charging pile operation and maintenance duration optimization (DDPG) model is constructed, considering the uncertainty of the number and types of faulty charging piles and the arrival rate of charging electric vehicles, minimizing the time cost lost by electric vehicle users and the operation and maintenance resource cost, to make online decisions on the charging pile operation and maintenance duration. 3) After the operation and maintenance of the current charging station is completed, the DDPG model makes online decisions and proceeds to the next charging station for charging pile operation and maintenance. This strategy coordinates emergency repair and maintenance. The differentiated charging pile operation and maintenance duration obtained by the DDPG model can reasonably allocate operation and maintenance time and resources, overcoming the problems of coarse and untargeted solutions in traditional operation and maintenance. Furthermore, the online decision-making process boasts superior speed, accuracy, and generalization ability.

Description

Technical Field

[0002] This invention relates to the operation and maintenance technology of electric vehicle charging stations, and in particular to an operation and maintenance optimization strategy for electric vehicle charging stations based on deep reinforcement learning. Background Technology

[0003] Electric vehicles, as a low-carbon and environmentally friendly mode of transportation, have experienced rapid development in recent years. my country has formed the world's most complete industrial ecosystem for pure electric vehicles, ranking first globally for 11 consecutive years. Driven by both policy support and market demand, the construction of electric vehicle charging infrastructure has grown rapidly. By the end of December 2025, my country had 20.092 million electric vehicle charging facilities, forming the world's largest electric vehicle charging network, capable of supporting the charging needs of over 40 million new energy vehicles. However, charging piles at charging stations are typically located outdoors, and the main type of DC public charging piles has a complex structure, high output power, and fast charging speed, leading to a high failure rate. On the other hand, charging stations mostly serve electric vehicle users with high service demand in cities. When charging piles malfunction, it will have a significant impact on users and even normal traffic operations, and may even lead to fire accidents. Therefore, to ensure the efficient operation of large-scale charging infrastructure, developing efficient charging station operation and maintenance strategies has become an urgent practical problem to be solved.

[0004] The operation and maintenance (O&M) of charging stations mainly includes two parts: emergency repair of faulty charging piles and maintenance of non-faulty charging piles. Emergency repair refers to the work of O&M personnel to restore the normal operation of charging piles after a malfunction. Maintenance of non-faulty charging piles is routine maintenance, which refers to preventative checks and optimizations performed when charging piles are not malfunctioning, to ensure their normal operation and extend their service life. Currently, charging stations have carried out corresponding O&M work, but traditionally, emergency repair and maintenance are mostly separated, and maintenance plans usually allocate the same amount of time and resources to each charging pile. The traditional charging station O&M model has many drawbacks: the separate deployment of emergency repair and maintenance personnel does not coordinate the work; maintenance plans are often crude, reliant on experience, and lack specificity, resulting in low O&M efficiency and difficulty in meeting the needs of the rapidly growing number of charging stations.

[0005] Currently, with the continuous increase in the number of charging stations and the importance of their operation and maintenance, and the development of real-time equipment monitoring technology, as well as status prediction and lifespan management technology, charging station operation and maintenance strategies face new challenges: the uncertainty of the number of faulty charging piles, the uncertainty of the fault types of faulty charging piles, and the temporal characteristics of electric vehicle arrival rates bring complexity to charging station operation and maintenance strategies. Differentiated operation and maintenance strategies can be formulated based on the risk level of the charging station and the health status of the charging piles, and operation and maintenance time and resources can be rationally allocated to overcome the problems of traditional operation and maintenance solutions being too rough and lacking specificity. Deep reinforcement learning combines the powerful decision-making capabilities of reinforcement learning with the excellent perception capabilities of deep learning for complex high-dimensional data. Through continuous interaction with the environment, it automatically explores the optimal strategy, featuring model-free learning and flexible response to time-varying environments. In summary, this invention proposes to determine the charging stations with priority for maintenance and the maintenance order of the charging piles in the charging stations based on the risk index values ​​of the charging stations and the health status values ​​of the charging piles in the charging stations. Based on the maintenance order of the charging piles in the charging stations, the problem of optimizing the maintenance time of the charging piles in the charging stations is transformed into a deep reinforcement learning decision framework, and a deep reinforcement learning model for optimizing the maintenance time of the charging piles is constructed. Summary of the Invention

[0006] This invention aims to solve one of the technical problems existing in the prior art. To this end, this invention proposes an operation and maintenance optimization strategy for electric vehicle charging stations based on deep reinforcement learning, comprising the following steps:

[0007] 1) Based on the risk index values ​​of the charging station and the health status values ​​of the charging piles in the charging station, determine the charging station that should be prioritized for maintenance and the order of maintenance of the charging piles in that charging station.

[0008] 2) Construct a DDPG model to optimize the charging pile operation and maintenance time, taking into account the uncertainty of the number and types of faulty charging piles and the arrival rate of electric vehicles, the time cost of electric vehicle users' lost time and the goal of minimizing operation and maintenance resource costs, and make online decisions on the charging pile operation and maintenance time of priority charging stations.

[0009] 3) After the operation and maintenance of the charging station is completed, the next charging station to be prioritized for operation and maintenance and its charging pile operation and maintenance order are determined in real time among the remaining charging stations. The DDPG model makes online decisions on the operation and maintenance duration and proceeds to the next charging station for charging pile operation and maintenance.

[0010] Specifically, the process of determining priority charging stations and their charging pile maintenance order based on risk index values ​​and health status values ​​of charging piles within the charging station is as follows:

[0011] 1) The health status evaluation index values ​​of charging station equipment are obtained by adopting online data monitoring and evaluation by practitioners. The health status value of charging station equipment is obtained by adopting a comprehensive evaluation method. The charging station equipment includes transformer components and charging pile components.

[0012] 2) The risk index value of a charging station is defined as the product of the probability of a charging station failure event and its impact, where the impact is the lost time for electric vehicle users. Based on the health status values ​​of the components in the charging station, the real-time failure rate of the components is obtained. Then, a reliability model of the components is established. The Latin hypercube sampling method is used to sample the state of all components. The sampled states of all components are combined to obtain the sampled state of the charging station. The impact of failure on the sampled state of the charging station is analyzed, and the risk index value is calculated as follows:

[0013] (1)

[0014] Among them, I risk t represents the risk indicator value for charging stations. loss,n Let N be the user loss time corresponding to the state in the nth Latin hypercube sampling, where N is the number of Latin hypercube samplings.

[0015] User loss time is defined as the difference between the average time required for an electric vehicle user to arrive at a charging station and complete charging under normal operating conditions, when there is a faulty component at the charging station. It is expressed as:

[0016] (2)

[0017] Among them, t loss For the user's lost time, t f-av t uf-av These represent the average time required for an electric vehicle user to complete charging from arrival at the charging station, under conditions of faulty components and normal operation, respectively.

[0018] The process of an electric vehicle arriving at a charging station and completing its charge follows an M / M / S / FCFS queuing model. Let the arrival rate of electric vehicles be γ, expressed in vehicles per hour. -1 This can be obtained from statistical data. The arrival rate of electric vehicle users follows a Poisson distribution with parameter γ. Let the average charging service rate of a single charging station be α, in units of vehicles per hour. -1 If the charging service time follows a negative exponential distribution with parameter α, then the average time required for an electric vehicle user to complete charging from arriving at the charging station can be obtained, expressed as:

[0019] (3)

[0020] Among them, t av t represents the average time it takes for an electric vehicle user to complete charging from arriving at a charging station. wait-av t represents the average queuing time. char-av N represents the average charging service time. avaP represents the number of available charging piles within the charging station, ρ represents the service intensity, and P represents the service intensity. i For charging power, W batt For the average battery capacity of electric vehicle users, δ av This represents the average percentage of electricity demand.

[0021] 3) Based on the risk index values ​​of charging stations, the charging station with the lowest risk index value is identified as the priority charging station for operation and maintenance; then, the operation and maintenance order of the charging piles at the charging station is determined: emergency repair of faulty charging piles takes priority over maintenance of non-faulty charging piles; the emergency repair order of faulty charging piles is determined according to the fault type, which is divided into simple fault type, medium complex fault type, and high complex fault type, with simple fault type being repaired first and high complex fault type being repaired last; the maintenance order of non-faulty charging piles is determined by the order of their health status values, with the smaller the health status value, the earlier its maintenance order, and maintenance is not performed when the health status value is greater than the set health value of the charging pile.

[0022] The construction of the DDPG model for optimizing charging pile maintenance time, which considers the uncertainty of the number and types of faulty charging piles and the arrival rate of electric vehicles, as well as the objectives of minimizing the time cost lost by electric vehicle users and the cost of operation and maintenance resources, and makes online decisions on the priority of charging station maintenance time, is as follows:

[0023] Considering the uncertainty of the number of faulty charging piles at charging stations, the uncertainty of the fault type of the faulty charging piles, the uncertainty of the arrival rate of electric vehicles, and the goal of minimizing the time cost and operation and maintenance resource cost for electric vehicle users, the problem of optimizing the operation and maintenance time of charging piles at charging stations is transformed into a deep reinforcement learning decision framework based on the order of operation and maintenance of charging piles at charging stations. A deep reinforcement learning model for optimizing the operation and maintenance time of charging piles is constructed to decide the priority of the operation and maintenance time of charging piles at charging stations.

[0024] Design a deep reinforcement learning model for optimizing the operation and maintenance time of charging piles, including actions, states, rewards, and deep reinforcement learning algorithm elements. Actions include the emergency repair time of faulty charging piles and the maintenance time of non-faulty charging piles, represented as:

[0025] (4)

[0026] Where 'a' represents the action space. , These represent the emergency repair time for faulty charging pile i and the maintenance time for non-faulty charging pile j, respectively. fr J ufm These are the sets of numbers for faulty and non-faulty charging piles in the priority maintenance charging stations, respectively.

[0027] The state space consists of the status identifier of the charging pile, the fault type of the faulty charging pile, and the arrival rate of the charging electric vehicle, and is represented as:

[0028] (5)

[0029] Where s is the state space. The status identifier for charging pile k is divided into fault status and non-fault status. The value is 1, indicating a non-fault state. The value is 0, where K is the number of charging piles in the priority maintenance charging station. The fault types of charging pile i are categorized into simple fault types, medium-complexity fault types, and highly complex fault types, with corresponding... The values ​​are 0, 1, and 2 respectively. The electric vehicle arrival rate during operation and maintenance period t;

[0030] The rewards include target rewards and constraint rewards, both of which are negative. Target rewards include emergency repair target rewards and maintenance target rewards. Emergency repair target rewards are the sum of user time loss costs and emergency repair resource costs, while maintenance target rewards are the sum of maintenance loss value costs and maintenance resource costs. User time loss costs are converted into economic loss costs by converting user time lost due to charging pile operation and maintenance. Maintenance loss value costs are defined as the product of failure rate and user time loss costs. It is set that the failure rate of non-faulty charging piles is reduced by half for each maintenance period.

[0031] The reward for emergency repair targets is:

[0032] (6)

[0033] in, Rewards will be given for the repair targets of faulty charging pile i. , The user time loss cost and repair resource cost of faulty charging pile i are separated. i This refers to the time period during which the faulty charging station was repaired. For a time period l i The user's lost time due to emergency repair of the faulty charging pile i, in minutes per vehicle, is calculated using equations (2) and (3). For a time period l i The electric vehicle arrival rate at the charging station, where c1 is the economic cost coefficient for lost time per user, expressed in yuan per minute. For a time period l i The unit time repair resource cost of the faulty charging pile i;

[0034] The maintenance target reward is:

[0035] (7)

[0036] in, A maintenance target reward is given for non-faulty charging pile j. , The maintenance loss value cost and maintenance resource cost of non-faulty charging pile j are respectively... j The time period during which non-faulty charging pile j is maintained, t e This represents the duration of a maintenance period for a non-faulty charging station, where FL(.) is used for rounding down, and λ... j The failure rate of non-faulty charging piles before maintenance. For a time period l j The user loss time caused by maintenance of non-faulty charging pile j, in minutes per vehicle, is calculated using equations (2) and (3). For a time period l j Electric vehicle arrival rate at charging stations For a time period l j The unit time maintenance resource cost of non-faulty charging pile j;

[0037] The constraints include the repair time constraint for the faulty charging pile i, the maintenance time constraint for the non-faulty charging pile j, and the operation and maintenance sequence constraint. The reward for the repair time constraint of the faulty charging pile i is:

[0038] (8)

[0039] in, Incentives are awarded based on the time constraints of emergency repairs for faulty charging pile i. , , The faulty charging pile i is set as the repair time value under simple fault type, medium complex fault type and high complex fault type respectively, and k1 is set as the penalty value for violating the repair time constraint.

[0040] The maintenance time constraint reward for non-faulty charging pile j is:

[0041] (9)

[0042] in, For non-faulty charging pile j, the maintenance time constraint reward is t. min t max These are the minimum and maximum maintenance times for a single charging pile, respectively, and k2 is the penalty value for violating the maintenance time constraint.

[0043] The reward for operation and maintenance time-sequence constraints is:

[0044] (10)

[0045] in, For the time-series constraint reward of the operation and maintenance of charging pile k, l k l k+1 Let t represent the time periods during the maintenance of charging piles k and (k+1), respectively. k K is the maintenance duration of charging pile k, and k3 is the penalty value of the set maintenance timing constraint.

[0046] In the deep reinforcement learning model for optimizing the operation and maintenance time of charging piles, the deep reinforcement learning algorithm adopts the DDPG algorithm to form the DDPG model for optimizing the operation and maintenance time of charging piles, and makes online decisions on the operation and maintenance time of charging piles for priority charging stations.

[0047] After the maintenance of the charging station is completed, the next charging station to be prioritized for maintenance and its charging pile maintenance order are determined in real time from the remaining charging stations. The DDPG model makes online decisions on the maintenance duration and proceeds to the next charging station for charging pile maintenance, as detailed below:

[0048] After the maintenance of the charging piles at the priority charging stations is completed, the next priority charging station and its charging pile maintenance order are determined in real time based on the risk index values ​​and health status values ​​of the charging piles in the remaining charging stations. The maintenance duration is then determined online by the DDPG model, and the maintenance of the charging piles at the next charging station is initiated. This process is repeated until all charging stations are maintained.

[0049] The embodiments of the present invention have at least the following beneficial technical effects:

[0050] 1) This invention proposes to determine the priority of charging stations for operation and maintenance based on the risk index value of the charging station and to determine the order of operation and maintenance of charging piles based on the health status value of the charging piles in the power station. While coordinating emergency repair and maintenance, different priority orders of operation and maintenance are carried out, which has good engineering application value.

[0051] 2) The deep reinforcement learning model for optimizing the operation and maintenance time of charging piles proposed in this invention can make online decisions on the priority of the operation and maintenance time of charging piles in charging stations. This can solve the problems caused by the uncertainty of the number of faulty charging piles and their fault types and the arrival rate of electric vehicles. The differentiated operation and maintenance time of charging piles can reasonably allocate operation and maintenance time and resources, so as to overcome the problems of the rough and untargeted solutions of traditional operation and maintenance. Moreover, the online decision-making has superior speed, accuracy and generalization ability. Attached Figure Description

[0052] Figure 1 This is a flowchart of an electric vehicle charging station operation and maintenance optimization strategy based on deep reinforcement learning, provided by an embodiment of the present invention. Detailed Implementation

[0053] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the following embodiments are by no means intended to limit the present invention.

[0054] An operation and maintenance optimization strategy for electric vehicle charging stations based on deep reinforcement learning, such as... Figure 1 As shown, it includes the following steps:

[0055] 1) Based on the risk index values ​​of the charging station and the health status values ​​of the charging piles in the charging station, determine the charging station that should be prioritized for maintenance and the order of maintenance of the charging piles in that charging station.

[0056] 2) Construct a DDPG model to optimize the charging pile operation and maintenance time, taking into account the uncertainty of the number and types of faulty charging piles and the arrival rate of electric vehicles, the time cost of electric vehicle users' lost time and the goal of minimizing operation and maintenance resource costs, and make online decisions on the charging pile operation and maintenance time of priority charging stations.

[0057] 3) After the operation and maintenance of the charging station is completed, the next charging station to be prioritized for operation and maintenance and its charging pile operation and maintenance order are determined in real time among the remaining charging stations. The DDPG model makes online decisions on the operation and maintenance duration and proceeds to the next charging station for charging pile operation and maintenance.

[0058] Specifically, the process of determining priority charging stations and their charging pile maintenance order based on risk index values ​​and health status values ​​of charging piles within the charging station is as follows:

[0059] 1) The health status evaluation index values ​​of charging station equipment are obtained by adopting online data monitoring and evaluation by practitioners. The health status value of charging station equipment is obtained by adopting a comprehensive evaluation method. The charging station equipment includes transformer components and charging pile components.

[0060] 2) The risk index value of a charging station is defined as the product of the probability of a charging station failure event and its impact, where the impact is the lost time for electric vehicle users. Based on the health status values ​​of the components in the charging station, the real-time failure rate of the components is obtained. Then, a reliability model of the components is established. The Latin hypercube sampling method is used to sample the state of all components. The sampled states of all components are combined to obtain the sampled state of the charging station. The impact of failure on the sampled state of the charging station is analyzed, and the risk index value is calculated as follows:

[0061] (1)

[0062] Among them, I risk t represents the risk indicator value for charging stations. loss,n Let N be the user loss time corresponding to the state in the nth Latin hypercube sampling, where N is the number of Latin hypercube samplings.

[0063] User loss time is defined as the difference between the average time required for an electric vehicle user to arrive at a charging station and complete charging under normal operating conditions, when there is a faulty component at the charging station. It is expressed as:

[0064] (2)

[0065] Among them, t loss For the user's lost time, t f-av t uf-av These represent the average time required for an electric vehicle user to complete charging from arrival at the charging station, under conditions of faulty components and normal operation, respectively.

[0066] The process of an electric vehicle arriving at a charging station and completing its charge follows an M / M / S / FCFS queuing model. Let the arrival rate of electric vehicles be γ, expressed in vehicles per hour. -1 This can be obtained from statistical data. The arrival rate of electric vehicle users follows a Poisson distribution with parameter γ. Let the average charging service rate of a single charging station be α, in units of vehicles per hour. -1 If the charging service time follows a negative exponential distribution with parameter α, then the average time required for an electric vehicle user to complete charging from arriving at the charging station can be obtained, expressed as:

[0067] (3)

[0068] Among them, t av t represents the average time it takes for an electric vehicle user to complete charging from arriving at a charging station. wait-av t represents the average queuing time. char-av N represents the average charging service time. ava P represents the number of available charging piles within the charging station, ρ represents the service intensity, and P represents the service intensity. i For charging power, W batt For the average battery capacity of electric vehicle users, δ av This represents the average percentage of electricity demand.

[0069] 3) Based on the risk index values ​​of charging stations, the charging station with the lowest risk index value is identified as the priority charging station for operation and maintenance; then, the operation and maintenance order of the charging piles at the charging station is determined: emergency repair of faulty charging piles takes priority over maintenance of non-faulty charging piles; the emergency repair order of faulty charging piles is determined according to the fault type, which is divided into simple fault type, medium complex fault type, and high complex fault type, with simple fault type being repaired first and high complex fault type being repaired last; the maintenance order of non-faulty charging piles is determined by the order of their health status values, with the smaller the health status value, the earlier its maintenance order, and maintenance is not performed when the health status value is greater than the set health value of the charging pile.

[0070] The construction of the DDPG model for optimizing charging pile maintenance time, which considers the uncertainty of the number and types of faulty charging piles and the arrival rate of electric vehicles, as well as the objectives of minimizing the time cost lost by electric vehicle users and the cost of operation and maintenance resources, and makes online decisions on the priority of charging station maintenance time, is as follows:

[0071] Considering the uncertainty of the number of faulty charging piles at charging stations, the uncertainty of the fault type of the faulty charging piles, the uncertainty of the arrival rate of electric vehicles, and the goal of minimizing the time cost and operation and maintenance resource cost for electric vehicle users, the problem of optimizing the operation and maintenance time of charging piles at charging stations is transformed into a deep reinforcement learning decision framework based on the order of operation and maintenance of charging piles at charging stations. A deep reinforcement learning model for optimizing the operation and maintenance time of charging piles is constructed to decide the priority of the operation and maintenance time of charging piles at charging stations.

[0072] Design a deep reinforcement learning model for optimizing the operation and maintenance time of charging piles, including actions, states, rewards, and deep reinforcement learning algorithm elements. Actions include the emergency repair time of faulty charging piles and the maintenance time of non-faulty charging piles, represented as:

[0073] (4)

[0074] Where 'a' represents the action space. , These represent the emergency repair time for faulty charging pile i and the maintenance time for non-faulty charging pile j, respectively. fr J ufm These are the sets of numbers for faulty and non-faulty charging piles in the priority maintenance charging stations, respectively.

[0075] The state space consists of the status identifier of the charging pile, the fault type of the faulty charging pile, and the arrival rate of the charging electric vehicle, and is represented as:

[0076] (5)

[0077] Where s is the state space. The status identifier for charging pile k is divided into fault status and non-fault status. The value is 1, indicating a non-fault state. The value is 0, where K is the number of charging piles in the priority maintenance charging station. The fault types of charging pile i are categorized into simple fault types, medium-complexity fault types, and highly complex fault types, with corresponding... The values ​​are 0, 1, and 2 respectively. The electric vehicle arrival rate during operation and maintenance period t;

[0078] The rewards include target rewards and constraint rewards, both of which are negative. Target rewards include emergency repair target rewards and maintenance target rewards. Emergency repair target rewards are the sum of user time loss costs and emergency repair resource costs, while maintenance target rewards are the sum of maintenance loss value costs and maintenance resource costs. User time loss costs are converted into economic loss costs by converting user time lost due to charging pile operation and maintenance. Maintenance loss value costs are defined as the product of failure rate and user time loss costs. It is set that the failure rate of non-faulty charging piles is reduced by half for each maintenance period.

[0079] The reward for emergency repair targets is:

[0080] (6)

[0081] in, Rewards will be given for the repair targets of faulty charging pile i. , The user time loss cost and repair resource cost of faulty charging pile i are separated. i This refers to the time period during which the faulty charging station was repaired. For a time period l i The user's lost time due to emergency repair of the faulty charging pile i, in minutes per vehicle, is calculated using equations (2) and (3). For a time period l i The electric vehicle arrival rate at the charging station, where c1 is the economic cost coefficient for lost time per user, expressed in yuan per minute. For a time period l i The unit time repair resource cost of the faulty charging pile i;

[0082] The maintenance target reward is:

[0083] (7)

[0084] in, A maintenance target reward is given for non-faulty charging pile j. , The maintenance loss value cost and maintenance resource cost of non-faulty charging pile j are respectively... j The time period during which non-faulty charging pile j is maintained, t e This represents the duration of a maintenance period for a non-faulty charging station, where FL(.) is used for rounding down, and λ... j The failure rate of non-faulty charging piles before maintenance. For a time period l j The user loss time caused by maintenance of non-faulty charging pile j, in minutes per vehicle, is calculated using equations (2) and (3). For a time period l j Electric vehicle arrival rate at charging stations For a time period l j The unit time maintenance resource cost of non-faulty charging pile j;

[0085] The constraints include the repair time constraint for the faulty charging pile i, the maintenance time constraint for the non-faulty charging pile j, and the operation and maintenance sequence constraint. The reward for the repair time constraint of the faulty charging pile i is:

[0086] (8)

[0087] in, Incentives are awarded based on the time constraints of emergency repairs for faulty charging pile i. , , The faulty charging pile i is set as the repair time value under simple fault type, medium complex fault type and high complex fault type respectively, and k1 is set as the penalty value for violating the repair time constraint.

[0088] The maintenance time constraint reward for non-faulty charging pile j is:

[0089] (9)

[0090] in, For non-faulty charging pile j, the maintenance time constraint reward is t. min t max These are the minimum and maximum maintenance times for a single charging pile, respectively, and k2 is the penalty value for violating the maintenance time constraint.

[0091] The reward for operation and maintenance time-sequence constraints is:

[0092] (10)

[0093] in, For the time-series constraint reward of the operation and maintenance of charging pile k, l k l k+1 Let t represent the time periods during the maintenance of charging piles k and (k+1), respectively. k K is the maintenance duration of charging pile k, and k3 is the penalty value of the set maintenance timing constraint.

[0094] In the deep reinforcement learning model for optimizing the operation and maintenance time of charging piles, the deep reinforcement learning algorithm adopts the DDPG algorithm to form the DDPG model for optimizing the operation and maintenance time of charging piles, and makes online decisions on the operation and maintenance time of charging piles for priority charging stations.

[0095] After the maintenance of the charging station is completed, the next charging station to be prioritized for maintenance and its charging pile maintenance order are determined in real time from the remaining charging stations. The DDPG model makes online decisions on the maintenance duration and proceeds to the next charging station for charging pile maintenance, as detailed below:

[0096] After the maintenance of the charging piles at the priority charging stations is completed, the next priority charging station and its charging pile maintenance order are determined in real time based on the risk index values ​​and health status values ​​of the charging piles in the remaining charging stations. The maintenance duration is then determined online by the DDPG model, and the maintenance of the charging piles at the next charging station is initiated. This process is repeated until all charging stations are maintained.

[0097] This invention takes the operation and maintenance optimization of 10 charging stations within a certain operation and maintenance management area as an example. The 10 charging stations are distributed in different highway service areas, numbered 1 to 10. Each charging station has 12 charging piles, also numbered 1 to 12. Let W be the average battery capacity of electric vehicle users. batt Average percentage electricity demand δ av The values ​​are 30 kWh and 80%, respectively. The repair times for simple, medium, and high-complexity fault types are set at 30 minutes, 45 minutes, and 60 minutes, respectively. The minimum and maximum maintenance times for a single charging pile are 10 minutes and 40 minutes, respectively. The duration of a maintenance period for a non-faulty charging pile is 20 minutes. The economic cost coefficient c1 for lost time per user is 1.5 yuan / minute. One maintenance team is set up, with daily maintenance hours from 8:00 to 18:00. With the number of faulty charging piles at 10 charging stations being 0, 0, 2, 1, 3, 2, 1, 0, 0, and 0, respectively, the method proposed in this invention is used for the first maintenance strategy decision.

[0098] 1) Based on the risk index values ​​of the 10 charging stations and the health status values ​​of the charging piles in the charging stations, the charging station with priority for maintenance is determined to be No. 5. Then, based on the size of the health status value of the charging piles in charging station No. 5, the maintenance order of the charging piles is determined as: 10, 1, 3, 7, 5, 11, 2, 9, 6, 12, 4, 8.

[0099] 2) Construct a DDPG model to optimize the operation and maintenance time of charging piles. The operation and maintenance time of charging piles at charging station No. 5 is determined online as shown in Table 1. Among them, the fault types of 2 faulty charging piles are high-complexity fault types and the fault type of 1 faulty charging pile is simple fault type.

[0100] As shown in Table 1, emergency repairs of faulty charging piles take priority over maintenance of non-faulty charging piles, with simpler fault types requiring earlier repairs. The lower the health status value of a charging pile, the earlier its maintenance priority and the longer its corresponding maintenance time. This indicates that the method proposed in this invention performs maintenance with different priorities while coordinating emergency repairs and maintenance. Furthermore, the differentiated maintenance time of charging piles obtained by the charging pile maintenance time optimization DDPG model can reasonably allocate maintenance time and overcome the problems of rough and untargeted solutions in traditional maintenance.

[0101] 3) After the maintenance of the charging piles at the priority charging stations is completed, the next priority charging station and its charging pile maintenance order are determined in real time based on the risk index values ​​of the remaining charging stations and the health status values ​​of the charging piles in the charging stations. The maintenance duration is determined online by the DDPG model, and the maintenance of the charging piles at the next charging station is started. The steps are repeated until all charging stations are maintained.

[0102] Assuming a maintenance time of 40 minutes per charging pile under conventional operation and maintenance methods, the maintenance times for 10 charging stations using the method of this invention are 4160.8 minutes and 4800 minutes, respectively. The sum of the time loss cost for emergency repairs and the value loss cost for maintenance are RMB 14950.8 and RMB 16200, respectively, a reduction of 7.71%. The operation and maintenance resource costs are RMB 41608 and RMB 48000, respectively, a reduction of 13.32%. This significantly reduces the charging pile maintenance time, the value loss cost of operation and maintenance, and the operation and maintenance resource costs, thereby improving the overall operation and maintenance efficiency and effectiveness.

[0103] Table 1 Maintenance time of charging piles at charging station No. 5

[0104] Operation and maintenance order Charging station number Operation and maintenance type Maintenance time / minute 1 10 Emergency repair 30 2 1 Emergency repair 60 3 3 Emergency repair 60 4 7 maintain 28 5 5 maintain 28 6 11 maintain 25 7 2 maintain 19 8 9 maintain 18 9 6 maintain 15 10 12 maintain 15 11 4 maintain 14 12 8 maintain 11

Claims

1. A deep reinforcement learning-based operation and maintenance optimization strategy for electric vehicle charging stations is proposed, characterized in that... The proposed operation and maintenance optimization strategy for electric vehicle charging stations based on deep reinforcement learning includes the following steps: 1) Based on the risk index values ​​of the charging station and the health status values ​​of the charging piles in the charging station, determine the charging station that should be prioritized for maintenance and the order of maintenance of the charging piles in that charging station. 2) Construct a DDPG model to optimize the charging pile operation and maintenance time, taking into account the uncertainty of the number and types of faulty charging piles and the arrival rate of electric vehicles, the time cost of electric vehicle users' lost time and the goal of minimizing operation and maintenance resource costs, and make online decisions on the charging pile operation and maintenance time of priority charging stations. 3) After the operation and maintenance of the charging station is completed, the next charging station to be prioritized for operation and maintenance and its charging pile operation and maintenance order are determined in real time among the remaining charging stations. The DDPG model makes online decisions on the operation and maintenance duration and proceeds to the next charging station for charging pile operation and maintenance.

2. The electric vehicle charging station operation and maintenance optimization strategy based on deep reinforcement learning according to claim 1, characterized in that, Based on the risk index values ​​of the charging stations and the health status values ​​of the charging piles within them, the charging stations with priority for maintenance and the order of maintenance for their charging piles are determined, as follows: Based on the risk index values ​​of charging stations, the charging station with the lowest risk index value is identified as the priority charging station for operation and maintenance; then the order of operation and maintenance of charging piles at the charging station is determined: emergency repair of faulty charging piles takes priority over maintenance of non-faulty charging piles. The repair order is determined according to the fault type of the charging pile. The fault types are divided into simple fault types, medium complex fault types, and high complex fault types. Simple fault types are repaired first, and high complex fault types are repaired last. The maintenance order is determined by the order of the health status values ​​of non-faulty charging piles. The smaller the health status value, the earlier the maintenance order. When the health status value is greater than the set health value of the charging pile, no maintenance is performed.

3. The electric vehicle charging station operation and maintenance optimization strategy based on deep reinforcement learning according to claim 1, characterized in that, A DDPG model is constructed to optimize the charging pile maintenance time, taking into account the uncertainties of the number and types of faulty charging piles and the arrival rate of electric vehicles, as well as the objectives of minimizing the lost time cost for electric vehicle users and the cost of operation and maintenance resources. This model enables online decision-making regarding the priority of charging station maintenance time, as detailed below: Considering the uncertainty of the number of faulty charging piles at charging stations, the uncertainty of the fault type of the faulty charging piles, the uncertainty of the arrival rate of electric vehicles, and the goal of minimizing the time cost and operation and maintenance resource cost for electric vehicle users, the problem of optimizing the operation and maintenance time of charging piles at charging stations is transformed into a deep reinforcement learning decision framework based on the order of operation and maintenance of charging piles at charging stations. A deep reinforcement learning model for optimizing the operation and maintenance time of charging piles is constructed to decide the priority of the operation and maintenance time of charging piles at charging stations. Design a deep reinforcement learning model for optimizing the operation and maintenance time of charging piles, including actions, states, rewards, and deep reinforcement learning algorithm elements. Actions include the emergency repair time of faulty charging piles and the maintenance time of non-faulty charging piles, represented as: (1) Where 'a' represents the action space. , These represent the emergency repair time for faulty charging pile i and the maintenance time for non-faulty charging pile j, respectively. fr J ufm These are the sets of numbers for faulty and non-faulty charging piles in the priority maintenance charging stations, respectively. The state space consists of the status identifier of the charging pile, the fault type of the faulty charging pile, and the arrival rate of the charging electric vehicle, and is represented as: (2) Where s is the state space, The status identifier for charging pile k is divided into fault status and non-fault status. The value is 1, indicating a non-fault state. The value is 0, where K is the number of charging piles in the priority maintenance charging station. The fault types of charging pile i are categorized into simple fault types, medium-complexity fault types, and highly complex fault types, with corresponding... The values ​​are 0, 1, and 2 respectively. The electric vehicle arrival rate during operation and maintenance period t; The rewards include target rewards and constraint rewards, both of which are negative. Target rewards include emergency repair target rewards and maintenance target rewards. Emergency repair target rewards are the sum of user time loss costs and emergency repair resource costs, while maintenance target rewards are the sum of maintenance loss value costs and maintenance resource costs. User time loss costs are converted into economic loss costs by converting user time lost due to charging pile operation and maintenance. Maintenance loss value costs are defined as the product of failure rate and user time loss costs. It is set that the failure rate of non-faulty charging piles is reduced by half for each maintenance period. The reward for emergency repair targets is: (3) in, Rewards will be given for the repair targets of faulty charging pile i. , The user time loss cost and repair resource cost of faulty charging pile i are separated. i This refers to the time period during which the faulty charging station was repaired. For a time period l i The time lost to users due to emergency repairs of faulty charging stations, expressed in minutes per vehicle. For a time period l i The electric vehicle arrival rate at the charging station, where c1 is the economic cost coefficient for lost time per user, expressed in yuan per minute. For a time period l i The unit time repair resource cost of the faulty charging pile i; The maintenance target reward is: (7) in, A maintenance target reward is given for non-faulty charging pile j. , The maintenance loss value cost and maintenance resource cost of non-faulty charging pile j are respectively... j The time period during which non-faulty charging pile j is maintained, t e This represents the duration of a maintenance period for a non-faulty charging station, where FL(.) is used for rounding down, and λ... j The failure rate of non-faulty charging piles before maintenance. For a time period l j The time lost to users due to maintenance of non-faulty charging piles, in minutes per vehicle. For a time period l j Electric vehicle arrival rate at charging stations For a time period l j The unit time maintenance resource cost of non-faulty charging pile j; The constraints include the repair time constraint for the faulty charging pile i, the maintenance time constraint for the non-faulty charging pile j, and the operation and maintenance sequence constraint. The reward for the repair time constraint of the faulty charging pile i is: (8) in, Incentives are awarded based on the time constraints of emergency repairs for faulty charging pile i. , , The faulty charging pile i is set as the repair time value under simple fault type, medium complex fault type and high complex fault type respectively, and k1 is set as the penalty value for violating the repair time constraint. The maintenance time constraint reward for non-faulty charging pile j is: (9) in, For non-faulty charging pile j, the maintenance time constraint reward is t. min t max These are the minimum and maximum maintenance times for a single charging pile, respectively, and k2 is the penalty value for violating the maintenance time constraint. The reward for operation and maintenance time-sequence constraints is: (10) in, For the time-series constraint reward of the operation and maintenance of charging pile k, l k l k+1 Let t represent the time periods during the maintenance of charging piles k and (k+1), respectively. k k3 represents the maintenance duration of charging pile k, and k3 is the penalty value for the set maintenance timing constraint. In the deep reinforcement learning model for optimizing the operation and maintenance time of charging piles, the deep reinforcement learning algorithm adopts the DDPG algorithm to form the DDPG model for optimizing the operation and maintenance time of charging piles, and makes online decisions on the operation and maintenance time of charging piles for priority charging stations.

Images