Charging assisting method and device of electric vehicle, computer equipment, readable storage medium and program product

Abstract

The invention relates to a charging assisting method, device and equipment for an electric vehicle, a storage medium and a program product, and relates to the technical field of power dispatching. By adopting the method, the sustainability and environmental friendliness of charging scheduling can be improved. The method comprises the following steps: constructing a relation transfer function between carbon emission intensity and marginal output increase; according to the operation mode and the charge state, action spaces are determined, the action spaces are summarized according to a preset number of the electric vehicle, and a total action space is obtained; determining state spaces according to the current position and the charge state, and summarizing the state spaces according to a preset number to obtain an overall state space; based on the overall action space, the overall state space and the relation transfer function, constructing a state transition process model of the electric vehicle; inputting model information of the action space, the state space and the state transition process model into a reinforcement learning model to obtain a target charging strategy; the target charging strategy is used for assisting the electric vehicle in charging.

Description

Technical Field

[0001] The present application relates to the technical field of electric power dispatching, and in particular to a charging assistance method, device, computer equipment, computer-readable storage medium and computer program product for an electric vehicle. Background Art

[0002] Electric vehicles are an emerging technology with great potential to reduce carbon dioxide, with emissions from electric vehicles being able to reduce carbon emissions by up to 60% in some cases compared to conventional hybrid vehicles. Although electric vehicles do not produce carbon emissions directly, carbon dioxide emissions are generated during the charging process of electric vehicles because the electricity may be generated from fossil fuels.

[0003] In traditional technologies, the focus in electric vehicle charging scheduling is mainly on reducing operating costs and alleviating grid load, and the environmental impact of carbon emissions is usually ignored. Even though some studies have considered the impact of grid emission factors on the environment, these methods often put reducing power generation costs first and lack comprehensive control of carbon emissions, resulting in obvious deficiencies in current technologies in terms of environmental friendliness and carbon emission control. Summary of the invention

[0004] Based on this, it is necessary to provide a charging assistance method, device, computer equipment, computer readable storage medium and computer program product for an electric vehicle in response to the above technical problems.

[0005] In a first aspect, the present application provides a charging assistance method for an electric vehicle, comprising:

[0006] Obtain the carbon emission intensity of the power generation equipment, and obtain the marginal output growth of the power generation equipment due to the charging behavior of the electric vehicle, and construct a relationship conversion function between the carbon emission intensity and the marginal output growth;

[0007] Determine the action space of the electric vehicle according to the operation mode and the state of charge of the electric vehicle, and summarize the action space according to the preset number of the electric vehicle to obtain the overall action space;

[0008] Determine the state space of the electric vehicle according to the current position and charge state of the electric vehicle, and summarize the state space according to the preset number to obtain the overall state space;

[0009] Based on the overall action space, the overall state space and the relationship conversion function, construct a state transition process model of the electric vehicle;

[0010] The model information of the action space, the state space and the state transition process model is input into a trained reinforcement learning model, and the reinforcement learning model outputs a target charging strategy applied to the electric vehicle based on the action space, the state space and the model information; the target charging strategy is used to assist the electric vehicle in charging.

[0011] In one embodiment, determining the action space of the electric vehicle according to the operation mode and the state of charge of the electric vehicle includes:

[0012] When the operation mode is operation, a first decision result of the electric vehicle is obtained according to the charge state; when the operation mode is charging, a second decision result of the electric vehicle is obtained according to the number of charging positions and the charge state; and the action space of the electric vehicle is determined according to the first decision result and the second decision result.

[0013] In one embodiment, the state transition process model of the electric vehicle is constructed based on the overall action space, the overall state space and the relationship conversion function, including:

[0014] When the operation mode in the overall action space is operation, the position change and the first charge change of the electric vehicle are obtained, and the updated first state space is determined according to the position change and the first charge change; when the operation mode in the overall action space is charging, the node number corresponding to the charging station where the electric vehicle is located and the second charge change of the electric vehicle are obtained, and the updated second state space is determined according to the node number and the second charge change; the state transition process model is constructed according to the overall state space, the first state space, the second state space and the relationship conversion function.

[0015] In one embodiment, obtaining the marginal output growth of the power generation equipment due to the charging behavior of the electric vehicle includes:

[0016] According to the historical power generation parameters of the power generation equipment, the first optimal power flow calculation is performed on the power generation equipment to obtain the historical output of the power generation equipment; according to the current power generation parameters of the power generation equipment, the second optimal power flow calculation is performed on the power generation equipment to obtain the current output of the power generation equipment; based on the historical output and the current output, the marginal output growth of the power generation equipment is determined.

[0017] In one embodiment, before inputting the model information of the action space, the state space and the state transition process model into the trained reinforcement learning model, the method further includes:

[0018] The charging energy consumption of the power generation equipment is obtained, and a feedback function corresponding to the power generation equipment is determined according to the carbon emission intensity and the charging energy consumption; the feedback function is input into a PPO (Proximal Policy Optimization) algorithm for calculation to obtain a loss function, and according to the loss function, an initial reinforcement learning model is trained in a preset training environment to obtain the trained reinforcement learning model.

[0019] In one embodiment, the method further comprises:

[0020] The target charging strategy is set as the experimental group for a simulation experiment to obtain simulation experiment results, and the disordered charging strategy is set as the control group for a simulation control to obtain simulation control results; based on the simulation experiment results and the simulation control results, the total carbon emission reduction of the power generation equipment is obtained, and based on the total carbon emission reduction, the carbon emission intensity reduction is determined.

[0021] In a second aspect, the present application also provides a charging auxiliary device for an electric vehicle, comprising:

[0022] A relationship conversion module is used to obtain the carbon emission intensity of the power generation equipment, obtain the marginal output growth of the power generation equipment due to the charging behavior of the electric vehicle, and construct a relationship conversion function between the carbon emission intensity and the marginal output growth;

[0023] An action space module, used for determining the action space of the electric vehicle according to the operation mode and the state of charge of the electric vehicle, and summarizing the action space according to the preset number of the electric vehicle to obtain an overall action space;

[0024] A state space module, used for determining the state space of the electric vehicle according to the current position and charge state of the electric vehicle, and summarizing the state space according to the preset number to obtain an overall state space;

[0025] A state transfer module, used for constructing a state transfer process model of the electric vehicle based on the overall action space and the overall state space;

[0026] A strategy generation module is used to input the model information of the action space, the state space and the state transition process model into a trained reinforcement learning model, and the reinforcement learning model outputs a target charging strategy applied to the electric vehicle based on the action space, the state space and the model information; the target charging strategy is used to assist the electric vehicle in charging.

[0027] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0028] Obtain the carbon emission intensity of the power generation equipment, and obtain the marginal output growth of the power generation equipment due to the charging behavior of the electric vehicle, and construct a relationship conversion function between the carbon emission intensity and the marginal output growth; determine the action space of the electric vehicle according to the operation mode and charge state of the electric vehicle, and summarize the action space according to the preset number of the electric vehicle to obtain the overall action space; determine the state space of the electric vehicle according to the current position and charge state of the electric vehicle, and summarize the state space according to the preset number to obtain the overall state space; construct a state transition process model of the electric vehicle based on the overall action space, the overall state space and the relationship conversion function; input the model information of the action space, the state space and the state transition process model into a trained reinforcement learning model, and the reinforcement learning model outputs a target charging strategy applied to the electric vehicle according to the action space, the state space and the model information; the target charging strategy is used to assist the electric vehicle in charging.

[0029] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0030] Obtain the carbon emission intensity of the power generation equipment, and obtain the marginal output growth of the power generation equipment due to the charging behavior of the electric vehicle, and construct a relationship conversion function between the carbon emission intensity and the marginal output growth; determine the action space of the electric vehicle according to the operation mode and charge state of the electric vehicle, and summarize the action space according to the preset number of the electric vehicle to obtain the overall action space; determine the state space of the electric vehicle according to the current position and charge state of the electric vehicle, and summarize the state space according to the preset number to obtain the overall state space; construct a state transition process model of the electric vehicle based on the overall action space, the overall state space and the relationship conversion function; input the model information of the action space, the state space and the state transition process model into a trained reinforcement learning model, and the reinforcement learning model outputs a target charging strategy applied to the electric vehicle according to the action space, the state space and the model information; the target charging strategy is used to assist the electric vehicle in charging.

[0031] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0032] Obtain the carbon emission intensity of the power generation equipment, and obtain the marginal output growth of the power generation equipment due to the charging behavior of the electric vehicle, and construct a relationship conversion function between the carbon emission intensity and the marginal output growth; determine the action space of the electric vehicle according to the operation mode and charge state of the electric vehicle, and summarize the action space according to the preset number of the electric vehicle to obtain the overall action space; determine the state space of the electric vehicle according to the current position and charge state of the electric vehicle, and summarize the state space according to the preset number to obtain the overall state space; construct a state transition process model of the electric vehicle based on the overall action space, the overall state space and the relationship conversion function; input the model information of the action space, the state space and the state transition process model into a trained reinforcement learning model, and the reinforcement learning model outputs a target charging strategy applied to the electric vehicle according to the action space, the state space and the model information; the target charging strategy is used to assist the electric vehicle in charging.

[0033] The above-mentioned electric vehicle charging assistance method, device, computer equipment, computer-readable storage medium and computer program product use the reduction of carbon emissions as the main driving force for optimizing charging scheduling, surpassing the traditional technology method that takes reducing operating costs and alleviating grid load as the main goal. By combining the PPO reinforcement learning algorithm to optimize the charging scheduling of electric vehicles, it can dynamically adapt to unpredictable changes and randomness in the environment, and continuously adjust and optimize the charging strategy, so as to achieve precise control of the charging process, effectively reduce the negative impact of electric vehicle charging on the environment, improve the sustainability and environmental friendliness of charging scheduling, and provide technical support for the green development of electric vehicle charging. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0035] FIG1 is a diagram showing an application environment of a charging assistance method for an electric vehicle in one embodiment;

[0036] Figure 2 A schematic diagram of a flow chart of a charging assistance method for an electric vehicle in one embodiment;

[0037] Figure 3 A schematic diagram of a flow chart of steps for determining an action space of an electric vehicle in one embodiment;

[0038] Figure 4 A schematic diagram of a flow chart of steps for constructing a state transition process model in one embodiment;

[0039] Figure 5 A schematic diagram of a framework of a target charging strategy for an electric vehicle in one embodiment;

[0040] Figure 6 It is a schematic flow chart of a charging assistance method for an electric vehicle in a specific embodiment;

[0041] Figure 7 is a structural block diagram of a charging auxiliary device for an electric vehicle in one embodiment;

[0042] FIG. 8 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] The charging assistance method for an electric vehicle provided in an embodiment of the present application can be applied in an application environment as shown in FIG1 . The terminal communicates with the server via a network. The data storage system can store data that the server needs to process. The data storage system can be integrated on the server, or placed on a cloud or other network server.

[0045] Specifically, the charging assistance method for an electric vehicle provided in the embodiment of the present application can be executed by a server.

[0046] Exemplarily, the server obtains the carbon emission intensity of the power generation equipment, and obtains the marginal output growth of the power generation equipment due to the charging behavior of the electric vehicle, and constructs a relationship conversion function between the carbon emission intensity and the marginal output growth; the server determines the action space of the electric vehicle according to the operation mode and charge state of the electric vehicle, and summarizes the action space according to the preset number of the electric vehicle to obtain the overall action space; the server determines the state space of the electric vehicle according to the current position and charge state of the electric vehicle, and summarizes the state space according to the preset number to obtain the overall state space; the server constructs a state transition process model of the electric vehicle based on the overall action space, the overall state space and the relationship conversion function; the server inputs the model information of the action space, the state space and the state transition process model into the trained reinforcement learning model, and the reinforcement learning model outputs the target charging strategy applied to the electric vehicle according to the action space, the state space and the model information; the target charging strategy is used to assist the electric vehicle in charging.

[0047] In the application environment shown in FIG1 , the terminal may be, but is not limited to, various personal computers, notebook computers, smart phones, and tablet computers. The server may be implemented as an independent server or a server cluster consisting of multiple servers.

[0048] In one embodiment, Figure 2 As shown, a charging auxiliary method for an electric vehicle is provided, and the method is applied to the server in FIG1 as an example for description, and includes the following steps:

[0049] Step S201, obtaining the carbon emission intensity of the power generation equipment, and obtaining the marginal output growth of the power generation equipment due to the charging behavior of the electric vehicle, and constructing a relationship conversion function between the carbon emission intensity and the marginal output growth.

[0050] Specifically, in order to describe the relationship between the output of power generation equipment and carbon emissions, the carbon emission intensity of power generation equipment is defined here, that is, the carbon emissions per unit of power generation:

[0051]

[0052] in, For the power generation node Carbon emission intensity, in ; For power generation nodes Historical average annual total carbon emissions, in ; is the number of hours in a year, where is 365 days; is the average output of power node i at time t. Assuming that traditional energy sources are all coal-fired units, and new energy sources consider wind energy and solar energy, the total carbon emissions of each power generation node are The composition is:

[0053]

[0054] in, Power generation nodes Historical average annual carbon emissions from traditional power generation, wind power generation, and solar power generation.

[0055] Step S202 , determining the action space of the electric vehicle according to the operation mode and the state of charge of the electric vehicle, and summarizing the action space according to the preset number of the electric vehicle to obtain the overall action space.

[0056] Specifically, assuming that there are M electric vehicles and N charging stations, and each electric vehicle has two operation modes: charging and operation, then the action space of each electric vehicle is:

[0057]

[0058] in, When means that the electric vehicle will execute the operation mode at time t, When it is t, it means that the charging mode will be executed at time t, and go to Charging station for charging. Constrained by the state of charge of the electric vehicle, when the SOC of the electric vehicle is greater than When the SOC of an electric vehicle is less than When the SOC of an electric vehicle is In between, you can decide whether to charge, that is:

[0059]

[0060] in, are the upper and lower thresholds for charging electric vehicles; the overall action space of the system It is composed of the actions of all electric vehicles, namely:

[0061]

[0062] The time window is 30 minutes, that is, a decision is made every 30 minutes. At the same time, the overall action is constrained by the charging location, and the number of electric vehicles charging in charging station n at the same time cannot exceed , is the number of charging positions in charging station n.

[0063] Step S203, determining the state space of the electric vehicle according to the current position and charge state of the electric vehicle, and summarizing the state space according to preset numbers to obtain the overall state space.

[0064] Specifically, the state of each electric vehicle consists of its current location and state of charge, namely:

[0065]

[0066] in, is the current position of electric vehicle m at time t, defined by the node number; is the state of charge of electric vehicle m at time t. The overall state space of the system It is composed of the states of all electric vehicles, namely:

[0067]

[0068] Step S204: constructing a state transition process model of the electric vehicle based on the overall action space, the overall state space and the relationship conversion function.

[0069] Specifically, when the server identifies that the operation mode in the overall action space is operation, the server obtains the position change and the first charge change of the electric vehicle, and determines the updated first state space based on the position change and the first charge change; when the server identifies that the operation mode in the overall action space is charging, the server obtains the node number corresponding to the charging station where the electric vehicle is located and the second charge change of the electric vehicle, and determines the updated second state space based on the node number and the second charge change; and constructs a state transition process model based on the overall state space, the first state space, the second state space and the relationship conversion function.

[0070] Step S205, inputting the model information of the action space, state space and state transition process model into the trained reinforcement learning model, and the reinforcement learning model outputs a target charging strategy applied to the electric vehicle according to the action space, state space and model information; the target charging strategy is used to assist the electric vehicle in charging.

[0071] Specifically, the PPO algorithm can be used as a reinforcement learning model. This model is based on the actor-critic architecture, has high computational efficiency, and is easier to parallelize. The PPO algorithm uses the ratio of the action probability of the current strategy to the action probability of the previous strategy, which can be expressed as:

[0072]

[0073] in Indicates the current policy parameters Next, take action at time step t The probability ratio, Given the current state Take action at time step t The probability of strategy parameters , Given the current state and the previous strategy parameters Take action at time step t The probability of . This ratio is used to calculate the loss function and alternative target in PPO. The loss function can be expressed as:

[0074]

[0075] In the above formula, Indicates the current policy parameters, is the expectation at time step t, is an estimate of the advantage function, implemented by GAE (Generalized Advantage Estimation), which can be expressed as:

[0076]

[0077]

[0078] in, is the discount factor, is the eligibility tracking parameter, is the parameter of the value function, Based on the current parameters An estimate of the state value function of . Updated by the critic network, the calculation formula is as follows:

[0079]

[0080] in, is the index of iteration, is the learning rate of the actor network, Trajectory sampling iteration , is the length of the set, m is the trajectory number, and T is the number of time steps. Represents the time step The sum of the initial bonus discounts is calculated as follows:

[0081]

[0082] In order to limit the scope of each update, the PPO algorithm can use a clip function, which is expressed as follows:

[0083]

[0084] in, It is the proxy target to be clipped. Clip is a function that changes the value to 1. and 1+ The function between is a hyperparameter that controls the clipping range. The formula used to update the strategy is as follows:

[0085]

[0086] in, is the learning rate of the actor network. In the PPO algorithm, some hyperparameters such as clipping range and learning rate are automatically adjusted.

[0087] In the above-mentioned electric vehicle charging assistance method, reducing carbon emissions is used as the main driving force for optimizing charging scheduling, which goes beyond the traditional technology method that takes reducing operating costs and alleviating grid load as the main goal. By combining the PPO reinforcement learning algorithm to optimize the charging scheduling of electric vehicles, it can dynamically adapt to unpredictable changes and randomness in the environment, and continuously adjust and optimize the charging strategy, thereby achieving precise control of the charging process, effectively reducing the negative impact of electric vehicle charging on the environment, and improving the sustainability and environmental friendliness of charging scheduling, thereby providing technical support for the green development of electric vehicle charging.

[0088] In one embodiment, if Figure 3 As shown, in the above step S202, the action space of the electric vehicle is determined according to the operation mode and the state of charge of the electric vehicle, which specifically includes the following steps:

[0089] Step S301, when the operation mode is operation, a first decision result of the electric vehicle is obtained according to the state of charge; the first decision result indicates that the electric vehicle cannot be charged or has not been charged.

[0090] Step S302, when the operation mode is charging, a second decision result of the electric vehicle is obtained according to the number of charging positions and the state of charge; the second decision result indicates that the electric vehicle chooses to charge.

[0091] Step S303, determining the action space of the electric vehicle according to the operation modes in the first decision result and the second decision result.

[0092] Specifically, when the server identifies that the operating mode is operation, it determines the first decision result of the electric vehicle according to the state of charge; when the operating mode is charging, it determines the second decision result of the electric vehicle according to the number of charging positions and the state of charge; finally, according to the operating modes in the first decision result and the second decision result, the action space of the electric vehicle is determined.

[0093] In this embodiment, by distinguishing different operation modes and respectively determining corresponding decision results, the action space of the electric vehicle can be accurately determined according to the decision results.

[0094] In one embodiment, if Figure 4 As shown, in the above step S204, based on the overall action space, the overall state space and the relationship conversion function, a state transition process model of the electric vehicle is constructed, which specifically includes the following steps:

[0095] When the operation mode in the overall action space is operation, the position change and the first charge change of the electric vehicle are obtained, and the updated first state space is determined according to the position change and the first charge change; when the operation mode in the overall action space is charging, the node number corresponding to the charging station where the electric vehicle is located and the second charge change of the electric vehicle are obtained, and the updated second state space is determined according to the node number and the second charge change; a state transition process model is constructed according to the overall state space, the first state space, the second state space and the relationship conversion function.

[0096] Specifically, when the operation mode in the overall action space is operation, that is, When , the position of the electric car will change, and the change amount conforms to the binomial distribution:

[0097]

[0098] in, is the node change amount, For the original location move The new node position reached after units, is the probability of node change, where n and p are 2 and 0.3 respectively. The transfer process of the state of charge of electric vehicles conforms to the Poisson distribution:

[0099]

[0100]

[0101] in, is the change in state of charge, where The value of is 7.

[0102] When the operation mode in the overall action space is charging, that is, When , the electric vehicle position will become the node corresponding to the charging station, that is:

[0103]

[0104] in, For charging stations The corresponding node number; the charge state changes as follows:

[0105]

[0106] in, is the charging power, is the time window interval, and They are respectively the time and power consumed in the trip to the charging station.

[0107] In this embodiment, by distinguishing different operation modes and respectively determining the corresponding first state space and second state space, a state transition process model of the electric vehicle can be accurately constructed according to the overall state space, the first state space and the second state space.

[0108] In one embodiment, in the above step S201, obtaining the marginal output growth of the power generation equipment due to the charging behavior of the electric vehicle specifically includes the following steps:

[0109] According to the historical power generation parameters of the power generation equipment, the first optimal power flow calculation is performed on the power generation equipment to obtain the historical output of the power generation equipment; according to the current power generation parameters of the power generation equipment, the second optimal power flow calculation is performed on the power generation equipment to obtain the current output of the power generation equipment; based on the historical output and current output, the marginal output growth of the power generation equipment is determined.

[0110] Specifically, the impact of electric vehicle charging behavior on the marginal output growth of power generation equipment can be approximated by the difference between two DC optimal power flow calculation results. First, the first optimal power flow calculation is performed under the premise of known basic load to determine the output of each power generation equipment. Then, assuming that the load caused by electric vehicle charging is increased at load node j, a second optimal power flow calculation is performed in this case to obtain the new power generation equipment output. The difference in power generation equipment output obtained from the two optimal power flow calculations can be regarded as the marginal output growth of power generation equipment caused by electric vehicle charging at load node j. Define the marginal output growth factor is the increase in marginal output of power generation equipment at power node i caused by charging of electric vehicles at load node j at a specific time t. The calculation formula is as follows:

[0111]

[0112] in, is the marginal output growth factor, is the charging power of the electric vehicle at load node j; It is the difference in output of power generation equipment at power node i.

[0113] In one embodiment, before inputting the model information of the action space, the state space, and the state transition process model into the trained reinforcement learning model, the solution in the present application further includes the following steps:

[0114] The charging energy consumption of the power generation equipment is obtained, and the feedback function corresponding to the power generation equipment is determined according to the carbon emission intensity and the charging energy consumption; the feedback function is input into the PPO algorithm for calculation to obtain the loss function, and according to the loss function, the initial reinforcement learning model is trained in a preset training environment to obtain a trained reinforcement learning model.

[0115] Specifically, the server obtains the charging energy consumption of the power generation equipment, and determines the feedback function corresponding to the power generation equipment according to the carbon emission intensity and the charging energy consumption; the feedback function is input into the PPO algorithm for calculation to obtain the loss function, and according to the loss function, the initial reinforcement learning model is trained in a preset training environment to obtain a trained reinforcement learning model.

[0116] For example, the feedback function is normalized based on carbon emissions as follows:

[0117]

[0118] in, is the carbon emissions at time t, is the charging energy consumption of electric vehicle m at time t, is the feedback function, is the standardized processing function. Finally, the total carbon emissions of all electric vehicles are:

[0119]

[0120] In one embodiment, after the reinforcement learning model outputs the target charging strategy for the electric vehicle based on the action space, the state space and the model information, the solution in the present application further includes the following steps:

[0121] The target charging strategy is set as the experimental group for simulation experiment to obtain the simulation experiment results, and the disordered charging strategy is set as the control group for simulation control to obtain the simulation control results; according to the simulation experiment results and the simulation control results, the total carbon emission reduction of the power generation equipment is obtained, and according to the total carbon emission reduction, the carbon emission intensity reduction is determined.

[0122] Specifically, the server sets up an experimental simulation control, the control group is a disordered charging strategy, and only when the SOC of the electric vehicle is lower than By designing examples, electric vehicles are analyzed according to indicators such as carbon emissions. Compared with disorderly charging of electric vehicles, a method is proposed to effectively reduce the marginal carbon emission growth and carbon cost of electric vehicle charging.

[0123] In one embodiment, Figure 5 and Figure 6 As shown, a charging assistance method for an electric vehicle in a specific embodiment is provided, which specifically includes the following steps:

[0124] Step S601, obtain the carbon emission intensity of the power generation equipment, perform the first optimal power flow calculation on the power generation equipment according to the historical power generation parameters of the power generation equipment, and obtain the historical output of the power generation equipment; perform the second optimal power flow calculation on the power generation equipment according to the current power generation parameters of the power generation equipment, and obtain the current output of the power generation equipment; based on the historical output and the current output, determine the marginal output growth of the power generation equipment.

[0125] Step S602: construct a relationship conversion function between carbon emission intensity and marginal output growth.

[0126] Step S603, when the operation mode is operation, a first decision result of the electric vehicle is obtained according to the state of charge; when the operation mode is charging, a second decision result of the electric vehicle is obtained according to the number of charging positions and the state of charge; an action space of the electric vehicle is determined according to the first decision result and the second decision result; and the action space is summarized according to the preset number of the electric vehicle to obtain an overall action space.

[0127] Step S604, according to the current position and charge state of the electric vehicle, determine the state space of the electric vehicle, and summarize the state space according to preset numbers to obtain the overall state space; based on the overall action space, the overall state space and the relationship conversion function, construct a state transition process model of the electric vehicle.

[0128] Step S605, obtain the charging energy consumption of the power generation equipment, and determine the feedback function corresponding to the power generation equipment according to the carbon emission intensity and the charging energy consumption; input the feedback function into the PPO algorithm for calculation to obtain the loss function, and according to the loss function, train the initial reinforcement learning model in a preset training environment to obtain a trained reinforcement learning model.

[0129] Step S606, inputting the model information of the action space, state space and state transition process model into the trained reinforcement learning model, and the reinforcement learning model outputs a target charging strategy applied to the electric vehicle based on the action space, state space and model information; the target charging strategy is used to assist the electric vehicle in charging.

[0130] Step S607, setting the target charging strategy as the experimental group for simulation experiment to obtain simulation experiment results, setting the disordered charging strategy as the control group for simulation control to obtain simulation control results; according to the simulation experiment results and the simulation control results, the total carbon emission reduction of the power generation equipment is obtained, and according to the total carbon emission reduction, the carbon emission intensity reduction is determined.

[0131] The beneficial effects brought by the above embodiments are as follows:

[0132] The innovation of this application is that it proposes an electric vehicle charging strategy with carbon emission minimization as the core goal. This strategy takes carbon emission reduction as the main driving force for optimizing charging scheduling, surpassing the traditional method of reducing operating costs and grid burden as the main goal. By focusing on environmental friendliness, this application achieves precise control of the charging process, effectively reduces carbon emissions, and improves the sustainability of charging scheduling, thereby providing a new solution for the green development of electric vehicle charging. At the same time, the use of the PPO reinforcement learning algorithm in the electric vehicle charging decision model has many advantages. Reinforcement learning can dynamically adapt to unpredictable changes and randomness in the environment through repeated interactions with the environment, thereby continuously adjusting and optimizing strategies. The PPO algorithm improves the stability and efficiency of learning by limiting the strategy update amplitude, and is suitable for handling complex dynamic environments involved in electric vehicle charging scheduling. In addition, the PPO algorithm performs well in balancing exploration and utilization, avoiding falling into local optimality and finding better solutions. Finally, the PPO algorithm has high computational efficiency and is easier to parallelize, which is particularly important for charging scheduling tasks that process large amounts of data.

[0133] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0134] Based on the same inventive concept, the embodiment of the present application also provides an electric vehicle charging assistance device for implementing the above-mentioned electric vehicle charging assistance method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in the embodiments of one or more electric vehicle charging assistance devices provided below can refer to the limitations of the electric vehicle charging assistance method above, and will not be repeated here.

[0135] In an exemplary embodiment, Figure 7 As shown, a charging auxiliary device for an electric vehicle is provided, comprising:

[0136] The relationship conversion module 701 is used to obtain the carbon emission intensity of the power generation equipment, obtain the marginal output growth of the power generation equipment due to the charging behavior of the electric vehicle, and construct a relationship conversion function between the carbon emission intensity and the marginal output growth;

[0137] The action space module 702 is used to determine the action space of the electric vehicle according to the operation mode and the state of charge of the electric vehicle, and summarize the action space according to the preset number of the electric vehicle to obtain the overall action space;

[0138] The state space module 703 is used to determine the state space of the electric vehicle according to the current position and charge state of the electric vehicle, and summarize the state space according to preset numbers to obtain the overall state space;

[0139] A state transfer module 704 is used to construct a state transfer process model of the electric vehicle based on the overall action space and the overall state space;

[0140] The strategy generation module 705 is used to input the model information of the action space, state space and state transition process model into the trained reinforcement learning model, and the reinforcement learning model outputs the target charging strategy applied to the electric vehicle based on the action space, state space and model information; the target charging strategy is used to assist the electric vehicle in charging.

[0141] In one embodiment, the action space module 702 is also used to obtain a first decision result of the electric vehicle according to the state of charge when the operation mode is operation; to obtain a second decision result of the electric vehicle according to the number of charging positions and the state of charge when the operation mode is charging; and to determine the action space of the electric vehicle according to the first decision result and the second decision result.

[0142] In one embodiment, the state transfer module 704 is also used to obtain the position change and the first charge change of the electric vehicle when the operation mode in the overall action space is operation, and determine the updated first state space according to the position change and the first charge change; when the operation mode in the overall action space is charging, obtain the node number corresponding to the charging station where the electric vehicle is located and the second charge change of the electric vehicle, and determine the updated second state space according to the node number and the second charge change; and construct a state transfer process model according to the overall state space, the first state space, the second state space and the relationship conversion function.

[0143] In one embodiment, the relationship conversion module 701 is also used to perform a first optimal power flow calculation on the power generation equipment according to the historical power generation parameters of the power generation equipment to obtain the historical output of the power generation equipment; perform a second optimal power flow calculation on the power generation equipment according to the current power generation parameters of the power generation equipment to obtain the current output of the power generation equipment; and determine the marginal output growth of the power generation equipment based on the historical output and the current output.

[0144] In one embodiment, the charging auxiliary device of the electric vehicle also includes a model building module, which is used to obtain the charging energy consumption of the power generation equipment, and determine the feedback function corresponding to the power generation equipment according to the carbon emission intensity and the charging energy consumption; the feedback function is input into the PPO algorithm for calculation to obtain the loss function, and according to the loss function, the initial reinforcement learning model is trained in a preset training environment to obtain a trained reinforcement learning model.

[0145] In one embodiment, the charging auxiliary device of the electric vehicle also includes an experimental comparison module, which is used to set the target charging strategy as the experimental group for simulation experiment to obtain simulation experiment results, and set the disordered charging strategy as the control group for simulation comparison to obtain simulation comparison results; according to the simulation experiment results and the simulation comparison results, the total carbon emission reduction of the power generation equipment is obtained, and according to the total carbon emission reduction, the carbon emission intensity reduction is determined.

[0146] Each module in the above-mentioned charging auxiliary device for electric vehicles can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0147] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG8. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store target charging strategy data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a charging auxiliary method for an electric vehicle is implemented.

[0148] Those skilled in the art will understand that the structure shown in FIG. 8 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0149] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0150] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0151] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0152] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0153] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0154] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0155] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A charging assistance method for an electric vehicle, characterized in that: The method comprises: Obtain the carbon emission intensity of the power generation equipment, and obtain the marginal output growth of the power generation equipment due to the charging behavior of the electric vehicle, and construct a relationship conversion function between the carbon emission intensity and the marginal output growth; Determine the action space of the electric vehicle according to the operation mode and the state of charge of the electric vehicle, and summarize the action space according to the preset number of the electric vehicle to obtain the overall action space; Determine the state space of the electric vehicle according to the current position and charge state of the electric vehicle, and summarize the state space according to the preset number to obtain the overall state space; Based on the overall action space, the overall state space and the relationship conversion function, construct a state transition process model of the electric vehicle; The model information of the action space, the state space and the state transition process model is input into a trained reinforcement learning model, and the reinforcement learning model outputs a target charging strategy applied to the electric vehicle based on the action space, the state space and the model information; the target charging strategy is used to assist the electric vehicle in charging.

2. The method according to claim 1, characterized in that The step of determining the action space of the electric vehicle according to the operation mode and the state of charge of the electric vehicle comprises: When the operation mode is operation, a first decision result of the electric vehicle is obtained according to the state of charge; the first decision result indicates that the electric vehicle cannot be charged or has not been charged; When the operation mode is charging, a second decision result of the electric vehicle is obtained according to the number of charging positions and the state of charge; the second decision result indicates that the electric vehicle chooses to charge; The action space of the electric vehicle is determined according to the operation modes in the first decision result and the second decision result.

3. The method according to claim 2, characterized in that The state transition process model of the electric vehicle is constructed based on the overall action space, the overall state space and the relationship conversion function, including: When the operation mode in the overall action space is operation, obtaining a position change amount and a first charge change amount of the electric vehicle, and determining an updated first state space according to the position change amount and the first charge change amount; When the operation mode in the overall action space is charging, obtaining a node number corresponding to the charging station where the electric vehicle is located and a second charge change of the electric vehicle, and determining an updated second state space according to the node number and the second charge change; The state transition process model is constructed according to the overall state space, the first state space, the second state space and the relationship conversion function.

4. The method according to claim 1, characterized in that: The obtaining of the marginal output growth of the power generation equipment due to the charging behavior of the electric vehicle includes: According to the historical power generation parameters of the power generation equipment, a first optimal power flow calculation is performed on the power generation equipment to obtain the historical output of the power generation equipment; According to the current power generation parameters of the power generation equipment, a second optimal power flow calculation is performed on the power generation equipment to obtain the current output of the power generation equipment; Based on the historical output conditions and the current output conditions, the marginal output growth of the power generation equipment is determined.

5. The method according to claim 1, characterized in that: Before inputting the model information of the action space, the state space and the state transition process model into the trained reinforcement learning model, the method further includes: Obtaining the charging energy consumption of the power generation equipment, and determining a feedback function corresponding to the power generation equipment according to the carbon emission intensity and the charging energy consumption; The feedback function is input into the PPO algorithm for calculation to obtain a loss function, and according to the loss function, the initial reinforcement learning model is trained in a preset training environment to obtain the trained reinforcement learning model.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: The target charging strategy is set as an experimental group to perform a simulation experiment to obtain a simulation experiment result, and the disordered charging strategy is set as a control group to perform a simulation control to obtain a simulation control result; According to the simulation experiment results and the simulation control results, the total carbon emission reduction of the power generation equipment is obtained, and according to the total carbon emission reduction, the carbon emission intensity reduction is determined.

7. A charging auxiliary device for an electric vehicle, characterized in that: The device comprises: A relationship conversion module is used to obtain the carbon emission intensity of the power generation equipment, obtain the marginal output growth of the power generation equipment due to the charging behavior of the electric vehicle, and construct a relationship conversion function between the carbon emission intensity and the marginal output growth; An action space module, used for determining the action space of the electric vehicle according to the operation mode and the state of charge of the electric vehicle, and summarizing the action space according to the preset number of the electric vehicle to obtain an overall action space; A state space module, used for determining the state space of the electric vehicle according to the current position and charge state of the electric vehicle, and summarizing the state space according to the preset number to obtain an overall state space; A state transfer module, used for constructing a state transfer process model of the electric vehicle based on the overall action space and the overall state space; A strategy generation module is used to input the model information of the action space, the state space and the state transition process model into a trained reinforcement learning model, and the reinforcement learning model outputs a target charging strategy applied to the electric vehicle based on the action space, the state space and the model information; the target charging strategy is used to assist the electric vehicle in charging.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images