Unmanned charging station resource configuration method and system based on deep reinforcement learning

Abstract

A kind of unmanned charging station resource configuration method and system based on deep reinforcement learning, comprising: obtaining the road information and electric vehicle quantity of current intersection in rated time period, and calculating the saturation content of charging demand of current road;Set time interval, obtain the saturation content of charging demand in rated time period according to time interval, obtain a group of saturation content of charging demand in time sequence;Based on deep neural network and reinforcement learning algorithm, a charging strategy model is constructed, the charging strategy model is trained through random time period in simulation environment, and the trained charging strategy model is obtained;The saturation content of charging demand is used as the input of the trained charging strategy model, the optimal value is output by the model, and the allocation of charging station resources is carried out based on the optimal value output by the model.The present application provides more accurate and comprehensive basis for resource configuration by real-time collection and analysis of charging data, effectively improves the efficiency and accuracy of unmanned charging station resource configuration.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent charging stations for electric vehicles, and more particularly to a method and system for resource allocation in unmanned charging stations based on deep reinforcement learning. BACKGROUND

[0002] With the widespread use of electric vehicles, unmanned charging stations, as important places for charging electric vehicles, have a significant impact on improving charging service quality and meeting charging demand due to the rationality and efficiency of resource allocation.

[0003] However, existing methods for resource allocation in unmanned charging stations are often based on simple rules or heuristic algorithms, making it difficult to cope with complex charging demands and environmental changes, resulting in low resource allocation efficiency and inability to meet different charging needs of users in different regions.

[0004] Prior art document 1 (CN112874369B) proposes a charging and energy supply optimization method and device for a charging management system, which includes obtaining charging station power supply and distribution information, charging facility capacity information, charging terminal output information, and electric vehicle charging demand information, and determining the charging capacity, power supply capacity, and actual charging capacity of the charging station charging facility system. According to the pre-trained deep learning time series prediction algorithm model, the model output result is obtained, and the actual charging capacity of the charging station charging facility system and the charging demand of the electric vehicle to be charged are combined to generate a charging power distribution instruction to distribute power to each charging facility. The charging and energy supply optimization method and device for the charging management system provided by the present application use deep learning to establish a continuously optimized management control model, optimize the energy supply and charging capacity resources of the charging facility, and improve the utilization efficiency.

[0005] Prior art document 1 also has the following technical problems: the charging and energy supply optimization method provided only studies the distribution of charging power based on conventional charging information, resulting in insufficient optimization results in terms of real-time and accuracy, and unable to cope with complex environmental changes. SUMMARY

[0006] To solve the problems in the prior art, the present application provides a method for resource allocation in unmanned charging stations based on deep reinforcement learning, which solves the problem that existing methods for resource allocation in unmanned charging stations are usually based on simple rules or heuristic algorithms, making it difficult to cope with complex charging demands and environmental changes, resulting in low resource allocation efficiency and inability to meet different charging needs of users in different regions.

[0007] The present application adopts the following technical solutions.

[0008] A method for resource allocation in unmanned charging stations based on deep reinforcement learning, comprising the following steps:

[0009] S1: acquiring road information and the number of electric vehicles at the current intersection within a rated time period, and calculating a saturation content of charging demand of the current road;

[0010] S2: setting a time interval, acquiring the saturation content of charging demand within the rated time period according to the time interval, and calculating a group of saturation contents of charging demand in time sequence in real time;

[0011] S3: constructing a charging strategy model based on a deep neural network and a reinforcement learning algorithm, training the charging strategy model through a random time period in a simulation environment, and obtaining a trained charging strategy model;

[0012] S4: taking the saturation content of charging demand as an input of the trained charging strategy model, outputting an optimal value by the model, and allocating charging station resources based on the optimal value output by the model.

[0013] Preferably, in S1, a monitoring sensor is arranged at a preset charging site to acquire road information at the current intersection within a rated time period, electric vehicles are identified from electric green plates of the vehicles, the number of electric vehicles is screened out by an embedded MCU processor, and the saturation content of charging demand of the current road is calculated according to the acquired information.

[0014] Preferably, the road information specifically includes the number of vehicles passing through within the rated time period, the maximum traffic density of the road within the rated time period, the minimum traffic density of the road within the rated time period, the maximum vehicle speed passing through the road within the rated time period, and the minimum vehicle speed passing through the road within the rated time period.

[0015] Preferably, the calculation formula of the saturation content of charging demand is as follows:

[0016]

[0017] wherein, δ is the saturation content of charging demand, α 电 is the number of electric vehicles passing through within the rated time period, α 总 is the total number of vehicles passing through within the rated time period, T is the rated time period, β 大 is the maximum traffic density of electric vehicles on the road within the rated time period, β 小 is the minimum traffic density of electric vehicles on the road within the rated time period, γ 大 is the maximum vehicle speed of electric vehicles passing through the road within the rated time period, γ 小 is the minimum vehicle speed of electric vehicles passing through the road within the rated time period, and dx is an integral operation.

[0018] Preferably, the rated time period is set to 1h, and the time interval is set to 10min.

[0019] Preferably, the deep neural network and reinforcement learning algorithm-based charging strategy model specifically comprises:

[0020] A deep reinforcement learning model for resource allocation of unmanned charging stations is constructed, which includes a deep neural network for simulating resource allocation of charging stations and a reinforcement learning algorithm for learning an optimal resource allocation strategy.

[0021] Preferably, the constructed charging strategy model specifically comprises:

[0022] The state space S={(s1, s2,..., sn)} where si represents a specific state of the charging station, n represents the total number of specific states, and the specific state includes the charging power, charging efficiency, and charging state of each charging pile, as well as the total number of charging piles.

[0023] The action space A={a1, a2,..., am} where aj represents an action of resource allocation for the charging pile, and m represents the total number of actions, and the action of resource allocation for the charging pile at least includes the allocation of power and the configuration number of charging piles.

[0024] The reward function R(s, a)=r1(s, a)+r2(s, a), where r1(s, a) represents the reward of the number of electric vehicles or charging power that meet the charging demand after performing the action a, and r2(s, a) represents the reward of the resource allocation effect of the charging station after performing the action a.

[0025] The Q value function Q(s, a)=f(W*h(s)+b*h(a)), where W and b are the weights and biases of the neural network, h is the neural network layer, f is the activation function (such as sigmoid or relu), s and a are the state and action.

[0026] Loss function: the squared difference of Q value is used as the loss function to optimize the prediction accuracy of Q network. The loss function L=∑[(y-Q(s, a; θ)) 2 ], where y is the true reward, and θ is the parameter of the neural network.

[0027] Preferably, the saturation content of the charging demand is taken as the input quantity of the charging strategy model, and the required configuration number of each charging pile in the charging site and the required configuration power of each charging pile are taken as the output quantity of the charging strategy model for model training. During the training process, the model parameters are continuously updated to enable the model to autonomously learn and optimize the resource allocation strategy according to the charging demand and the station resource situation, and a trained charging strategy model is obtained.

[0028] Preferably, in S4, the optimal value output by the model includes the optimal configuration number of each charging pile in the charging site and the optimal configuration power of each charging pile.

[0029] The application further provides an unmanned charging station resource configuration system based on deep reinforcement learning.

[0030] The information acquisition module is used for acquiring road information and the number of electric vehicles of the current intersection in a rated time period.

[0031] The analysis and calculation module calculates the charging demand saturation content in the rated time period according to the acquired information.

[0032] The model construction and training module is used for constructing a saturation content model, training the policy network through random time periods in a simulation environment, and obtaining the trained saturation content model.

[0033] The resource configuration module takes the charging demand saturation content as the input of the trained saturation content model, outputs an optimal value through the model, and allocates resources of different stations based on the proportional relationship of the optimal value and the inventory of charging resources.

[0034] The application further provides a terminal comprising a processor and a storage medium.

[0035] The storage medium is used for storing instructions.

[0036] The processor is used for operating according to the instructions to execute the steps of the method for configuring resources of an unmanned charging station based on deep reinforcement learning.

[0037] The application further provides a computer readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to realize the steps of the method for configuring resources of an unmanned charging station based on deep reinforcement learning.

[0038] Compared with the prior art, the method for configuring resources of an unmanned charging station based on deep reinforcement learning provided by the application can quantitatively output the charging demand saturation content of the current intersection by detecting information of different road conditions and combining an optimized specific model, the accuracy of the obtained optimization result is higher, and the result is more in line with the actual environment, including acquiring road information of the current intersection through the setting of corresponding monitoring sensors, synchronously screening electric vehicle information, realizing real-time acquisition of information, and improving the applicability of the method; according to the acquired real-time information, the charging demand saturation content is quantitatively acquired, and the accuracy of resource configuration is improved through quantitative analysis and optimized configuration of the saturation content model.

[0039] The intelligent learning and optimization of the resource configuration of the unmanned charging station are realized through a deep reinforcement learning algorithm, the deep reinforcement learning algorithm has strong learning ability and adaptability, and can autonomously learn and optimize the resource configuration strategy according to the charging demand and the resource condition of the station;

[0040] The present application does not need to rely on complex mathematical models and tedious parameter settings, and can effectively improve the efficiency and accuracy of the resource configuration of the unmanned charging station; by collecting and analyzing charging data in real time, the charging demand and the operation condition of the station can be better understood, and more accurate and comprehensive basis can be provided for resource configuration, which can be applicable to unmanned charging stations of different scales and types. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is the overall method flowchart of the resource configuration method of the unmanned charging station based on deep reinforcement learning in the present application;

[0042] Figure 2 is the overall structure diagram of the resource configuration system of the unmanned charging station based on deep reinforcement learning in the present application. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. The embodiments described in the present application are only a part of the embodiments of the present application, but not all the embodiments. Based on the spirit of the present application, other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0044] With reference to Figure 1 The present application provides a resource configuration method of an unmanned charging station based on deep reinforcement learning. It is assumed that there is a chargeable area in which a certain number of charging sites are configured, and a certain number of charging piles are arranged in each charging site. The resource configuration method of the unmanned charging station proposed in the present application can analyze the charging demand of electric vehicles passing through a certain intersection within a rated time, and obtain the number of charging piles required for the corresponding charging site and the configuration power of each charging pile.

[0045] The method comprises the following steps:

[0046] S1: Obtain the road information and the number of electric vehicles of the current intersection within a rated time period, and calculate the charging demand saturation content of the current road;

[0047] The monitoring sensor is arranged at a preset charging site, and real-time road information of a current intersection corresponding to the charging site and electric vehicles passing through the current intersection in a rated time period are acquired, the road information of the electric green-plate vehicle is screened by an embedded MCU processor, and the charging demand saturation content of the current road is acquired;

[0048] Further, the monitoring sensor is a high-precision image video recognition sensor, each charging site corresponds to an intersection, and the acquired road information specifically includes the number of vehicles passing through in the rated time period, the maximum traffic density of the road in the rated time period, the minimum traffic density of the road in the rated time period, the maximum vehicle speed passing through the road in the rated time period, and the minimum vehicle speed passing through the road in the rated time period.

[0049] Preferably, the electric vehicle can be judged by recognizing whether the vehicle license plate is a green plate, and if so, it indicates that the vehicle is an electric vehicle.

[0050] Further, a calculation model of the charging demand saturation content is constructed according to the acquired real-time road information and vehicle information, quantitative analysis and calculation of the charging demand saturation content of each road intersection are realized, and specifically, when the calculation model of the charging demand saturation content is constructed, the following charging demand and environmental factors are considered, including: the proportion of electric vehicles passing through the intersection in the rated time to the total vehicles, the maximum traffic density and the minimum traffic density of the road electric vehicles in the rated time period,

[0051] The charging demand saturation content is calculated in real time through the following calculation model:

[0052]

[0053] Wherein, δ is the charging demand saturation content, α electric is the number of electric vehicles passing through in the rated time period, α total is the total number of vehicles passing through in the rated time period, T is the rated time, β 大 is the maximum traffic density of the road electric vehicles in the rated time period, β 小 is the minimum traffic density of the road electric vehicles in the rated time period, γ big is the maximum vehicle speed of the electric vehicles passing through the road in the rated time period, γ small is the minimum vehicle speed of the electric vehicles passing through the road in the rated time period, 1.614, 0.202, -1.35, 1.36 are constant adjustment items, and dx is an integral operation.

[0054] S2: A time interval is set, the charging demand saturation content in the rated time period is acquired according to the time interval, a group of charging demand saturation contents is acquired in time sequence, and the group of charging demand saturation contents is taken as input information;

[0055] Through the time interval and the rated time period, a set of charging demand saturation content parameters in the rated time period can be obtained.

[0056] As an optional implementation, the rated time period is set to 1h, and the time interval is set to 10min.

[0057] S3: Constructing a charging strategy model based on a deep neural network and a reinforcement learning algorithm, training the charging strategy model through random time periods in a simulation environment, and obtaining a trained charging strategy model;

[0058] Establishing a charging strategy model, training the strategy network through random time periods in a simulation environment, and obtaining a trained charging strategy model specifically includes the following steps:

[0059] S3-1: Constructing a deep reinforcement learning model for resource allocation of unmanned charging stations, the model including a deep neural network for simulating resource allocation of charging stations and a reinforcement learning algorithm for learning an optimal resource allocation strategy;

[0060] Specifically, the following specific charging strategy model details are included:

[0061] 1. State space definition: defining the state space of resource allocation of unmanned charging stations, including a set of charging demand saturation content. Assuming that the state space is S, each state s∈S represents a specific state of resource allocation of the station.

[0062] 2. Action space definition: defining the action space of resource allocation of unmanned charging stations, including actions of resource allocation for each charging station. Assuming that the action space is A, each action a∈A represents resource allocation for a charging station.

[0063] 3. Reward function definition: defining a reward function R(s,a) representing the reward obtained after performing action a in state s. The reward function can be designed according to the charging demand situation and resource allocation effect of the station.

[0064] 4. Q-value function definition: defining a state-action value function Q(s,a) representing the long-term return obtained after performing action a in state s, i.e., the value of selecting action a in state s.

[0065] The Q-value function satisfies the Bellman equation:

[0066] Q(s,a)=R(s,a)+γ*max(Q(s',a'));

[0067] Where s' represents the new state entered after performing action a, and γ is a discount factor affecting the weight of current reward and future reward.

[0068] 5. Deep Q Network: A deep neural network is constructed to approximate the Q-value function. The input of the network is the state s, and the output is the Q-value corresponding to each action a. By training the network to optimize the prediction of Q-value, the optimal resource allocation strategy is achieved.

[0069] S3-2: Input a set of charging demand saturation content into the deep reinforcement learning model for model training. During the training process, the model parameters are continuously updated to enable the model to autonomously learn and optimize the resource allocation strategy based on charging demand and station resource conditions.

[0070] Specifically, the deep Q-learning algorithm is used for model training. By continuously interacting with the environment, taking actions, and obtaining rewards, the network parameters are optimized to learn the optimal resource allocation strategy. Experience replay and target network methods are used during training to improve learning efficiency and stability. The following are all complete technical details and formulas:

[0071] 1. State space representation: State space S = {(s1, s2,..., sn)}, where si represents a specific state of the charging station, including at least the charging power, charging efficiency, charging status of each charging pile, and the total number of charging piles.

[0072] 2. Action space representation: Action space A = {a1, a2,..., an}, where aj represents the action of resource allocation for charging piles, including at least the allocation of power and the configuration number of charging piles.

[0073] 3. Reward function definition: Reward function R(s, a) = r1(s, a) + r2(s, a), where r1(s, a) represents the reward of the number of electric vehicles or charging power that meet the charging demand after executing action a, and r2(s, a) represents the reward of the resource allocation effect of the charging station after executing action a.

[0074] 4. Q-value function definition: Q-value function Q(s, a) = f(W * h(s) + b * h(a)), where W and b are the weights and biases of the neural network, h is the neural network layer, f is the activation function (such as sigmoid or relu), s and a are the state and action.

[0075] 5. Loss function definition: The squared difference of Q-value is used as the loss function to optimize the prediction accuracy of Q-network. Loss function L = ∑[(y - Q(s, a; θ)) 2 ], where y is the true reward, and θ is the parameter of the neural network.

[0076] 6. Learning process: using optimization algorithms such as stochastic gradient descent (SGD) or Adam, the parameters of the neural network are updated to minimize the loss function L. The update formula is: theta = theta - alpha * grad (L), where alpha is the learning rate, and grad is the gradient.

[0077] S4: The saturation content of the charging demand is taken as the input of the trained charging strategy model, the model outputs the optimal value, and the allocation of the charging station resources is based on the optimal value output by the model.

[0078] After the model training is completed, the trained deep Q network is applied to the resource allocation of the unmanned charging station, and the optimal resource allocation action is selected according to the real-time state of the station to maximize the long-term return.

[0079] Specifically, a group of charging demand saturation contents in a rated time period is collected and obtained, which is input into the trained deep reinforcement learning model of the resource allocation of the unmanned charging station, and the optimal value output by the model corresponding to each charging station includes the required number of charging piles and the optimal configuration power of each charging pile corresponding to each charging station.

[0080] Further, the required configuration number of the charging pile and the required configuration power of each charging pile output by the model are used to complete the resource allocation, and the electric vehicles are guided to the charging piles of the corresponding charging station for charging.

[0081] The present application obtains the required charging resource data of each road station in the whole region, and then distributes the total charging resources in the whole region according to the calculated data, so as to maximize the utilization of the total resources.

[0082] As shown in Figure 2 The present application also provides an unmanned charging station resource allocation system based on deep reinforcement learning, which is used to realize the above-mentioned unmanned charging station resource allocation method based on deep reinforcement learning. The system specifically comprises: an information acquisition module, an analysis and calculation module, a model construction and training module, and a resource allocation module.

[0083] The information acquisition module is used to acquire the road information and the number of electric vehicles at the current intersection in a rated time period.

[0084] The analysis and calculation module calculates the saturation content of the charging demand in the rated time period according to the acquired information.

[0085] The model construction and training module is used to construct the saturation content model, train the strategy network through random time periods in the simulation environment, and obtain the trained saturation content model.

[0086] The resource configuration module takes the charging demand saturation content as the input of the post-training saturation content model, outputs the optimal value through the model, and allocates resources of different stations based on the proportional relationship of the optimal value and the stock of charging resources.

[0087] In order to verify the technical effect of the present application, the same charging scene is simulated by using the optimization strategy proposed in the present application and the original strategy of the traditional method respectively, and the resource configuration result data obtained is shown in Table 1 as follows:

[0088] Table 1: Part of the simulation data table

[0089]

[0090] It can be seen that the optimization strategy proposed in the present application can shorten the charging waiting time and complete the charging in a shorter time, and improve the charging efficiency.

[0091] The beneficial effects of the present application are that, compared with the prior art, the present application does not need to rely on complex mathematical models and tedious parameter settings, and can effectively improve the efficiency and accuracy of resource configuration of unmanned charging stations; by collecting and analyzing charging data in real time, the charging demand and station operation condition can be better understood, and more accurate and comprehensive basis can be provided for resource configuration, which can be applicable to unmanned charging stations of different scales and types.

[0092] The present disclosure can be a system, a method and / or a computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions loaded thereon, which are used to enable a processor to implement various aspects of the present disclosure.

[0093] Computer readable storage media can be tangible storage media which can retain and store instructions for use by an instruction execution device. Computer readable storage media can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer readable storage media include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0094] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0095] Computer readable program instructions for carrying out operations of the present disclosure can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0096] Finally, it should be noted that the above-mentioned embodiments are merely used to illustrate the technical solutions of the present application, but not to limit it. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that the specific embodiments of the present application can be modified or replaced, and any modification or replacement without departing from the spirit and scope of the present application should be covered in the protection scope of the claims of the present application.

Claims

1. A resource allocation method for unmanned charging stations based on deep reinforcement learning, characterized in that, Includes the following steps: S1: Obtain road information and number of electric vehicles at the current intersection within the specified time period, and calculate the saturation content of charging demand on the current road. In S1, a monitoring sensor is set up at a preset charging site to obtain road information of the current intersection within a specified time period. Electric vehicles are identified by their green license plates. The number of electric vehicles is filtered out by the embedded MCU processor, and the saturation content of the current road's charging demand is calculated based on the obtained information. The road information specifically includes the number of vehicles passing through within a specified time period, the maximum traffic density of the road within a specified time period, the minimum traffic density of the road within a specified time period, the maximum vehicle speed passing through the road within a specified time period, and the minimum vehicle speed passing through the road within a specified time period. The formula for calculating the saturation content of charging demand is as follows: Where δ represents the saturation content of charging demand, and α 电 α represents the number of electric vehicles passing through within a specified time period. 总 β represents the total number of vehicles passing through within the specified time period, where T is the specified time period. 大 β represents the maximum traffic density of electric vehicles on the road within the rated time period. 小 γ represents the minimum traffic density of electric vehicles on the road within a specified time period. 大 γ is the maximum vehicle speed of an electric vehicle passing through the road within a rated time period. 小 dx represents the minimum vehicle speed of an electric vehicle passing through the road within the rated time period, and dx is the integral operation. S2: Set the time interval, obtain the saturated charging demand content within the rated time period according to the time interval, and calculate a set of saturated charging demand content in real time in chronological order. S3: Construct a charging strategy model based on deep neural networks and reinforcement learning algorithms, and train the charging strategy model through random time periods in the simulation environment to obtain the trained charging strategy model. S4. The charging demand saturation content is used as the input to the trained charging strategy model. The model outputs the optimal value, and the charging station resources are allocated based on the optimal value output by the model. In S4, the optimal values ​​output by the model include the optimal number of charging piles for each charging station in the charging area and the optimal power of each charging pile.

2. The resource allocation method for unmanned charging stations based on deep reinforcement learning according to claim 1, characterized in that, The rated time period is set to 1 hour, and the time interval is set to 10 minutes.

3. The method for resource allocation of unmanned charging stations based on deep reinforcement learning according to claim 1, characterized in that, The charging strategy model constructed based on deep neural networks and reinforcement learning algorithms specifically includes: A deep reinforcement learning model for resource allocation of unmanned charging stations is constructed. The model includes a deep neural network for simulating resource allocation of charging stations and a reinforcement learning algorithm for learning the optimal resource allocation strategy.

4. The resource allocation method for unmanned charging stations based on deep reinforcement learning according to claim 1, characterized in that, The constructed charging strategy model specifically includes: The state space S = {(s1,s2,...,sn)}, where si represents a specific state of the charging station, and n represents the total number of specific states. Specific states include the charging power, charging efficiency, charging status of each charging pile, and the total number of charging piles. Action space A = {a1, a2, ..., am}, where aj represents the action of allocating resources to charging piles, and m represents the total number of actions. The actions of allocating resources to charging piles include at least the allocation of power and the configuration of the number of charging piles. The reward function R(s,a) = r1(s,a) + r2(s,a) is given, where r1(s,a) represents the reward for the number of electric vehicles that meet the charging demand or the charging power after performing action a, and r2(s,a) represents the reward for the effect of charging station resource allocation after performing action a. The Q-value function Q(s,a) = f(W*h(s) + b*h(a)), where W and b are the weights and biases of the neural network, h is the neural network layer, f is the activation function (either sigmoid or ReLU), and s and a are the state and action. Loss function: The squared difference of Q values ​​is used as the loss function to optimize the prediction accuracy of the Q network; Loss function L=∑[(yQ(s,a;θ)) 2 ], where y is the actual reward and θ is the parameter of the neural network.

5. The resource allocation method for unmanned charging stations based on deep reinforcement learning according to claim 4, characterized in that, The saturation content of charging demand is used as the input of the charging strategy model, and the required number of charging piles and the required power of each charging pile in each charging station in the charging site are used as the output of the charging strategy model for model training. During the training process, the model parameters are continuously updated so that the model can learn and optimize the resource allocation strategy autonomously according to the charging demand and the site resource conditions, thus obtaining the trained charging strategy model.

6. A resource allocation system for unmanned charging stations based on deep reinforcement learning, utilizing the resource allocation method for unmanned charging stations based on deep reinforcement learning as described in any one of claims 1-5, characterized in that, include: Information acquisition module, analysis and calculation module, model building and training module, resource allocation module; The information acquisition module is used to acquire road information and the number of electric vehicles at the current intersection within a specified time period; Specifically, monitoring sensors are set up at the preset charging sites to obtain road information at the current intersection within a specified time period. Electric vehicles are identified by their green license plates, and the number of electric vehicles is filtered out by the embedded MCU processor. The road information specifically includes the number of vehicles passing through within a specified time period, the maximum traffic density of the road within a specified time period, the minimum traffic density of the road within a specified time period, the maximum vehicle speed passing through the road within a specified time period, and the minimum vehicle speed passing through the road within a specified time period. The analysis and calculation module calculates the saturation content of charging demand within the rated time period based on the acquired information. The formula for calculating the saturation content of charging demand is as follows: Where δ represents the saturation content of charging demand, and α 电 α represents the number of electric vehicles passing through within a specified time period. 总 β represents the total number of vehicles passing through within the specified time period, where T is the specified time period. 大 β represents the maximum traffic density of electric vehicles on the road within the rated time period. 小 γ represents the minimum traffic density of electric vehicles on the road within a specified time period. 大 γ is the maximum vehicle speed of an electric vehicle passing through the road within a rated time period. 小 dx represents the minimum vehicle speed of an electric vehicle passing through the road within the rated time period, and dx is the integral operation. The model building and training module is used to build a saturated content model. The policy network is trained through random time periods in the simulation environment to obtain the trained saturated content model. The resource allocation module takes the saturation content of charging demand as input to the saturation content model after training. The model outputs the optimal value, and the allocation of resources for different charging stations is based on the proportional relationship of the optimal value and the stock of charging resources. The optimal value output by the model includes the optimal configuration number of charging piles for each charging station in the charging site and the optimal configuration power of each charging pile.

7. A terminal, comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the deep reinforcement learning-based unmanned charging station resource allocation method according to any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the resource allocation method for unmanned charging stations based on deep reinforcement learning as described in any one of claims 1-5.

Images