Charging pile dynamic load balancing scheduling system and method based on reinforcement learning

Abstract

The invention discloses a charging pile dynamic load balancing scheduling system based on reinforcement learning, and the system comprises a data collection module which comprises 5G communication units installed at the charging piles, and is used for obtaining the power of the charging piles, vehicle battery parameters and power grid load data in real time; the reinforcement learning decision module maps the data in the data acquisition module into a state vector through a feature extraction network, and generates a power distribution strategy by adopting a PPO + DQN hybrid algorithm; the communication protocol module realizes state synchronization and control instruction transmission between the charging piles based on an OCPP2.0 standard; the execution control module converts the control instruction into a PWM control signal, and adjusts the output power of each charging pile in real time; through the reinforcement learning decision module, the system can analyze the charging pile power, the vehicle battery parameters and the power grid load data in real time, generate an accurate power distribution strategy, ensure accurate matching of the charging pile power and the electric vehicle demand power, and thus significantly improve the charging efficiency.

Description

Technical Field

[0001] The present invention relates to a dynamic load balancing scheduling system and method for charging piles, and particularly to a dynamic load balancing scheduling system and method for charging piles based on reinforcement learning. Background Art

[0002] With the continuous increase in the market share of new energy electric vehicles, charging stations have configured a certain number of charging piles to meet the charging needs of various types of electric vehicles. These charging piles have different output powers and can meet the charging requirements of electric vehicles with different powers. The charging duration is not only determined by the power of the charging pile, but also jointly affected by the remaining power of the power battery and the power matching degree between the electric vehicle and the charging pile. In fact, the power of the charging pile is not the larger the better: if its power far exceeds the requirements of the electric vehicle, it will cause waste of resources; if the power is much lower than the requirements of the electric vehicle, it will significantly extend the charging time and reduce the charging efficiency. Therefore, only when the power of the charging pile is accurately matched with the required power of the electric vehicle can the best charging effect be achieved.

[0003] However, the existing charging pile scheduling systems mainly rely on static power distribution or rule-based heuristic algorithms and are difficult to adapt to the dynamic fluctuations of the grid load, the differentiation of vehicle charging demands, and the changes in the operating states of equipment.

[0004] Therefore, there is an urgent need for a dynamic load balancing scheduling system and method for charging piles based on reinforcement learning to solve the technical problems existing in the above-mentioned prior art. Summary of the Invention

[0005] The present invention overcomes the deficiencies of the prior art and provides a dynamic load balancing scheduling system and method for charging piles based on reinforcement learning.

[0006] To achieve the above object, the technical solution adopted by the present invention is: a dynamic load balancing scheduling system for charging piles based on reinforcement learning, including: a data acquisition module, a reinforcement learning decision module, a communication protocol module, and an execution control module;

[0007] The data acquisition module includes a 5G communication unit installed at each charging pile, which is used to obtain the power of the charging pile, the vehicle battery parameters, and the grid load data in real time, and is also used for data interaction between the charging piles;

[0008] The reinforcement learning decision module maps the data in the data acquisition module into a state vector through a feature extraction network and adopts a hybrid algorithm of PPO+DQN to generate a power distribution strategy; at the same time, a multi-objective reward mechanism is constructed according to the load balancing degree, charging efficiency, and equipment loss rate of the charging pile;

[0009] The communication protocol module is based on the OCPP2.0 standard to realize the state synchronization and control instruction transmission between the charging piles;

[0010] The execution control module converts the control instruction into a PWM control signal to adjust the output power of each charging pile in real time.

[0011] In a preferred embodiment of the present invention, the power distribution strategy generated by the reinforcement learning decision module includes:

[0012] S1. Embed an LSTM time series modeling layer in the feature extraction network, construct a sliding sequence with a time window length of T for the charging pile power data and the grid load data, and extract a time correlation vector containing dynamic fluctuation features;

[0013] S2. Use the attention mechanism to perform priority weighting on the vehicle battery parameters, and generate an attention weight coefficient that is 30% - 50% higher than the standard value for vehicles with urgent charging requirements;

[0014] S3. Introduce a constraint optimization layer in the PPO algorithm, and convert the grid total load threshold and the single pile maximum power limit into hard constraint conditions of the policy network through the Lagrange multiplier method;

[0015] S4. Dynamically correct the device loss rate calculation term in the reward function based on the historical working cycle times and real-time temperature rise data of the charging pile. When a charging pile is detected to be offline or a new node is added, start the topology adaptive mechanism, reconstruct the adjacency relationship matrix between charging piles through a graph neural network, and synchronously update the state vector dimension.

[0016] In a preferred embodiment of the present invention, the reward mechanism further includes:

[0017] D1. Establish a dynamic weight adjustment model, predict the change trend of the grid load data in the next 3 - 5 minutes through a long short-term memory network, and update the weight coefficient of the dynamic weight adjustment model in real time according to the prediction result;

[0018] D2. Introduce a charging urgency factor, perform normalization processing based on the demand for the remaining charging time of the vehicle and the decline rate of the vehicle battery parameters, and generate a priority reward addition term;

[0019] D3. Construct a power distribution smoothness penalty term, and impose a quadratic penalty coefficient on charging piles whose power adjustment amplitude in adjacent cycles exceeds 12% - 15%;

[0020] D4. When the charging pile is in the discharging state, impose a linear decay reward on the total power consumption of the charging pile, and set a dynamic threshold for the load balance degree. When the grid load data of the charging pile is detected to be lower than the dynamic threshold of the load balance degree, automatically switch to the charging efficiency priority mode.

[0021] In a preferred embodiment of the present invention, the implementation steps of the communication protocol module include:

[0022] Z1. Dynamically adjust the data sampling period based on the fluctuation frequency of grid load data. When the volatility exceeds the set threshold, switch to the millisecond-level high-frequency sampling mode, and transmit key status data through the URLLC channel of the 5G communication unit.

[0023] Z2. Construct a hierarchical verification protocol, adopt a dual verification mechanism of CRC32 + hash chain for power distribution instructions, and embed a timing mark in the packet header to prevent out-of-order execution of instructions.

[0024] Z3. Establish a multi-level cache queue, allocate transmission bandwidth according to the priority of charging piles, and enable a preemptive data transmission channel for charging piles with a load rate exceeding 80%.

[0025] Z4. Expand the dynamic load balancing instruction set in the OCPP2.0 standard, add power flexible adjustment instruction codes and topology status synchronization instruction codes, and support online reconfiguration of the maximum power deviation value parameter.

[0026] In a preferred embodiment of the present invention, the execution control module includes a multi-level security regulation mechanism, specifically including:

[0027] T1. Adaptively adjust the duty cycle of the PWM signal according to the real-time power deviation value of the charging pile. When the power deviation is 8% - 10%, enable the piecewise linear interpolation algorithm to smoothly transition to the target power.

[0028] T2. Deploy an over-temperature and over-current dual-threshold trigger mechanism. When it is detected that the temperature of the charging pile exceeds 70°C - 75°C or the current volatility exceeds the limit for 3 consecutive cycles, automatically switch to the safe derating mode.

[0029] T3. Collect the actual output power of the charging pile in real time, use a PID controller to perform real-time compensation on the PWM control signal, and incorporate the calculated compensation amount into the power distribution strategy of the reinforcement learning decision module.

[0030] In a preferred embodiment of the present invention, in step S1, the LSTM time series modeling layer in the feature extraction network adopts a bidirectional LSTM structure, which is used to capture the past and future information of the charging pile power data and the grid load data simultaneously, and then extract the time correlation vector containing dynamic fluctuation features.

[0031] In a preferred embodiment of the present invention, in step S3, the constraint optimization layer adopts the adaptive Lagrange multiplier method, and dynamically adjusts the Lagrange multiplier according to the real-time changes of the total grid load threshold and the single-pile maximum power limit, so that the output value of the power distribution strategy always satisfies the hard constraint conditions.

[0032] In a preferred embodiment of the present invention, in step D1, the long short-term memory network is optimized using a gated recurrent unit to reduce the parameters of the dynamic weight adjustment model. At the same time, combining the seasonal characteristics of historical power grid load data, the change trend of power grid load data in the next 3-5 minutes is predicted.

[0033] In a preferred embodiment of the present invention, a dynamic load balancing scheduling method for charging piles based on reinforcement learning is provided, which is applied to the above-mentioned dynamic load balancing scheduling system for charging piles based on reinforcement learning, and includes the following steps:

[0034] A1. Real-time obtain the charging pile power, vehicle battery parameters and power grid load data through the 5G communication unit;

[0035] A2. Extract dynamic feature vectors through a bidirectional LSTM time series network, combine the attention mechanism to perform weighted processing on vehicles with urgent charging requirements, adopt a PPO+DQN hybrid algorithm, allocate power that meets the power grid threshold and single pile limit through an adaptive constraint optimization layer, and dynamically correct the charging pile loss reward based on historical working data and temperature rise;

[0036] A3. Dynamically adjust the sampling period, transmit key data through the URLLC channel, and adopt a dual verification instruction of CRC32+hash chain to ensure communication of high-load charging piles;

[0037] A4. Adaptively adjust the PWM control signal, trigger safety derating based on over-temperature and over-current events, and perform real-time compensation for the output power. The compensation amount is fed back to the decision-making module to optimize the strategy.

[0038] In a preferred embodiment of the present invention, in step A2, when the remaining battery life of the vehicle with urgent charging requirements is lower than 40-50 km, the attention weight coefficient is increased to 1.5-2 times the standard value.

[0039] The present invention solves the defects existing in the background technology, and the present invention has the following beneficial effects:

[0040] (1) Through the reinforcement learning decision-making module, the system can analyze the charging pile power, vehicle battery parameters and power grid load data in real time, generate accurate power distribution strategies, ensure the accurate matching of the charging pile power and the electric vehicle demand power, thereby significantly improving the charging efficiency and reducing the charging time.

[0041] (2) The system adopts a hybrid algorithm of PPO + DQN, combined with a multi-objective reward mechanism, which can dynamically adjust the power output of the charging piles, achieve load balancing, avoid waste of charging pile resources, and ensure the stable operation of the power grid. Moreover, by introducing a constraint optimization layer, the total power grid load threshold and the maximum power limit of a single pile are transformed into hard constraint conditions for the policy network through the Lagrange multiplier method, ensuring that the power distribution strategy always meets the constraint requirements of the power grid and the charging piles.

[0042] (3) The communication protocol module is based on the OCPP2.0 standard to achieve state synchronization and control instruction transmission among charging piles, ensuring the real-time and accuracy of data. Meanwhile, when the power grid load data fluctuates greatly, it switches to the high-frequency sampling mode and transmits key state data through the URLLC channel of the 5G communication unit to improve the efficiency and reliability of data transmission.

[0043] (4) The long short-term memory network is used to predict the change trend of future power grid load data, and the weight coefficient of the reward mechanism is dynamically adjusted, enabling the system to respond in advance and optimize the power distribution strategy. Moreover, a charging urgency factor and a power distribution smoothness penalty term are introduced to further refine the reward mechanism and improve the charging urgency and power distribution smoothness. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings;

[0045] Figure 1 It is the system flowchart of the preferred embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0047] Many specific details are set forth in the following description in order to fully understand the present invention, but the present invention can also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0048] Such as Figure 1As shown, a dynamic load balancing scheduling system and method for charging piles based on reinforcement learning, including: a data acquisition module, a reinforcement learning decision-making module, a communication protocol module, and an execution control module.

[0049] The data acquisition module includes 5G communication units installed at each charging pile, which are used to obtain the charging pile power P i (t) in real time, where i represents the i-th charging pile and t represents time; the vehicle battery parameter B j (t), where j represents the j-th vehicle; the grid load data is represented as L(t), and it is also used for data interaction between charging piles.

[0050] The reinforcement learning decision-making module maps the data in the data acquisition module into a state vector through a feature extraction network, and uses a hybrid algorithm of PPO+DQN to generate a power distribution strategy. At the same time, a multi-objective reward mechanism is constructed based on the load balance degree, charging efficiency, and equipment loss rate of the charging piles.

[0051] Further, the power distribution strategy generated by the reinforcement learning decision-making module includes:

[0052] S1. Embed an LSTM time series modeling layer in the feature extraction network, construct a sliding sequence with a time window length of T for the charging pile power data and the grid load data, and extract a time correlation vector containing dynamic fluctuation features.

[0053] Further, in step S1, the LSTM time series modeling layer in the feature extraction network adopts a bidirectional LSTM structure, which is used to capture the past and future information of the charging pile power data and the grid load data at the same time, and then extract a time correlation vector containing dynamic fluctuation features.

[0054] To further explain step S1, let X(t) be the input sequence, X(t) = [P1(t), P2(t),..., P n (t), L(t)], and the time correlation vector extracted after passing through the bidirectional LSTM is h t , and its calculation formula is:

[0055] Forward LSTM:

[0056] Backward LSTM:

[0057] Final time correlation vector:

[0058] S2. Use the attention mechanism to perform priority weighting on the vehicle battery parameters. For vehicles with urgent charging requirements, let the standard attention weight coefficient be w0, and generate an attention weight coefficient w j that is 30% - 50% higher than the standard value.

[0059] When the remaining driving range of the vehicle is less than 10 - 50 km:

[0060]

[0061] S3. Introduce a constraint optimization layer in the PPO algorithm, and transform the total grid load threshold and the maximum power limit of a single pile into hard constraint conditions of the policy network through the Lagrange multiplier method.

[0062] Further, in step S3, the constraint optimization layer adopts the adaptive Lagrange multiplier method, and dynamically adjusts the Lagrange multiplier according to the real-time changes of the total grid load threshold L max and the maximum power limit of a single pile so that the output value of the power distribution strategy always satisfies the hard constraint conditions.

[0063] Let the power distribution strategy of the output value of the power distribution strategy be π(a|s), and the Lagrangian function is:

[0064]

[0065] where θ represents the network parameters of the power distribution strategy; λ1 and λ2 are both Lagrange multipliers; A(a, s) is the advantage function; the adaptive Lagrange multiplier method dynamically adjusts λ1 and λ2 according to the real-time changes of the total grid load threshold and the maximum power limit of a single pile.

[0066] S4. Based on the historical working cycle number N of the charging pile i and the real-time temperature rise data ΔT i (t), dynamically correct the calculation term of the equipment loss rate in the reward function. When the charging pile is detected to be offline or a new node is added, start the topology adaptive mechanism, and reconstruct the adjacency relationship matrix between the charging piles through the graph neural network and synchronously update the dimension of the state vector.

[0067] Let the original equipment loss rate be The corrected equipment loss is r i :

[0068]

[0069] where f(N i , ΔT i (t)) is a function of N i and ΔT i (t).

[0070] Even further, the reward mechanism further includes:

[0071] D1. Establish a dynamic weight adjustment model, predict the change trend of grid load data in the next 3 - 5 minutes through a long short - term memory network, and update the weight coefficient of the dynamic weight adjustment model in real time according to the prediction result.

[0072] Further, in step D1, the long short - term memory network is optimized using gated recurrent units to reduce the parameters of the dynamic weight adjustment model. At the same time, combined with the seasonal characteristics of historical grid load data, predict the change trend of grid load data in the next 3 - 5 minutes.

[0073] Let the predicted grid load data be The weight coefficient of the dynamic weight adjustment model is α(t), then:

[0074]

[0075] where m is the number of time steps of historical data; g(·) is a function that updates the weight coefficient according to the prediction result.

[0076] D2. Introduce a charging urgency factor, and perform normalization processing based on the demand for the remaining charging time of the vehicle and the degradation rate of the vehicle battery parameters to generate a priority reward addition term R priority .

[0077]

[0078] where k is a proportionality coefficient.

[0079] D3. Construct a power distribution smoothness penalty term, and impose a quadratic penalty coefficient on the charging piles whose power adjustment amplitude in adjacent cycles exceeds 12% - 15%.

[0080] Let the power in adjacent cycles t and t - 1 be P i (t) and P i (t - 1), and the power adjustment amplitude is:

[0081]

[0082] The penalty term R penalty is:

[0083] where c is the penalty coefficient.

[0084] D4. When the charging pile is in the discharging state, impose a linear decay reward on the total power consumption of the charging pile, and set a dynamic threshold for the load balance degree. When it is detected that the grid load data of the charging pile is lower than the dynamic threshold of the load balance degree, automatically switch to the charging efficiency priority mode.

[0085] Let the total power consumption of the charging pile be Ptotal If the linear decay reward coefficient is β, then the reward R discharge is:

[0086] R discharge = β·P total

[0087] When the grid load data of the charging pile is detected to be lower than the dynamic threshold L of the load balance degree threshold automatically switch to the charging efficiency priority mode.

[0088] The communication protocol module is based on the OCPP2.0 standard to achieve state synchronization and control instruction transmission between charging piles.

[0089] Furthermore, the implementation steps of the communication protocol module include:

[0090] Z1. Dynamically adjust the data sampling period Δt based on the fluctuation frequency of the grid load data s , and when the volatility exceeds the set threshold, switch to the millisecond-level high-frequency sampling mode and transmit key state data through the URLLC channel of the 5G communication unit.

[0091] Let the volatility threshold then:

[0092] where is the millisecond-level high-frequency sampling period; is the low-frequency sampling period.

[0093] Z2. Construct a hierarchical verification protocol, adopt a double verification mechanism of CRC32 + hash chain for the power distribution instruction, and embed a timing mark in the packet header to prevent out-of-order execution of the instruction.

[0094] Z3. Establish a multi-level cache queue, allocate transmission bandwidth according to the charging pile priority, and enable a preemptive data transmission channel for charging piles with a load rate exceeding 80%.

[0095] Z4. Expand the dynamic load balancing instruction set in the OCPP2.0 standard, add power flexible adjustment instruction codes and topology state synchronization instruction codes, and support online reconfiguration of the maximum power deviation value parameter.

[0096] The execution control module converts the control instruction into a PWM control signal to adjust the output power of each charging pile in real time.

[0097] Furthermore, the execution control module includes a multi-level safety regulation mechanism, specifically including:

[0098] T1. Adaptively adjust the duty cycle D of the PWM signal according to the real-time power deviation value of the charging pile i(t), when the power deviation is large, 8% - 10%, the piecewise linear interpolation algorithm is enabled to smoothly transition to the target power. Let the target power be Then:

[0099]

[0100] Among them, is the initial duty cycle; LinearInterpolation(·) is the piecewise linear interpolation function.

[0101] T2. Deploy an over-temperature and over-current dual-threshold trigger mechanism. When it is detected that the charging pile temperature exceeds 70°C - 75°C or the current volatility exceeds the limit for 3 consecutive cycles, it automatically switches to the safe derating mode.

[0102] Let the charging pile temperature be T i (t), and the current volatility be ρ I , then:

[0103]

[0104] T3. Real-time collect the actual output power of the charging pile, use a PID controller to perform real-time compensation on the PWM control signal, and incorporate the compensation amount calculation into the power distribution strategy of the reinforcement learning decision module.

[0105] Let the compensation amount be ΔD i (t), then:

[0106]

[0107] Among them, K p 、K i 、K d are the proportional, integral, and differential coefficients respectively; the compensation amount calculation is incorporated into the power distribution strategy of the reinforcement learning decision module.

[0108] Based on the above charging pile dynamic load balancing scheduling system, a charging pile dynamic load balancing scheduling method based on reinforcement learning is proposed, including the following steps:

[0109] A1. Real-time obtain the charging pile power, vehicle battery parameters, and grid load data through the 5G communication unit.

[0110] A2. Extract dynamic feature vectors through a bidirectional LSTM time series network, perform weighted processing on vehicles with urgent charging requirements in combination with the attention mechanism, use the PPO+DQN hybrid algorithm, allocate power that meets the grid threshold and single pile limit through the adaptive constraint optimization layer, and dynamically correct the charging pile loss reward based on historical working data and temperature rise.

[0111] Further, in step A2, when the remaining cruising range of the vehicle with urgent charging demand is less than 40 - 50 km, the attention weight coefficient is increased to 1.5 - 2 times of the standard value.

[0112] A3. Dynamically adjust the sampling period, transmit key data through the URLLC channel, and adopt a dual verification instruction of CRC32 + hash chain to ensure the communication of high-load charging piles.

[0113] A4. Adaptively adjust the PWM control signal, trigger safety derating based on over-temperature and over-current events, and perform real-time compensation for the output power. The compensation amount is fed back to the decision-making module to optimize the strategy.

[0114] Based on the ideal embodiments of the present invention as an inspiration, through the above description, relevant personnel can completely make various changes and modifications without departing from the technical idea of this invention. The technical scope of this invention is not limited to the content in the specification, and the technical scope must be determined according to the scope of the claims.

Claims

1. A dynamic load balancing scheduling system for charging piles based on reinforcement learning, characterized in that It includes: a data acquisition module, a reinforcement learning decision-making module, a communication protocol module, and an execution control module; The data acquisition module includes 5G communication units installed at each charging pile, which are used to obtain the charging pile power, vehicle battery parameters, and grid load data in real time, and are also used for data interaction between charging piles; The reinforcement learning decision-making module maps the data in the data acquisition module into a state vector through a feature extraction network, and uses a hybrid algorithm of PPO+DQN to generate a power distribution strategy; at the same time, a multi-objective reward mechanism is constructed based on the load balance degree, charging efficiency, and equipment loss rate of the charging piles; The communication protocol module is based on the OCPP2.0 standard to achieve state synchronization and control instruction transmission between charging piles; The execution control module converts the control instruction into a PWM control signal to adjust the output power of each charging pile in real time.

2. The dynamic load balancing scheduling system for charging piles based on reinforcement learning according to claim 1, wherein: The power distribution strategy generated by the reinforcement learning decision-making module includes: S1. Embed an LSTM time series modeling layer in the feature extraction network, construct a sliding sequence with a time window length of T for the charging pile power data and grid load data, and extract a time correlation vector containing dynamic fluctuation characteristics; S2. Use an attention mechanism to perform priority weighting on the vehicle battery parameters, and generate an attention weight coefficient 30% - 50% higher than the standard value for vehicles with urgent charging needs; S3. Introduce a constraint optimization layer in the PPO algorithm, and convert the grid total load threshold and single-pile maximum power limit into hard constraint conditions of the policy network through the Lagrangian multiplier method; S4. Dynamically correct the equipment loss rate calculation term in the reward function based on the historical working cycle times and real-time temperature rise data of the charging piles. When a charging pile is detected to be offline or a new node is added, start a topology adaptive mechanism, reconstruct the adjacency relationship matrix between charging piles through a graph neural network, and synchronously update the state vector dimension.

3. The dynamic load balancing scheduling system for charging piles based on reinforcement learning according to claim 2, wherein: The reward mechanism further includes: D1. Establish a dynamic weight adjustment model, predict the change trend of grid load data in the next 3 - 5 minutes through a long short-term memory network, and update the weight coefficient of the dynamic weight adjustment model in real time according to the prediction result; D2. Introduce a charging urgency factor, perform normalization processing based on the demand for the remaining charging time of the vehicle and the decline rate of the vehicle battery parameters, and generate a priority reward addition term; D3. Construct a power distribution smoothness penalty term, and apply a quadratic penalty coefficient to the charging piles whose power adjustment amplitude in adjacent cycles exceeds 12% - 15%; D4. When the charging pile is in the discharging state, apply a linear decay reward to the total power consumption of the charging pile, and set a dynamic threshold for the load balance degree. When the grid load data of the charging pile is detected to be lower than the dynamic threshold of the load balance degree, automatically switch to the charging efficiency priority mode.

4. The dynamic load balancing scheduling system for charging piles based on reinforcement learning according to claim 1, wherein: The implementation steps of the communication protocol module include: Z1. Dynamically adjust the data sampling period based on the grid load data fluctuation frequency. When the volatility exceeds the set threshold, switch to the millisecond-level high-frequency sampling mode, and transmit key state data through the URLLC channel of the 5G communication unit; Z2. Construct a hierarchical verification protocol, adopt a dual verification mechanism of CRC32 + hash chain for power distribution instructions, and embed a timing mark in the packet header to prevent out-of-order execution of instructions; Z3. Establish a multi-level cache queue, allocate transmission bandwidth according to the priority of charging piles, and enable a preemptive data transmission channel for charging piles with a load rate exceeding 80%; Z4. Expand the dynamic load balancing instruction set in the OCPP2.0 standard, add power flexible adjustment instruction codes and topology state synchronization instruction codes, and support online reconfiguration of the maximum power deviation value parameter.

5. The dynamic load balancing scheduling system for charging piles based on reinforcement learning according to claim 1, wherein: The execution control module includes a multi-level security regulation mechanism, specifically including: T1. Adaptively adjust the duty cycle of the PWM signal according to the real-time power deviation value of the charging pile. When the power deviation is large by 8% - 10%, enable a piecewise linear interpolation algorithm to smoothly transition to the target power; T2. Deploy an over-temperature and over-current dual-threshold trigger mechanism. When it is detected that the temperature of the charging pile exceeds 70°C - 75°C or the current volatility exceeds the limit for 3 consecutive cycles, automatically switch to the safe derating mode; T3. Collect the actual output power of the charging pile in real time, use a PID controller to perform real-time compensation on the PWM control signal, and incorporate the compensation amount calculation into the power distribution strategy of the reinforcement learning decision module.

6. The dynamic load balancing scheduling system for charging piles based on reinforcement learning according to claim 2, characterized in that: In step S1, the LSTM timing modeling layer in the feature extraction network adopts a bidirectional LSTM structure, which is used to capture the past and future information of the charging pile power data and the grid load data simultaneously, and then extract a time correlation vector containing dynamic fluctuation features.

7. The dynamic load balancing scheduling system for charging piles based on reinforcement learning according to claim 2, wherein: In step S3, the constraint optimization layer adopts the adaptive Lagrange multiplier method. According to the real-time changes of the total grid load threshold and the maximum power limit of a single pile, dynamically adjust the Lagrange multiplier so that the output value of the power distribution strategy always satisfies the hard constraint conditions.

8. The dynamic load balancing scheduling system for charging piles based on reinforcement learning according to claim 3, characterized in that: In step D1, the long short-term memory network is optimized using gated recurrent units to reduce the parameters of the dynamic weight adjustment model. At the same time, combined with the seasonal characteristics of historical grid load data, predict the change trend of grid load data in the next 3 - 5 minutes.

9. The dynamic load balancing scheduling method for charging piles based on reinforcement learning is applied to the dynamic load balancing scheduling system for charging piles based on reinforcement learning according to any one of the above claims 1-8, and is characterized in that, It includes the following steps: A1. Real-time obtain the charging pile power, vehicle battery parameters, and grid load data through the 5G communication unit; A2. Extract dynamic feature vectors through a bidirectional LSTM timing network, perform weighted processing on vehicles with urgent charging requirements by combining the attention mechanism, adopt a PPO + DQN hybrid algorithm, allocate power that meets the grid threshold and single-pile limit through the adaptive constraint optimization layer, and dynamically correct the charging pile loss reward based on historical working data and temperature rise; A3. Dynamically adjust the sampling period, transmit key data through the URLLC channel, and adopt a dual verification instruction of CRC32 + hash chain to ensure communication for high-load charging piles; A4. Adaptively adjust the PWM control signal, trigger safe derating based on over-temperature and over-current events, and perform real-time compensation for the output power. The compensation amount is fed back to the decision module to optimize the strategy.

10. The method for dynamic load balancing scheduling of charging piles based on reinforcement learning according to claim 9, wherein: In step A2, when the remaining battery life of a vehicle with urgent charging requirements is less than 40 - 50 km, the attention weight coefficient is increased to 1.5 - 2 times the standard value.

Images