A trade-off between time and economic electric vehicle high-speed energy supplement opportunity planning method

Abstract

Aiming at the problems of queuing delay, dynamic electricity price and range anxiety in long-distance travel of electric vehicles, a method of electric vehicle highway energy supplement time planning is proposed to balance timeliness and economy. The application builds an energy supplement decision environment state integrating space-time multi-source information, combines maximum entropy reinforcement learning to establish an energy supplement time decision model, and realizes highway energy supplement planning considering energy supplement efficiency and economy. First, the driving style hidden state extracted by pre-training LSTM is fused with space-time coupled static features to build a high-fidelity normalized environment state. Second, a decision model based on Soft Actor-Critic algorithm is built, and a supplement energy strategy is generated by discrete mapping of continuous action. Finally, a double-channel experience replay mechanism is designed, combined with double Critic network update, to solve the problems of sparse reward and Q value overestimation, realize efficient optimization of energy supplement time, improve algorithm convergence speed and planning performance, and realize optimal balance between timeliness and economy.

Description

Technical Field

[0001] This invention belongs to the fields of intelligent control of electric vehicles, smart transportation and reinforcement learning decision planning technology, and specifically relates to a method for planning the timing of high-speed refueling of electric vehicles that balances timeliness and economy. Background Technology

[0002] In recent years, with the continuous improvement of highway charging infrastructure and the sustained growth of new energy vehicle ownership, the demand for long-distance intercity travel by electric vehicles has been increasing. Compared with urban charging scenarios, highway charging decisions are influenced by a variety of factors, including the vehicle's remaining battery power, charging station queues, dynamic electricity prices, charging power, and road traffic conditions, exhibiting characteristics such as time-varying conditions, dynamic environments, and multi-factor coupling.

[0003] On the other hand, actual vehicle energy consumption is not only affected by factors such as road gradient, vehicle speed, and driving behavior, but also constrained by factors such as changes in battery performance, making future vehicle energy consumption highly uncertain. How to rationally plan the timing and target energy for high-speed refueling based on vehicle operating status and external environmental information, while ensuring refueling safety and considering both refueling time and economic costs, has become an important research issue in the field of high-speed refueling decision-making.

[0004] Existing methods for high-speed refueling planning mainly include heuristic methods based on fixed rules, mathematical programming methods, and reinforcement learning methods. Among these, fixed-rule methods struggle to adapt to dynamically changing traffic and refueling environments; mathematical programming methods typically rely on precise environmental models, limiting their efficiency in complex dynamic scenarios; and while reinforcement learning methods possess online decision-making capabilities, they still suffer from the following shortcomings in high-speed long-distance refueling scenarios: (1) The future energy consumption of vehicles is significantly affected by historical driving behavior and road conditions. Traditional reinforcement learning usually only uses the current observation state to make decisions, which is difficult to fully represent historical time series information and is prone to state estimation errors in some observable environments. (2) High-speed energy replenishment decision-making includes continuous decisions such as "whether to replenish energy" and "to how much to replenish". However, actual charging equipment can usually only perform a limited number of discrete energy replenishment actions. There is a mapping error between continuous actions and discrete execution, which affects the stability of the strategy. (3) In high-speed long-distance scenarios, the probability of vehicle breakdown due to battery depletion is low but the impact is significant, resulting in sparse final rewards and uneven distribution of experience samples during training, which reduces the training efficiency and convergence stability of reinforcement learning algorithms.

[0005] Therefore, there is an urgent need to propose a high-speed refueling timing planning method that can integrate historical time-series information with the current environmental state, take into account continuous decision-making and discrete execution constraints, and improve the stability of reinforcement learning training, so as to achieve comprehensive optimization of refueling time, economic cost and refueling safety in high-speed scenarios. Summary of the Invention

[0006] This invention addresses the problems in existing technologies regarding long-distance highway refueling decision-making, such as incomplete state information, difficulty in directly executing continuous actions, and insufficient stability of reinforcement learning training. It proposes a method for planning the timing of high-speed refueling for electric vehicles that balances timeliness and economy. By constructing a spatiotemporal multi-source state perception model, a maximum entropy reinforcement learning decision-making model, a finite-level action mapping mechanism, and a dual-channel experience playback training mechanism, it achieves comprehensive optimization of refueling timeliness, economic cost, and refueling safety.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A method for planning the timing of high-speed refueling for electric vehicles that balances timeliness and economy includes the following steps: Step 100, the spatiotemporal multi-source state fusion stage, is used to construct a unified state space that can characterize the vehicle's historical operating state and the current environmental state. As the system approaches a charging station, it collects high-frequency historical driving time-series features of the vehicle over the past 10 minutes. The data is then input into a pre-trained LSTM network (130) loaded on the vehicle side to extract 32-dimensional hidden state features representing the vehicle's historical operating states. ; Simultaneously, the system collects 10-dimensional static spatiotemporal state characteristics reflecting the current environment. (120) The dimensional bias is eliminated by the physical normalization module to obtain the normalized characteristics. ; Finally, the fusion unit splices the spatiotemporal features of the two components and outputs a 42-dimensional fused state feature (150), which serves as the input for step 200.

[0008] Step 200, the maximum entropy reinforcement learning decision stage, is used to generate continuous energy replenishment decisions based on the fusion state and complete the state value estimation; The fused state features (150) are input into the SAC agent decision center, and the Gaussian Actor network (210) outputs continuous control actions with exploration noise according to the maximum entropy exploration principle. ; Meanwhile, two independent fits are performed using the dual Critic network (220) within the SAC algorithm framework. The valuation network is used to take its minimum value, which is used to accurately and conservatively assess the expected cumulative return of the state-action pair, providing precise and unbiased policy gradient guidance for the Actor policy network.

[0009] Step 300, Action Mapping and Physical Environment Deduction Stage, is used to realize the mapping of continuous actions to finite discrete energy replenishment levels and obtain environmental feedback information. The action mapping module (310) receives the first-dimensional continuous charging intention and the second-dimensional continuous target power output from the Actor network, and projects the continuous target action to the finite discrete charging level allowed by the vehicle charging environment (320) through the minimum Euclidean distance mapping method; The mapped discrete energy replenishment action is input into the vehicle energy replenishment environment (320) for state deduction, including the following specific steps: The expected queuing time for vehicles to enter the station is estimated using the M / G / c queuing model. Based on the absolute time of vehicle arrival at each station, calculate the spatiotemporal coupled dynamic electricity price and the segmented nonlinear charging time; The system combines a first-order RC battery equivalent circuit model to update the vehicle's real-time state of charge (SOC) and obtain the next moment's state and corresponding one-step instant feedback reward.

[0010] Step 400, the dual-channel experience replay stage, is used to construct training samples that take into account both process experience and end-game experience, thereby improving the sample utilization rate and policy learning stability in sparse reward scenarios. First, the state transition samples generated during the vehicle refueling environment simulation are sent to the hybrid experience sampling controller and classified and managed according to the task state corresponding to the sample. Among them, the process experience samples generated when the vehicle is in a non-terminal state. Stored in the process experience replay pool; final experience samples generated when a vehicle breaks down due to battery depletion or safely reaches its destination. It is stored in the endgame experience replay pool to enhance the efficiency of using endgame reward information during training.

[0011] During network parameter updates, the sampling controller performs mixed sampling from the process experience replay pool and the final experience replay pool according to a preset ratio. As an example, process experience samples and final experience samples are combined into training batches at a ratio of 80% and 20% respectively, which are used for subsequent network parameter updates to improve the problem of sample distribution imbalance and slow strategy convergence caused by the scarcity of final events in high-speed long-distance refueling scenarios.

[0012] Step 500, network parameter update stage, is used to update the parameters of Gaussian Actor network (210) and dual Critic network (220) based on mixed empirical samples, so as to realize the closed-loop optimization of the energy supplementation strategy; First, the temporal difference target value is constructed using the mixed empirical samples obtained in step 400, and the state-action value loss corresponding to the dual Critic network is calculated respectively. The parameter optimization gradient of the Critic network is obtained by minimizing the mean square error. Subsequently, based on the state value estimation results output by the dual Critic network (220) and the maximum entropy policy optimization objective, the policy loss of the Gaussian Actor network (210) is calculated, and the weight parameters of the Actor network and the dual Critic network are updated synchronously using the backpropagation algorithm. After multiple rounds of iterative training, the policy network gradually learns the optimal energy replenishment strategy for vehicles under different traffic conditions, queuing conditions, and electricity price conditions, achieving comprehensive optimization of energy replenishment time, energy replenishment cost, and energy replenishment safety.

[0013] Furthermore, as a preferred embodiment of the present invention, the spatiotemporal multi-source representation fusion stage described in step 100 is further explained as follows: The 10-minute high-frequency historical driving time sequence characteristics It contains four input channels, which are arranged in physical time order as follows: vehicle speed Real-time vehicle acceleration Road slope and ideal physical reference power deviation residual To eliminate the impact of different scales on network training, the temporal features are first Z-score normalized and then used as input to the pre-trained LSTM network (130). When the vehicle approaches the refueling decision station, the pre-trained LSTM network (130) encodes the historical driving time-series features and extracts the hidden state features that can characterize the vehicle's historical driving behavior and energy consumption evolution. The hidden state features It is used to compensate for information that is difficult to observe directly, such as the vehicle's future energy consumption and driving style, thereby enhancing the system's ability to represent historical environmental states and improving the observability of the reinforcement learning state space. Simultaneously, the system collects the static spatiotemporal state characteristics corresponding to the current power replenishment environment in real time, including: the current state of charge of the battery. Distance to destination Distance to the next charging station Expected queuing time at the next station The rated power of the charging pile at the next station The current time Current charging price at the site , Power safety margin for reaching the next station The nominal power of the charging pile at the current site and the actual queuing time at the current station. The above static state features are processed by the physical normalization module to obtain the normalized static state vector. ; Finally, the high-dimensional hidden state features Features of the normalized static spatiotemporal state Perform feature concatenation to construct fused state features:

[0014] The fused state features serve as the unified state input for the SAC agent, while simultaneously integrating historical driving information and current environmental state information, providing a complete and continuous state representation for subsequent maximum entropy reinforcement learning strategy decisions.

[0015] As a preferred implementation of the maximum entropy adaptive decision-making stage in step 200, the SAC agent receives the 42-dimensional fused state features output in step 100. As a unified state input, a Gaussian Actor network (210) is used to generate continuous control actions at the current decision moment; Specifically, the Gaussian Actor network performs a nonlinear mapping on the input state and outputs the mean vector of the probability distribution of continuous actions. Sum of logarithmic standard deviation vector The mean vector is used to represent the expected output of each action dimension in the current state, and the logarithmic standard deviation vector is used to represent the magnitude of action randomness, so as to achieve adaptive adjustment of strategy exploration capability. Furthermore, to ensure the differentiability of the action sampling process, the Gaussian Actor network employs a reparameterized sampling mechanism to generate continuous actions, the expression of which is:

[0016] in, A continuous action vector; This is the action mean vector; The standard deviation vector of the action; For random noise that follows a standard multivariate normal distribution; This represents Hadamard element-wise multiplication; tanh(·) is used to constrain actions to a continuous action space of [-1, 1]. Furthermore, to enhance the exploratory ability during policy learning and prevent premature policy convergence, a policy entropy regularization term is introduced into the loss function of the Gaussian Actor network (210), and its optimization objective is expressed as:

[0017] in: The weight parameters of the Actor network (210) are represented; This represents the action probability distribution output by the policy network; This represents the entropy temperature coefficient, used to adjust the randomness of the strategy; , These represent the state-action values ​​output by the first and second Critic networks, respectively; min(·) indicates taking the smaller of the two Q values ​​to reduce the overestimation error of the Q value. By using the above-mentioned strategy update method, the policy network can improve the stability and convergence speed of energy replenishment decisions while maintaining sufficient exploration capabilities.

[0018] As a preferred embodiment of the finite-level action mapping stage in step 300, the action mapping module (310) receives a continuous action vector output by the Gaussian Actor network (210), wherein the first dimension represents the continuous charging intention and the second dimension represents the continuous target SOC; Considering that actual high-speed charging devices only support a limited number of discrete charging targets, the system maps continuous target SOC to a pre-set set of limited discrete charging levels; Furthermore, the number of limited energy replenishment levels is set to N. In this embodiment... The corresponding discrete energy replenishment level set is defined as follows:

[0019] Each energy replenishment level represents the target SOC of the vehicle after it has been fully charged. This embodiment is only a preferred implementation method and does not constitute a limitation on the number of energy replenishment levels or their value range. Furthermore, the system calculates the Euclidean distance between the continuous target SOC value raw_target_soc output by the Actor network and each discrete energy replenishment level, and selects the level with the smallest distance as the final action to be executed. The calculation method is as follows:

[0020] The final mapped target SOC is represented as follows:

[0021] in, This represents the continuous target SOC output by the Actor network; This represents the index corresponding to the minimum energy replenishment level. This represents the actual execution target SOC after mapping; After completing the action mapping, the system inputs the discrete energy replenishment action into the highway vehicle energy replenishment environment model for physical deduction, and combines the queuing model, dynamic electricity price model and battery dynamics model to calculate the next state and immediate reward, providing state transition samples for subsequent experience storage and network parameter updates.

[0022] Furthermore, as a preferred implementation of the highway vehicle refueling environment model (320) in step 300, the vehicle refueling environment model determines the vehicle's next state and immediate reward based on the vehicle's operating state after performing the refueling action. Perform joint computation and output state transition samples. For storage by the experience playback module in step 400; The instant reward The evaluation is composed of three dimensions: time cost, economic cost, and power safety. When the vehicle reaches the destination, the final reward is further calculated based on the final state to reflect the overall advantages and disadvantages of the entire charging strategy.

[0023] Furthermore, the vehicle refueling environment model calculates the single-step instant reward based on the vehicle's operating results after refueling, and its expression is:

[0024] in, As a time penalty; As a penalty for economic costs; Penalty for violating battery safety regulations; The formula for calculating the time penalty is:

[0025] in, Weighted by time penalty; The time spent waiting in line at the current station; This refers to the actual charging time. Fixed time for operations such as vehicle entry and exit from service areas, parking, plugging and unplugging charging guns, and payment; The formula for calculating the economic cost penalty item is as follows:

[0026] in, As a weighted average of economic costs, The total cost of electricity and service for a single charge; To ensure the vehicle has a reasonable safety margin of remaining battery power, the environmental model calculates a safety penalty for battery power based on the current State of Charge (SOC):

[0027] in, For anxiety weight parameters, Safety concerns about minimum battery capacity; Furthermore, when the vehicle enters the terminated state (done=True), the system stops calculating the immediate reward for each step and calculates the final reward based on the vehicle's final running result:

[0028] Specifically, when SOC≤0, the vehicle is considered to have broken down due to low battery and is given a large negative reward; when the vehicle safely reaches the destination, a continuous safety reward is calculated based on the remaining SOC in addition to the fixed success reward, in order to encourage vehicles to reduce charging costs while ensuring safe charging. In this embodiment, -4000 and +2500 are parameters used in the embodiment and do not constitute a limitation of the present invention.

[0029] Furthermore, to avoid sudden changes in the final reward near the safe energy level, which could cause gradient oscillations during the training of the policy network, this invention designs the remaining energy level at the final stage as a piecewise continuous function, and calculates the final safe reward based on the remaining SOC when the vehicle reaches the destination.

[0030] When the vehicle safely reaches its destination, the remaining battery power reward consists of a fixed success reward and a battery power safety reward:

[0031] Specifically, when the remaining battery power at the endpoint is lower than the safe battery power threshold... The quadratic penalty function is used to constrain the vehicle to maintain a minimum safe battery level. Furthermore, when the vehicle's remaining SOC at the endpoint is between the safety threshold and the target remaining battery level, a linear continuous reward function is used:

[0032] The reward value increases continuously as the remaining battery power at the finish line increases; Furthermore, when the remaining SOC at the endpoint exceeds 25% of the target remaining capacity, to avoid the strategy being infinitely biased towards overcharging, the system adopts an inverse linear decay function:

[0033] The reward gradually decreases as the remaining battery power continues to increase; Through the above segmented continuous reward design, while ensuring that the vehicle has the necessary safe power reserve, overcharging behavior is suppressed, enabling the intelligent agent to autonomously learn a comprehensive optimization strategy that takes into account the safety of charging, charging time and economic cost.

[0034] As a preferred implementation of the dual-channel experience playback stage in step 400, the system classifies, stores, and mixes the experience data based on the state transition samples output by the vehicle refueling environment model to improve the utilization efficiency of effective samples during reinforcement learning training.

[0035] Specifically, after the vehicle refueling environment model completes one state transition, it generates the following empirical samples:

[0036] in, Indicates the current fusion status; Indicates the execution of an action; This indicates a reward for environmental feedback. Indicates the fusion state at the next moment; Indicates whether the current round has ended; Furthermore, the hybrid experience sampling controller is based on the termination flag. And the final operating status of the vehicle, to dynamically classify the experience samples: when When the sample is determined to be a state transition sample during normal vehicle operation, it is written into the process experience replay pool (410) for storage, and used to learn the continuous decision-making rules during the vehicle refueling process. when At that time, the system further judges whether the vehicle has run out of power and broken down or safely reached the destination based on the final state, and writes the corresponding experience sample as the final experience into the final experience replay pool (420) to enhance the agent's learning ability for final reward and sparse reward information. Furthermore, in order to balance the representativeness of the training samples with the importance of the final experience, the hybrid experience sampling controller (430) extracts experience samples from the process experience replay pool (410) and the final experience replay pool (420) respectively, and forms a unified training batch. In a preferred embodiment of the present invention, during each network parameter update, the training batch is mixed and sampled according to a preset ratio, wherein the sampling ratio of the process experience replay pool is 80% and the sampling ratio of the final experience replay pool is 20%. The above ratio is only a preferred embodiment, and those skilled in the art can adjust it according to the specific training task without departing from the protection scope of the present invention; Furthermore, the mixed training samples are uniformly input into the network parameter update module in step 500 to calculate the temporal difference target value and complete the parameter update of the Actor network and the dual Critic network. Since the final experience samples contain sparse reward information such as vehicle breakdown due to low battery and safe arrival at the destination, by establishing a dual-channel experience replay mechanism and performing proportional reinforcement sampling on the final experience, the frequency of the final reward in the training samples can be increased. This can alleviate the problem of slow training convergence speed and unstable policy learning caused by the sparse rewards in high-speed long-distance refueling scenarios, thereby improving the agent's ability to learn long-term refueling strategies.

[0037] Furthermore, as a preferred implementation of the network parameter update stage in step 500, the decision center randomly selects a training batch from the mixed experience samples obtained in step 400, and calculates the temporal difference target value (TD Target) based on the state transition samples as a supervision signal for the value network. Specifically, the first Critic network and the second Critic network calculate the corresponding value loss function based on the current state, the action to be performed, and the target Q value, respectively, and optimize it using mean squared error (MSE) to reduce the error between the predicted Q value and the target Q value. Furthermore, after completing the parameter update of the dual Critic network (220), the updated value network is used to perform policy gradient optimization on the Actor network, so that the output of the policy network can be continuously updated in the direction of increasing the long-term expected cumulative reward.

[0038] Furthermore, to improve training stability, the target Critic network employs a soft update method for parameter synchronization, with the update expression being:

[0039] in, Indicates the current network parameters; Indicates the target network parameters; This represents the soft update coefficient, which ranges from 0 to 1. By using the above network update method, the Actor network, dual Critic network, and target network form a closed-loop iterative training mechanism, which improves the convergence stability and generalization ability during the energy supplementation strategy learning process.

[0040] The present invention also provides a high-speed refueling timing planning system for electric vehicles that balances timeliness and economy, including a memory and a processor, wherein the processor executes a computer program stored in the memory to implement the steps of the above method.

[0041] The beneficial effects of this invention are: The present invention provides a method for planning the timing of high-speed refueling for electric vehicles that balances timeliness and economy, which has at least the following advantages compared with the prior art: 1. Improve environmental state representation capabilities and enhance the accuracy of refueling decisions. This invention constructs a spatiotemporal multi-source state fusion mechanism, fusing historical temporal features extracted by a pre-trained LSTM network with the vehicle's current static environmental features to form a fused state representation. This effectively represents the vehicle's historical operating information and current environmental information, improving the agent's state perception capability in high-speed refueling scenarios. This provides more accurate state input for reinforcement learning strategies, thereby improving the accuracy and stability of refueling decisions.

[0042] 2. Improve the efficiency of experience utilization and enhance training performance under sparse reward conditions. This invention establishes a dual-channel experience replay mechanism consisting of a process experience replay pool and a final experience replay pool, and performs mixed sampling according to a preset ratio. This ensures that the training process takes into account both ordinary samples and key final samples, thereby improving the efficiency of final experience utilization. It also mitigates the impact of sparse rewards and uneven sample distribution on reinforcement learning training in high-speed, long-distance refueling scenarios, which is beneficial for improving the convergence speed and training stability of the policy network.

[0043] 3. Balancing continuous strategy optimization capability with engineering feasibility. This invention employs a maximum entropy reinforcement learning algorithm to output continuous actions, and projects these continuous actions onto a preset finite energy replenishment level through an action mapping mechanism, achieving effective connection between the continuous strategy space and the actual discrete energy replenishment actions. While retaining the continuous action strategy optimization capability, it meets the engineering constraints of the discrete control mode of actual high-speed charging equipment, improving the engineering applicability and execution stability of the energy replenishment decision-making method.

[0044] 4. Improve the continuity of reward feedback and promote stable strategy optimization. This invention designs a comprehensive reward mechanism for vehicle refueling tasks, including time cost, economic cost, battery safety, and final reward. The final reward uses a continuously changing reward function instead of the traditional discrete reward and punishment method, making the reward feedback smoother and more continuous. This reduces training fluctuations caused by sudden changes in rewards during strategy updates, which is conducive to the stable convergence of reinforcement learning strategies and achieves comprehensive optimization of refueling efficiency, economic cost, and driving safety.

[0045] 5. Balancing Recharge Timeliness and Economic Cost to Achieve Comprehensive Performance Optimization for High-Speed ​​Recharge. This invention comprehensively considers various factors such as vehicle driving status, queuing time, dynamic electricity price, and battery status. Through maximum entropy reinforcement learning, it achieves adaptive optimization of recharge timing. While ensuring the vehicle safely reaches its destination, it achieves a comprehensive trade-off between recharge timeliness and economic cost, improving the overall performance of recharge decisions in high-speed long-distance travel scenarios. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be further described below with reference to the accompanying drawings. It should be understood that the following drawings only show preferred embodiments of the present invention. Those skilled in the art can obtain other forms of drawings based on these drawings without creative effort, and all of them fall within the protection scope of the present invention.

[0047] Figure 1 The present invention provides an overall flowchart of a method for planning the timing of high-speed refueling of electric vehicles that balances timeliness and economy.

[0048] Figure 2 This is a schematic diagram of a decision-making training framework for high-speed electric vehicle refueling that balances timeliness and economy, provided by the present invention. Detailed Implementation

[0049] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0050] like Figure 1 and Figure 2 As shown, the present invention provides a method for planning the timing of high-speed refueling for electric vehicles that balances timeliness and economy. When specifically implemented in the vehicle-side energy management controller (BMS) and the roadside edge cloud server, the specific implementation process of its micro-dynamic evolution and macro-reinforcement learning decision parameter update is as follows: Step 100: Spatiotemporal Multi-Source State Perception and Fusion Stage like Figure 1 As shown, this step is used to construct a fusion state that can comprehensively represent the vehicle's historical operating status and current refueling environment information, providing a unified state input for subsequent reinforcement learning decisions.

[0051] First, in this embodiment, the vehicle controller continuously collects historical driving information from the past 600 seconds to form a 10-minute high-frequency historical driving time sequence feature. This length covers approximately 10 minutes of driving behavior changes before a vehicle enters a highway service area, ensuring both the integrity of the temporal features and onboard computing efficiency. The sampling period is 1 second, therefore the temporal features comprise 600 time steps, each containing four physical quantities: vehicle speed, etc. longitudinal acceleration of vehicles Road slope and ideal physical reference power deviation residual ; The residual deviation of the ideal physical reference power is defined as follows:

[0052] in, This indicates the real-time driving power collected by the vehicle. This represents the theoretical driving power of the vehicle under ideal steady-state driving conditions, and its calculation process is as follows:

[0053] in, For vehicle curb weight; It is the acceleration due to gravity; This is the rolling resistance coefficient; air density; This refers to the air drag coefficient; The vehicle's frontal area; Further, the vehicle's mechanical output power is obtained:

[0054] Taking into account vehicle drive efficiency, the ideal electric drive power is calculated as follows:

[0055] in, For the overall vehicle transmission efficiency; For auxiliary load power such as air conditioners and vehicle electronic devices; Since the above ideal power model only considers the steady-state motion of the vehicle and does not take into account the driver's operating habits, the vehicle's transient acceleration and deceleration, and complex road disturbances, using the deviation between the real-time power and the ideal power as the residual input is beneficial for the reinforcement learning agent to learn the dynamic impact of driving behavior on energy consumption. Subsequently, the historical time series matrix is ​​standardized using Z-score, the calculation formula of which is as follows:

[0056] in, Represents the original historical time series matrix; This represents the mean vector obtained from statistics during the training phase. Represents the corresponding standard deviation vector; The standardized historical time-series features are input into a pre-trained LSTM network for feature encoding. The LSTM network employs a two-layer recurrent neural network structure, with its last hidden state serving as the historical driving state feature.

[0057] in, It can comprehensively characterize the vehicle's driving behavior, road conditions, and energy consumption trends over a period of time, and realize the hidden state reconstruction of some observable state information. At the same time, the vehicle-side controller collects the static state characteristics of the current environment in real time:

[0058] The parameters mentioned above represent: current battery state of charge, distance to destination, distance to the next charging station, estimated queuing time at the next station, charging power at the next station, current time, current charging price, battery safety margin, current station charging power, and current station actual queuing time. Since each static state feature has different physical dimensions and numerical ranges, in order to avoid one feature dominating the network weight update during training, this embodiment uses the physical normalization module (140) to perform dimensionless processing according to the engineering value range of each physical quantity, and obtains the normalized state vector:

[0059] The power safety margin is defined as follows:

[0060] in,

[0061] In the formula, This indicates the estimated electricity consumption of the vehicle as it travels to the next refueling station; Indicates the rated capacity of the power battery; The estimated energy consumption is determined by the theoretical energy consumption calculated from the future road physical model and the prediction residual of historical driving behavior, thus comprehensively reflecting the future driving risks. The features are normalized as follows: Battery state of charge:

[0062] Distance to destination:

[0063] The 720km is the typical long-distance high-speed planning distance set in this embodiment; Distance to the next charging station:

[0064] Where 100km is the longest distance between adjacent stations in the high-speed service area set in this embodiment; Estimated waiting time:

[0065] The 35-minute timeframe represents the statistically determined maximum queuing time at highway service areas. Next station charging power:

[0066] The 170kW corresponds to the maximum fast charging rated power allowed by the vehicle-side BMS used in this embodiment; Current moment:

[0067] Current charging price:

[0068] Safe power margin:

[0069] Current station charging power:

[0070] Current actual queuing time at the station:

[0071] After the above processing, each state feature is mapped to a unified numerical range, which improves the numerical stability of the state space and provides a consistent input scale for subsequent state fusion and reinforcement learning training. Finally, the historical hidden state features are concatenated with the current static state of the environment to form a unified state vector:

[0072] The 42-dimensional fusion state features are obtained and used as the unified state input for the SAC agent decision center in step 200. Through the above spatiotemporal multi-source state fusion process, this embodiment realizes a unified expression of historical driving behavior information and current refueling environment information, enabling the reinforcement learning agent to obtain a more complete description of the environmental state and improve the state observability and decision-making accuracy in complex high-speed refueling scenarios.

[0073] Step 200: Maximum Entropy Adaptive Decision-Making Phase As a specific implementation of the maximum entropy adaptive decision-making stage in step 200, the SAC agent central hub receives the 42-dimensional fused state features output from step 100. The Gaussian Actor network is used to generate continuous actions, while the dual Critic value network is used to evaluate the state-action value. In this embodiment, the Gaussian Actor network adopts a two-layer fully connected neural network structure. The first hidden layer contains 128 neurons, and the second hidden layer contains 64 neurons. The activation function is the ReLU function. The fusion state features... After being input into the Actor network, the mean vector of the probability distribution of continuous actions is obtained through nonlinear mapping. With the logarithmic standard deviation vector ; To avoid excessive policy variance leading to unstable action sampling in the early stages of training, upper and lower limits are set for the logarithmic standard deviation, and their expressions are as follows:

[0074] in, This represents the interval truncation operator. When the calculation result is less than -5, it takes -5; when the calculation result is greater than 1, it takes 1. This ensures that the action sampling variance is always kept within a reasonable range, thereby improving training stability. Subsequently, a series of potential actions are generated based on the reparameterization sampling mechanism, and their expression is:

[0075] in, It is a standard multivariate normal random variable; Represents element-wise product; To satisfy the value constraints of the vehicle's refueling decision-making actions, the continuous actions are further compressed using the hyperbolic tangent function to obtain the actual actions:

[0076] Among them, action vector It includes two dimensions, which correspond to whether the vehicle enters the current charging station for energy replenishment and the target amount of energy replenishment. Furthermore, to satisfy the non-negativity constraint of the target replenishment capacity, the second dimension of the action is mapped to an interval. The mapping relationship is expressed as:

[0077] in, This represents the second dimension of the action vector output. This represents the target SOC continuous control quantity in the subsequent action mapping module; Since the Tanh function is a nonlinear invertible transformation, it alters the probability density of the original Gaussian distribution. Therefore, the policy probability density needs to be corrected based on the principle of variable transformation. Its logarithmic probability calculation formula is as follows:

[0078] in, The probability density function is a Gaussian distribution. A very small positive number introduced to prevent the occurrence of logarithmic zero during numerical calculations; Indicates the first Safeguard potential actions; After generating the strategy action, the state features will be fused. The state-action value is evaluated by inputting the action vector into a dual-Critic value network. In this embodiment, the two Critic networks adopt the same network structure but with independent parameters. Each Critic network contains two fully connected hidden layers with 128 and 64 neurons respectively. The output layer contains one neuron, which is used to output the value function estimate of the corresponding state-action pair. First Critic network output:

[0079] Second Critic network output:

[0080] in, These are the parameters of the first Critic network; These are the parameters for the second Critic network; During training, to suppress the overestimation problem in traditional Q-learning, the double Critic minimum is used as the target value estimate, and its expression is:

[0081] Subsequently, the minimum value estimate is used in the calculation of the Bellman objective value and serves as the value basis for the gradient optimization of the Actor network policy, thereby guiding the policy to be continuously updated in the direction of better overall return. Furthermore, during the Actor network training process, a Shannon entropy regularization term is introduced to construct the maximum entropy optimization objective, the objective function of which is:

[0082] in, Indicates the parameters of the Actor network; It represents the entropy temperature coefficient, used to balance the ability to explore strategies and the ability to optimize value.

[0083] Maximizing the above objective function allows the agent to increase cumulative rewards while maintaining a high degree of policy randomness, thus avoiding getting trapped in local optima.

[0084] Through the above-mentioned maximum entropy strategy optimization mechanism, the system can effectively suppress Q-value overestimation while ensuring sufficient exploration, improve the stability and convergence speed of strategy learning in complex high-speed refueling scenarios, and provide continuous control action input for finite-level action mapping and vehicle refueling environment deduction in step 300.

[0085] Step 300: Discrete Motion Mapping and Vehicle Recharge Environment Deduction Stage (1) Action Discrete Mapping As a specific implementation of step 300, after the vehicle arrives at the current charging decision station, the continuous actions output in step 200 are first input into the finite-level action mapping module (310) to convert the continuous actions into discrete charging actions that are allowed to be executed by the vehicle charging environment. The continuous actions include two dimensions: The first dimension action indicates whether the vehicle enters the current service area for refueling; The second dimension represents the vehicle's target charging capacity.

[0086] To avoid unnecessary time loss caused by vehicles frequently entering service areas during the initial stage, this embodiment sets a starting distance constraint. When the vehicle's cumulative driving distance is less than 100km, the system forcibly implements a skip-service strategy, disallowing entry into the current service area for refueling. If the energy replenishment conditions are met, then the feasible region constraint is applied to the target energy replenishment amount:

[0087] In this embodiment, the target charging capacity of the vehicle is limited to between 60% and 90% in order to balance charging efficiency and subsequent driving safety. Subsequently, the action mapping module establishes a finite set of energy replenishment levels:

[0088] For the continuous target charge output by the Actor, the minimum Euclidean distance principle is used to complete the action mapping:

[0089] The discrete target energy replenishment amount actually performed by the vehicle is obtained.

[0090] (2) Queuing Model After the action mapping is completed, the vehicle enters the vehicle charging environment (320). The environment first calculates the vehicle's estimated waiting time based on the current queuing status of the target charging station. This embodiment uses the M / G / c queuing model to describe the queuing process for charging stations in highway service areas; Assume the charging station includes The system displays a charging station and then reads the current time. Vehicle arrival rate The simulation is a sinusoidal wave:

[0091] Among them, the charging service rate is fixed at 100%. (Average charging time per bike: 30 minutes); Subsequently, the system load intensity was calculated.

[0092] like Then forcibly restrict ; The queuing probability is calculated using the Erlang C formula:

[0093] Queue time is calculated and corrected using the Lee-Longton formula (preset service time variation coefficient). );

[0094] get

[0095] (3) Nonlinear charging model Set the maximum charging power to With power of power station piles Minimum value:

[0096] In step 300, the charging time is calculated from the charging power. The specific principles and formulas for calculating analytical integrals are as follows: According to the law of conservation of battery charge and energy, charging power Battery capacity With charging time The differential relationship between the infinitesimal elements is expressed as:

[0097] Separating variables from the above equation and converting the time units (h to min), we obtain the first... Within each charging decision interval, from the initial charge level... Charge to target capacity High-precision charging time The general formula for continuous integrals is:

[0098] In the formula, The total capacity constant of the vehicle's power battery; Considering the constant current-constant voltage (CC-CV) charging characteristics of the power battery, the charging power variation law is different in different SOC ranges. Therefore, this embodiment uses a three-stage nonlinear charging model to describe the actual charging process of the vehicle. ① Constant power charging range ( ): Within this range, the charging power Hengwei If charging upper and lower limits The charging time for that segment can then be obtained by integration. The closed-form solution for (min) is:

[0099] ② Linear decay charging range ( ): Within this range, the charging power Follow The power decreases linearly with increasing power. Its power change function is expressed as:

[0100] In the formula, the linear decay slope The physical calculation formula is:

[0101] Among them, the According to this embodiment, it is set as follows:

[0102] In this embodiment, 2.2 is a fitted value; If charging upper and lower limits By using the substitution method to differentiate and integrate the linear denominator, the charging time for that segment can be obtained. The closed-form solution is:

[0103] In the formula, the position of the denominator is... and These correspond to the real-time power calculation values ​​at the initial power level and the target power level, respectively. ③ Exponential decay charging range ( Within this range, the charging power decreases exponentially and is limited by... The minimum charging power; The system first solves for the exponential curve decay to The corresponding inflection point of the electricity level The calculation formula is:

[0104] For any charging upper and lower limits The system is based on the intersection inflection point Perform the following segmented integration to obtain the charging time for that segment. Closed-form solution: If the sub-interval is in the exponentially decaying segment (i.e.) ),at this time The closed-form integral solution is:

[0105] If the subinterval is in the constant baseline power segment (i.e.) At this time, the charging power is constant. The closed-form integral solution is:

[0106] The total time for this segment is the sum of the two items mentioned above:

[0107] The system makes decisions based on the true values ​​in the input. It automatically performs interval projection matching, sums the calculated time for each interval segment, and obtains the charging time.

[0108] Finally, add the fixed time required for operations such as vehicle entry and exit from service areas, parking, plugging and unplugging charging guns, and payment. This gives the total time spent on the actual charging of the vehicle.

[0109] ④ Electricity cost model After the vehicles finish queuing, the energy replenishment cost is calculated based on the electricity price model of the current area; The comprehensive charging unit price consists of the real-time base electricity price, service fee, and random disturbance, and its expression is:

[0110] Among them, dynamic benchmark electricity price The charging service fee was obtained by fitting a Gaussian mixture duck curve with mean regression noise. Determined by region; The preset random disturbances reflect market price fluctuations; Finally, the charging cost was:

[0111] in, This refers to the state of charge (SOC) after charging is complete. 62.5 represents the comprehensive charging unit price; 62.5 represents the battery capacity (kWh) in this embodiment.

[0112] ⑤ Status Update After the vehicle completes refueling, the vehicle refueling environment updates its environmental status based on the execution results.

[0113] Specifically, it includes: The vehicle's SOC is updated to the target SOC based on the first-order RC circuit model; The current time includes additional queuing time, charging time, and fixed operation time; The vehicle's location has been updated to the next highway service area. The cumulative driving distance is updated synchronously; Calculate the remaining SOC after driving to the next service area based on the vehicle dynamics model; Recalculate the safety margin; Update the static state information of the next decision node; Subsequently, the vehicle's charging environment outputs the next moment's fusion state. and corresponding instant rewards Together they form the state transition mechanism in reinforcement learning. And then send it to step 400 for experience storage; Through the above-mentioned continuous action mapping, vehicle queuing simulation, charging time calculation, electricity price estimation and environmental state update process, this embodiment realizes the closed-loop interaction between reinforcement learning actions and the real high-speed charging environment, providing complete state transition samples for experience replay in step 400.

[0114] Step 400: Dual-channel experience pool playback and preset ratio mixed sampling After the vehicle completes the environmental simulation in step 300, the vehicle's refueling environment generates a complete reinforcement learning state transition sample, which is represented as follows:

[0115] in: This represents the fusion state features at the current moment, i.e., the 42-dimensional fusion state vector constructed in step 100; This represents the actual energy replenishment action output by the SAC agent in the current state and mapped by the action in step 300; This refers to the immediate reward obtained by the vehicle after performing the action, which consists of time cost, economic cost, battery anxiety penalty, and end-game reward. This represents the fused state characteristics obtained at the next moment after the vehicle has completed recharging and traveled on subsequent road sections. This indicates the end of the journey, when the vehicle has safely reached its destination or the journey has ended due to depleted battery power. ;otherwise, .

[0116] Subsequently, the hybrid experience sampling controller follows the final flag. Classify and store state transition samples. When When the state transition sample is stored in the process experience replay pool, the state transition sample is stored in the pool. When the state transition sample is stored in the final experience replay pool, the state transition sample is stored in the final experience replay pool.

[0117] The process experience replay pool is used to store state transition samples during normal vehicle operation; The endgame experience replay pool is used to store endgame experiences generated when a vehicle safely reaches the finish line or breaks down due to low battery. In this embodiment, the capacity of the process experience replay pool is set to 50,000 samples, and the capacity of the final experience replay pool is set to 10,000 samples. During network training, the sampling controller randomly selects samples from the two experience pools according to a preset ratio to form a mixed training batch. In this embodiment, an 8:2 ratio is used for mixed sampling, with each batch containing 128 training samples, of which process experience accounts for 80% and final experience accounts for 20%. By employing a dual-experience replay mechanism, the utilization rate of endgame experience is improved, effectively alleviating the training difficulties caused by reward sparsity and increasing the policy convergence speed.

[0118] Step 500: Gradient backpropagation guides the policy update phase As a specific implementation of the gradient backpropagation strategy update stage in step 500, the SAC agent receives the mixed training batch output in step 400 and optimizes the parameters of the Actor network and the dual Critic network according to the maximum entropy reinforcement learning algorithm, thereby realizing the continuous iterative update of the energy replenishment decision strategy.

[0119] First, based on the next state in the current training batch Generate the next action using the target Actor network. And calculate the corresponding policy probability logarithmic value. .

[0120] Subsequently, the next state and action are input into the target dual-Critic network to obtain two target state-action value estimates, and the smaller value is taken as the target value:

[0121] Furthermore, the time-difference objective value is calculated based on the Bellman equation:

[0122] in: Indicates an immediate reward; Indicates the discount factor; Indicates the endgame; Represents the entropy temperature coefficient; Subsequently, the value function loss of the two Critic networks is calculated separately, and their objective functions are expressed as follows:

[0123] in, This indicates the number of samples in the training batch.

[0124] Based on the Critic loss function, the parameters of the dual Critic network are updated through backpropagation algorithm so that the value function gradually approximates the true state – action value. After completing the value network update, the Actor network policy loss is calculated using the current Critic network output, and its objective function is expressed as:

[0125] The first term is the strategy entropy regularization term, which is used to maintain the strategy's exploration capability; the second term is the state-action value term, which is used to guide the strategy to optimize towards higher cumulative returns. Subsequently, the Actor network parameters are updated using the gradient backpropagation algorithm, enabling the policy network to continuously improve its overall cumulative reward.

[0126] To improve the stability of the training process, this embodiment further employs a soft update method to synchronize the target Critic network parameters, the update formula of which is expressed as:

[0127] in: Indicates the soft update coefficient; Indicates the current Critic network parameters; Indicates the target Critic network parameters; Through the aforementioned gradient backpropagation and target network soft update process, the SAC agent continuously optimizes strategy parameters by utilizing new experiences generated by the vehicle refueling environment, so that the refueling decision strategy gradually converges to the optimal strategy that takes into account time efficiency, refueling economy and range safety.

[0128] Once training is complete, only the trained Actor policy network is deployed in the vehicle controller. During actual operation, the vehicle only needs to input the current fused state features to quickly output the corresponding high-speed power replenishment decision, realizing online real-time inference without needing to train the network again.

[0129] Based on the above description, the technical solution of the present invention, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, cloud server, vehicle network device, vehicle controller ECU, battery management system BMS, intelligent driving computing platform, or roadside edge computing node, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention.

[0130] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems, devices, or chips disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0131] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for planning the timing of high-speed refueling for electric vehicles that balances timeliness and economy, characterized in that, Includes the following steps: Step 100: Spatiotemporal Multi-Source State Perception and Fusion Stage Collect high-frequency historical driving time-series features of vehicles over the past 10 minutes The data is then input into a pre-trained LSTM network (130) to extract 32-dimensional hidden state features representing the historical operating states of the vehicle. ; Synchronously collect 10-dimensional static spatiotemporal state features corresponding to the vehicle's current operating environment. And normalization is performed through the physical normalization module (140); The 32-dimensional hidden state features With normalized 10-dimensional static spatiotemporal state features The fusion process is performed to construct a 42-dimensional fusion state feature. , as the state input of the reinforcement learning agent; Step 200, Maximum Entropy Reinforcement Learning Decision Stage: The 42-dimensional fused state features are input into the SAC agent decision center (200), and the Gaussian Actor network (210) outputs continuous actions according to the maximum entropy strategy. The dual Critic network (220) estimates the state action value respectively, and takes the smaller value of the two as the value estimation result, which provides a basis for the Actor network to update the strategy. Step 300: Action Discrete Mapping and Vehicle Recharge Environment Deduction Stage: The action mapping module (310) receives the continuous actions output by the Actor network and projects the continuous target actions to the finite discrete energy replenishment levels allowed to be executed by the vehicle energy replenishment environment (320) through the minimum Euclidean distance mapping method; The mapped energy replenishment action is input into the vehicle energy replenishment environment (320) for state deduction. The vehicle queuing waiting time is calculated through the M / G / c queuing model. The spatiotemporal coupled dynamic electricity price and segmented nonlinear charging time are calculated based on the vehicle arrival time. The real-time charge state of the vehicle is updated in combination with the first-order RC battery equivalent circuit model to obtain the state of the next moment and the corresponding single-step instant feedback reward. Step 400, Dual-channel experience playback and mixed sampling stage: The state transition samples generated by the vehicle refueling environment (320) are input into the hybrid empirical sampling controller (430); Among them, the process experience samples generated during the normal driving of the vehicle are stored in the process experience replay pool (410), and the final experience samples generated when the vehicle runs out of power and breaks down or safely reaches the destination are stored in the final experience replay pool (420). During training, the hybrid experience sampling controller (430) performs hybrid sampling of process experience samples and final experience samples according to a preset sampling ratio to construct the training samples required for policy update; Step 500: Gradient backpropagation-guided strategy update phase: Using the mixed experience samples obtained in step 400, the temporal difference target value and state action value loss are calculated. The network parameters of the Gaussian Actor network (210) and the dual Critic network (220) are updated respectively through the backpropagation algorithm, so that the agent's strategy continues to iteratively optimize towards maximizing the comprehensive reward, thereby realizing the adaptive learning of the high-speed energy replenishment timing planning strategy.

2. The method for planning the timing of high-speed refueling for electric vehicles, balancing timeliness and economy, as described in claim 1, is characterized in that... The 10-minute high-frequency historical driving time sequence characteristics (110) contains four input channels, which are in the following order: vehicle speed Real-time vehicle acceleration Road slope and ideal physical reference power deviation residual ; When the vehicle approaches the decision-making station, step 100 will use the 10-minute high-frequency historical driving time sequence features. (110) Input into the pre-trained LSTM network (130) to extract the 32-dimensional hidden state features. The hidden state features are used to characterize the temporal dependencies in the vehicle's historical operating conditions, in order to compensate for the lack of current observation information and improve the observability of the vehicle's state.

3. The method for planning the timing of high-speed refueling for electric vehicles that balances timeliness and economy according to claim 1, characterized in that: The 10-dimensional static spatiotemporal state features (120) Used to characterize the current operating environment of the vehicle, including: Current battery state of charge Distance to destination Distance to the next charging station Expected queuing time at the next charging station The nominal power of the charging pile at the next charging station Current moment Current charging price at the site Battery safety margin to the next station The nominal power of the charging pile at the current site and the actual queuing time at the current station. .

4. The method for planning the timing of high-speed refueling for electric vehicles that balances timeliness and economy according to claim 3, characterized in that, The 32-dimensional high-dimensional hidden state The normalized 10-dimensional static spatiotemporal state features (120) Perform state splicing to construct the 42-dimensional fused state feature (150), which is represented as: ； The fusion state feature (150) includes both the vehicle's historical running sequence information and the current environmental state information. By using unified state coding, it enhances the vehicle's running state representation capability, improves the agent's observability of the vehicle's dynamic environment, and provides state input for the subsequent maximum entropy reinforcement learning strategy network.

5. The method for planning the timing of high-speed refueling for electric vehicles that balances timeliness and economy according to claim 1, characterized in that... : In step 200, the Gaussian Actor network (210) receives the 42-dimensional fused state features (150) as input and outputs the mean vector corresponding to the probability distribution of continuous actions. With the logarithmic standard deviation vector To construct the Gaussian policy probability distribution in the current state; The Gaussian Actor network (210) uses a reparameterized sampling mechanism to generate continuous actions, and its calculation formula is as follows: ； Wherein, ε follows a standard multivariate normal distribution. ⊙ represents element-wise multiplication. The tanh(·) function is used to constrain continuous actions to a preset action space to ensure that the actions meet the energy replenishment control constraints, while keeping the sampling process differentiable so that the policy gradient can update the network parameters through backpropagation. During training and updates, the loss function of the Gaussian Actor network (210) incorporates a Shannon Entropy regularization term for the policy probability distribution, and the parameters are optimized using the following objective function: ； in, The weight parameters of the Gaussian Actor network (210) are represented; α represents the entropy temperature coefficient, which is used to adjust the degree of exploration. This represents the action probability distribution output by the policy network; , These represent the state-action value estimates of the outputs of the first and second Critic networks, respectively.

6. The method for planning the timing of high-speed refueling for electric vehicles that balances timeliness and economy according to claim 1, characterized in that... : In step 300, in order to ensure that the continuous actions output by the reinforcement learning agent meet the constraints of the limited energy replenishment level of the actual charging device, a limited energy replenishment level set is set as the action execution space. As an example, the number of the finite energy replenishment levels is set to The corresponding target SOC set is defined as follows: ； The above-mentioned energy replenishment levels are only preferred embodiments and are not limited to the values ​​mentioned above; The Gaussian Actor network (210) outputs continuous target charge. Then, the action mapping module searches for the nearest energy replenishment level based on the minimum Euclidean distance criterion, the calculation formula of which is: ； in, This represents the continuous target power output by the Actor network; This indicates the index number corresponding to the nearest energy replenishment level; The final mapped actual target power level is represented as follows: ； The action mapping process is used to maintain the continuous optimization capability of the policy network while ensuring that the final output action meets the discrete execution constraints of the vehicle charging equipment, thereby improving the engineering feasibility of the algorithm in actual highway charging scenarios.

7. The method for planning the timing of high-speed refueling for electric vehicles that balances timeliness and economy according to claim 1, characterized in that: In the vehicle refueling environment (320), the system calculates the single-step instantaneous feedback reward based on the current state transition result. The single-step instant feedback reward is used to guide the intelligent agent to comprehensively optimize the charging timeliness, charging cost, and vehicle battery safety margin, and includes a route travel time reward item. Charging economic cost incentive items and battery anxiety reward items The calculation formula is as follows: ； The formula for calculating the route travel time bonus is as follows: ； in, Weighted by time penalty; The time spent waiting in line at the current station; This refers to the actual charging time. Fixed time for operations such as vehicle entry and exit from service areas, parking, plugging and unplugging charging guns, and payment; The formula for calculating the charging economic cost incentive is as follows: ； in, Assign economic cost weight; This is the sum of the electricity cost and service fee incurred during a single charging process; The formula for calculating the battery anxiety reward is: ； in, Penalize battery anxiety with weighting; The vehicle's safe battery level threshold; This indicates the vehicle's current remaining battery power. The aforementioned single-step instant feedback reward adopts a multi-objective weighted design to balance refueling timeliness, economic cost, and vehicle operation safety, providing reward signals for subsequent parameter updates of the policy network.

8. The method for planning the timing of high-speed refueling for electric vehicles that balances timeliness and economy according to claim 7, characterized in that, When the vehicle reaches the finish line, the system calculates the final settlement reward based on the final state. ; When the vehicle's remaining battery power meets When a vehicle is determined to have broken down due to a dead battery, a fixed negative reward will be given. Preferably, in this embodiment, the fixed negative reward is -4000; When the vehicle safely reaches the destination, the final settlement reward consists of two parts: a fixed reward for safe arrival and a penalty for remaining battery power anxiety. The calculation formula is as follows: ； Wherein, 2500 is the preset safe arrival reward in this embodiment. This represents the battery anxiety penalty calculated based on the remaining battery power when the vehicle reaches its destination, and it is designed using a piecewise continuous function. when hour: ； when hour: ； when hour: ； in, Penalize battery anxiety with weighting. This is the preset safe battery level threshold for the vehicle; Preferably, the piecewise function remains continuous at the junctions of each interval to avoid abrupt changes in reward values, thereby reducing gradient oscillations, improving the stability of the policy network training process, and taking into account both vehicle driving safety and the efficiency of utilizing the remaining battery power at the endpoint.

9. The method for planning the timing of high-speed refueling for electric vehicles that balances timeliness and economy according to claim 1, characterized in that: In step 100, the 10-minute high-frequency historical driving time sequence characteristics (110) All data must be normalized by zero-mean (Z-score) before they can be used as input to the pre-trained LSTM network (130); The 32-dimensional hidden state features output by the pre-trained LSTM network (130) The data is further input into a multilayer perceptron (MLP) nonlinear mapping layer for feature transformation to enhance the nonlinear representation capability of historical time-series features; the MLP nonlinear mapping layer includes two fully connected feedforward neural networks.

10. A method for planning the timing of high-speed refueling for electric vehicles that balances timeliness and economy, characterized in that, include: Memory, used to store computer programs; A processor is configured to execute a computer program stored in the memory to implement the method for planning the high-speed refueling timing of electric vehicles based on spatiotemporal multi-source state perception and maximum entropy reinforcement learning as described in any one of claims 1 to 9.

Images