Scheduling method and system for idle energy storage vehicles

Abstract

The invention relates to the technical field of electric vehicle charging and discharging scheduling, in particular to an idle energy storage vehicle scheduling method and system, and the method comprises the steps: obtaining a charging and replacing task or a power grid power transmission task, and determining a target position; selecting idle energy storage vehicles according to vehicle states in a set range of an area where the target position is located; according to the current position, the vehicle speed, the battery state, the mileage information and the idle state information of the idle energy storage vehicles, the power supply capacity is determined, and the idle energy storage vehicle with the minimum distance from the target position is screened out; based on a shortest path algorithm, an optimal path between the idle energy storage vehicles and the target position is obtained, a scheduling instruction is sent to the corresponding idle energy storage vehicles, and the idle energy storage vehicles execute corresponding tasks; and after the task is finished, obtaining the actual charging and switching quantity, the service time and the user feedback score to form a reward function, storing the reward function in historical data, updating the state-action value function during the scheduling period based on the historical data, and enabling the scheduling algorithm to autonomously learn the optimal decision strategy through updating.

Description

Technical Field

[0001] The present invention relates to the technical field of electric vehicle charging and discharging scheduling, and in particular to a scheduling method and system for idle energy storage vehicles. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Electric vehicles are typically charged using charging stations, which have limited coverage and are unevenly distributed. Some existing technologies attempt to use idle electric vehicles as mobile energy storage power stations, dispatching them to charge target electric vehicles in emergency situations. These technologies rely on complex scheduling algorithms, which struggle to handle high-dimensional dynamic variables (such as real-time electricity prices, road conditions, and battery degradation), resulting in weak dynamic scheduling capabilities. Furthermore, the components shared by multiple parties—the power grid, charging operators, and electric vehicle owners—are difficult to standardize, indirectly making it difficult for scheduling algorithms to integrate existing resources. Summary of the Invention

[0004] To address the technical issues presented in the aforementioned background technology, the present invention provides a method and system for dispatching idle energy storage vehicles. This system utilizes idle electric vehicles as mobile energy storage power stations. In emergency situations, these vehicles can be dispatched to provide charging and swapping services for target electric vehicles, or to supply power to the grid to achieve peak load shifting. After the charging and swapping services and power supply services are completed, the dispatching algorithm is optimized based on historical data. The results of each dispatch are used as feedback to continuously update the dispatching algorithm's decision-making strategy and improve its ability to adapt to changing needs.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: A first aspect of the present invention provides a method for dispatching idle energy storage vehicles, comprising the following steps: Obtain charging and swapping tasks or power supply tasks to the power grid and determine the target location; Select idle energy storage vehicles based on the vehicle status within the set range of the target location area; Determine the power supply capacity based on the current location, speed, battery status, mileage information, and idle status information of the idle energy storage vehicles, and select the idle energy storage vehicles with the shortest distance to the target location; Based on the shortest path algorithm, the optimal path between the idle energy storage vehicle and the target location is obtained, and a dispatch instruction is issued to the corresponding idle energy storage vehicle, and the idle energy storage vehicle performs the corresponding task; After the task is completed, the actual charging and swapping power, service time and user feedback score are obtained to form a reward function and saved to historical data. The state-action value function during the scheduling period is updated based on the historical data.

[0006] Furthermore, an idle energy storage vehicle is an electric vehicle that is equipped with an energy storage system and has no tasks to perform or no travel needs within a set time period.

[0007] Furthermore, based on the shortest path algorithm, the optimal path between the idle energy storage vehicle and the target location is obtained, specifically: A graph model is constructed with road intersections and key locations as nodes and road segments as edges between nodes. Each node stores location information, and each edge is associated with road segment information. The starting node represents the current location of the idle energy storage vehicle, the final node represents the target location, and the intermediate nodes represent key locations in the road network. Define the actual cost from the starting node to the intermediate node, which represents the actual driving cost, including at least energy consumption cost and time cost; define the heuristic estimated cost from the intermediate node to the final node, where the estimated cost represents the estimated cost; combine the number of road sections between each node passed by the idle energy storage vehicle, the distance of the road section, the corresponding energy consumption and time, and take the minimum total cost as the goal to obtain the optimal path between the idle energy storage vehicle and the target location.

[0008] Furthermore, the actual charging / swapping power, service time, and user feedback score are obtained to form a reward function, specifically: The ratio of actual charging and swapping power to required power is the reward function of completion, as shown in the following formula: ; The reward function corresponding to the service time is shown below: ; If the service is overtime or not completed, a negative reward will be given. User feedback scores are mapped to a reward function as shown below: ; Instant Rewards ; in, 、 and is the weight coefficient; The actual amount of charge and swap; is the power demand; Rate the user.

[0009] Furthermore, the state-action value function during the scheduling period is updated based on historical data, as shown in the following formula: ； in, Is the current state Take action of value; is the learning rate; Is in state Take action Instant rewards received; is the discount factor; Take action The next state after .

[0010] Furthermore, during the update of the state-action value function, the parameters of the Q value are represented as particles in the particle swarm. Each particle represents a set of possible strategy parameter combinations, and the strategy parameter combinations are updated through multiple iterations. In each iteration, each particle updates its own speed and position based on its own fitness and the fitness information of other particles in the swarm to obtain the adjusted strategy parameter combination.

[0011] Furthermore, during the update of the state-action value function, the revenue of the charging and swapping task, the revenue of the power transmission task to the grid, and the operating cost are calculated to obtain the revenue evaluation of this task, and the reward function is updated. The state-action value function is regularly updated using the updated reward function, specifically: ; in, Represent the weight coefficients of completion, service time, user feedback score and economic benefits respectively, and , The actual charging and swapping capacity, is the power demand, is the actual service time, is the maximum service time allowed, Rate the user, The final economic benefit value of this dispatching task is the sum of the income from the charging and swapping task and the income from the power transmission task to the grid, minus the operating cost.

[0012] A second aspect of the present invention provides a dispatching system for idle energy storage vehicles, comprising: A target location module is configured to: obtain a charging / swapping task or a power supply task to the power grid and determine a target location; An idle vehicle screening module is configured to: select idle energy storage vehicles based on the status of vehicles within a set range of the target location; determine the power supply capacity based on the current location, speed, battery status, mileage information and idle status information of the idle energy storage vehicles, and screen out the idle energy storage vehicles with the shortest distance to the target location; A path planning and scheduling module is configured to: obtain the optimal path between the idle energy storage vehicles and the target location based on the shortest path algorithm, and issue a scheduling instruction to the corresponding idle energy storage vehicles, so that the idle energy storage vehicles perform the corresponding tasks; The decision optimization module is configured to: after the task is completed, obtain the actual charging and swapping power, service time and user feedback score to form a reward function, save it to historical data, and update the state-action value function during the scheduling period based on the historical data.

[0013] A third aspect of the present invention provides a computer program product comprising computer-readable instructions, which, when executed on an electronic device, enables the electronic device to implement the above-mentioned method for dispatching idle energy storage vehicles.

[0014] A fourth aspect of the present invention provides an electronic device comprising at least one processor and a memory connected to the processor, the memory being used to store a computer program; the processor being used to execute the computer program, so that the electronic device can implement the above-mentioned method for dispatching idle energy storage vehicles.

[0015] Compared with the existing technology, one or more of the above technical solutions have the following beneficial effects: 1. Utilize idle electric vehicles as mobile energy storage power stations. In emergency situations, idle electric vehicles can be dispatched to provide charging and swapping services for target electric vehicles, or to supply power to the grid to achieve peak load shifting. After the charging and swapping services and power supply services are completed, the dispatch algorithm is optimized based on historical data. The results of each dispatch are used as feedback information, including actual charging and swapping power, service time, and user feedback scores as reward signals. The dispatch algorithm's decision-making strategy is continuously updated to improve its ability to adapt to different needs and its efficiency in responding to services.

[0016] 2. When planning the optimal route, in addition to considering the minimum distance in physical space, the hidden costs such as the distance, time, and energy consumption of idle vehicles from the initial location to the target location are also taken into account. This can more realistically reflect the impact of traffic conditions on the scheduling algorithm during the service execution period.

[0017] 3. The decision strategy of the scheduling algorithm is based on the state-action value function exists in the form of maintenance Q Table, storing the long-term value of each state-action pair ( Qvalue), enabling the scheduling algorithm to autonomously learn the optimal decision-making strategy and select the most reasonable action (such as which vehicle to dispatch, charging / battery swapping strategy, whether to participate in power supply to the grid, etc.) under different scenario conditions. During the update of the decision-making strategy, a combination of reinforcement learning and particle swarm optimization is adopted. Reinforcement learning can continuously learn and adapt to dynamically changing scheduling needs based on environmental feedback, while particle swarm optimization optimizes and searches for strategy parameters from a global perspective. The synergistic effect of the two enables the scheduling strategy to make optimal decisions more quickly and accurately when facing complex electric vehicle charging and discharging scheduling scenarios, effectively improving the overall performance and robustness of the scheduling system.

[0018] 4. During the scheduling algorithm update, a reward function is established using actual charging and swapping power as a measure of completion. This positively impacts system reliability and prevents the algorithm from leaning towards a "low-risk, low-return" strategy (e.g., dispatching only vehicles within short distances, which results in some demand not being met). User feedback (e.g., wait times and service experience) determines commercial viability, balancing the responsibilities of the power grid, charging operators, and electric vehicle owners. This prevents the algorithm from solely optimizing revenue for the power grid or charging operators while ignoring user experience, which can lead to user churn.

[0019] 5. The reward function includes positive rewards for successfully completing charging and swapping, high user satisfaction (such as short response time), grid peak-shaving benefits (high discharge electricity price) and energy saving (low idle rate). It also includes negative rewards for scheduling failure (vehicle failure to arrive), battery overcharge / over-discharge, delays caused by traffic congestion and excessive operating costs (such as long-distance idle driving). It also includes the benefits generated by each completed charging and swapping task and power transmission task and the system's operating vehicle costs to form an economic benefit evaluation. During the update of the state-action value function of reinforcement learning, the reward function is no longer simply derived from a linear combination of completion, service time and user feedback scores, but an economic benefit indicator is introduced. By regularly analyzing and mining the economic benefit data of historical scheduling tasks, the economic benefit change trend under different scheduling strategy parameter combinations is explored, so that the scheduling algorithm tends to select those scheduling actions that can generate higher economic benefits while meeting requirements such as completion, service time and user satisfaction, thereby achieving a balance between maximizing economic benefits and optimizing scheduling service quality, and alleviating the cost allocation problem among the three parties: the power grid, charging operators and electric vehicle owners. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0021] Figure 1 is a schematic diagram of a dispatching process for idle energy storage vehicles provided by one or more embodiments of the present invention; Figure 2 It is a schematic diagram of the principle of a dispatching system for idle energy storage vehicles provided by one or more embodiments of the present invention. DETAILED DESCRIPTION

[0022] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0023] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0024] It should be noted that the terms used herein are intended only to describe specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0025] Example 1: The idle energy storage vehicle dispatch system presented in this embodiment utilizes real-time data collection, dynamic route optimization (A* algorithm), and reinforcement learning algorithms to build an efficient and intelligent energy storage vehicle dispatch system. This system achieves bidirectional optimization of charging and swapping services and grid emergency power supply, significantly improving resource utilization and dispatch response speed. The A* algorithm dynamically adjusts routes based on real-time road conditions, shortening travel time and reducing vehicle energy consumption. The reinforcement learning algorithm optimizes dispatch strategies while also empowering the system with dynamic adaptability to address sudden changes in road conditions and fluctuations in user demand, thereby improving user satisfaction. Through a precise economic benefit evaluation model, the system reduces operating costs (for example, reducing vehicle idle rates by 15%) while exploring a new profit model for idle vehicles to participate in grid peak regulation. When idle, vehicles serve as distributed energy storage nodes, alleviating peak-time pressure on the grid, indirectly promoting the adoption of new energy and reducing carbon emissions, thereby optimizing the energy mix.

[0026] like Figure 1 As shown, the following embodiment provides a method for dispatching idle energy storage vehicles, including the following steps: Obtain charging and swapping tasks or power supply tasks to the power grid and determine the target location; Select idle energy storage vehicles based on the vehicle status within the set range of the target location area; Determine the power supply capacity based on the current location, speed, battery status, mileage information, and idle status information of the idle energy storage vehicles, and select the idle energy storage vehicles with the shortest distance to the target location; Based on the shortest path algorithm, the optimal path between the idle energy storage vehicle and the target location is obtained, and a dispatch instruction is issued to the corresponding idle energy storage vehicle, and the idle energy storage vehicle performs the corresponding task; After the task is completed, the actual charging and swapping power, service time and user feedback score are obtained to form a reward function and saved to historical data. The state-action value function during the scheduling period is updated based on the historical data.

[0027] Idle energy storage vehicles are electric vehicles equipped with energy storage systems. They can be electric passenger cars or commercial vehicles, or unmanned vehicles operating in a set area (such as unmanned electric vehicles performing logistics distribution within a park).

[0028] The idle state refers to the state in which the vehicle has no tasks to perform or the vehicle owner has no travel needs.

[0029] This example uses unmanned vehicles operating within a park as idle energy storage vehicles to further explain this solution. The vehicles in this example are equipped with energy storage systems (batteries). During daily use, these vehicles can store excess energy in the batteries during periods of inactivity. Due to their widespread distribution and large number, they possess a considerable energy storage capacity and can participate in charging and swapping services and emergency power supply tasks without affecting their normal operation.

[0030] Multiple sensors and communication modules are installed on each idle energy storage vehicle to collect real-time vehicle information. GPS modules accurately determine the vehicle's current location and monitor battery power, voltage, current, temperature, and other information in real time to determine its energy storage and power supply capabilities. The vehicle's onboard computer or other sensors capture information such as speed, direction, and remaining range to determine whether the vehicle is idle or in normal driving condition.

[0031] As a further implementation method, in addition to judging whether the vehicle is in an idle state or a normal driving state based on information such as driving speed, driving direction, and remaining cruising range, the vehicle owner can also actively send an idle state signal. The idle state signal reflects that the current electric vehicle is in an idle state during a set time period and can participate in charging and swapping services and emergency power supply tasks.

[0032] Based on location information, idle energy storage vehicles within a certain range are screened. Based on the battery status information of the screened vehicles, the available power and remaining power are calculated to assess whether they meet the charging needs of vehicles with insufficient power. Taking into account the vehicle's driving status information, priority is given to vehicles that are idle or about to enter idle mode to minimize the impact on normal vehicle use.

[0033] For idle energy storage vehicles within a certain range, the distance between vehicles is calculated using the Euclidean distance formula. and , its plane distance d It can be expressed as: ； in, and Vehicles A and B The latitude and longitude coordinates of .

[0034] Determine idle energy storage vehicles i Can it meet the needs of vehicles with insufficient power? j Charging needs, defining power satisfaction formula: ; in, Idle energy storage vehicles The remaining power available, The vehicle is low on battery If , then the vehicle Theoretically, it can meet the needs of vehicles charging needs.

[0035] After identifying an idle energy storage vehicle that can provide charging and swapping services, the A* shortest path algorithm is used to plan an optimal route to the vehicle with insufficient battery power. Taking into account road traffic conditions and obtaining real-time traffic information based on map information, the route is dynamically adjusted to reduce travel time and energy consumption.

[0036] In the A* algorithm, nodes represent geographical locations in path planning. The starting node represents the current location of an idle energy storage vehicle. The final node represents the target location, i.e., the location of a vehicle with insufficient power or the emergency power supply access point of the power grid. Intermediate nodes represent key locations in the road network (such as intersections and road segmentation points). Through discretization processing, continuous roads are converted into a set of nodes in a graph structure.

[0037] For the graph ,Certainly righteous From the starting node To the intermediate node The actual cost represents the actual driving cost, which includes energy consumption cost and time cost. From the intermediate node To the final node The heuristic estimated cost of node .

[0038] ,in From the starting point to the middle node The actual cost, From the starting point to the middle node Number of road sections traversed; For the The energy consumption per unit distance of a road section is related to factors such as road slope, vehicle type, and driving speed; For the the distance of the road section; is the congestion time correction factor; For the vehicle The travel time on each road segment.

[0039] , From the intermediate node Heuristic estimate of cost to the endpoint; The average energy consumption per unit distance is an average energy consumption value obtained based on the vehicle's historical driving data or statistical information; For the intermediate node The straight-line distance to the end point; The speed limit for a road is determined according to the type and regulations of the road.

[0040] In this embodiment, path planning is implemented by the A* algorithm, and the specific steps are as follows: 1. Graph Model Construction: Build a graph model with road intersections and key locations as nodes and road segments as edges. Each node stores its geographic coordinates, and each edge is associated with the corresponding road segment, including distance, slope, speed limit, and real-time traffic conditions. This is used to calculate energy and time costs.

[0041] 2. Initial cost setting: In the initial state, a basic map model is constructed based on electronic map data and historical average road conditions, and is updated in real time. The actual cost from the starting node to the intermediate node represents the actual driving cost, including energy consumption cost and time cost. The calculation formula is: ; in, is the energy consumption rate of the vehicle under different road conditions, is the road distance, is the travel time, is the time cost coefficient.

[0042] 3. Heuristic cost estimation: The heuristic cost estimation from the intermediate node to the final node is based on the straight-line distance between the intermediate node and the final node, the average speed limit of the road, and the average energy consumption rate of the vehicle. The calculation formula is: ; in, is the average energy consumption rate of the vehicle, is the straight-line distance between the intermediate node and the final node, is the average speed limit for the road.

[0043] 3. Node expansion and path update: During the execution of the algorithm, the nodes in the open list are updated according to The total cost of the nodes is sorted, and each time the node with the smallest total cost is selected for expansion, and its adjacent nodes are explored. For each expanded node, its actual cost and heuristic estimated cost are recalculated, and its optimal predecessor node in the path is updated.

[0044] 4. Path determination: When the final node is added to the open list, the algorithm terminates the search. The optimal path is obtained by backtracking the optimal predecessor sequence from the final node to the starting node.

[0045] 5. Dynamic update: After updating the relevant cost function, if there is a change in road conditions or vehicle status update, restart the path planning algorithm and search for a new optimal path in the graph model based on the latest cost function value.

[0046] Determine the charging and swapping strategy based on the power requirements of low-charge vehicles and the remaining power of idle energy storage vehicles, including charging power, charging duration, and battery swapping options (if supported). Consider the battery compatibility of different vehicles to ensure the safety and efficiency of the charging and swapping process.

[0047] According to the power demand of the vehicle with insufficient power , the maximum charging time allowed and the remaining power of idle energy storage vehicles Determine charging power : ； in, is the maximum charging power of an idle energy storage vehicle.

[0048] When a vehicle on the road is detected to be low on battery, the system initiates a dispatch to locate a nearby idle energy storage vehicle. The selected idle energy storage vehicle is dispatched to the location of the low-battery vehicle and provides charging and swapping services based on the established charging and swapping strategy. Taking into account vehicle mobility, the system continuously monitors service progress during the charging and swapping process to ensure the safety and integrity of the service, such as preventing overcharging and over-discharging.

[0049] When the grid load is at its peak, the system can dispatch energy storage vehicles to supply electricity to the grid to supplement the power demand and achieve peak shaving and valley filling. Specifically: by evaluating the power supply and demand situation, it determines whether it is necessary to call energy storage vehicles for discharge, and then uses intelligent scheduling algorithms to optimize the discharge of energy storage vehicles. Then, the energy storage vehicles provide power support to the grid or load. In order to ensure the safety and stability of the energy storage vehicle discharge process, the system adopts multiple protection mechanisms, including battery protection, load balancing protection, and inverter protection.

[0050] After charging and swapping services or power transmission to the grid are completed, the dispatching algorithm is optimized based on historical dispatch data using machine learning algorithms. Using a reinforcement learning algorithm, the results of each dispatch are used as feedback, including the timeliness of the dispatch, the completion of the charging and swapping service, and user satisfaction as reward signals. The algorithm's decision-making strategy is continuously updated to improve its adaptability and efficiency.

[0051] The decision-making strategy of the algorithm is the policy function used to select the scheduling action in the reinforcement learning model, which is usually expressed as a mapping from state to action. Common forms include state-action value function based on value function and policy-based parameterized policy networks .

[0052] In the value function method, the value with the maximum value is selected according to the current state. The action of the value is executed to construct the optimal strategy. In the implementation, the parameters that need to be updated include Q Entries in the value table or weights of the policy network Etc., as well as the corresponding formulas for parameters in the value function or policy function (such as learning rate, discount factor, etc.).

[0053] Dispatching idle vehicles for charging and swapping batteries and supplying power to the grid is essentially a resource scheduling task. The dispatcher can be considered an RL agent, the scheduling system the environment, scheduling decisions (such as task allocation and execution order selection) as actions, and system performance (such as completion time and resource utilization) as reward feedback. In other words, reinforcement learning strategies optimize scheduling decisions by continuously adjusting these parameters.

[0054] The completion rate is the ratio of the actual charging and replacement power to the required power. ,in, 、 and is the weight coefficient, reflecting the importance of completion; The actual amount of charge and swap; The power demand.

[0055] If the service is overtime or not completed, a negative reward will be given. .

[0056] Directly map user feedback ratings (e.g. 1-5 points) to rewards ,in, Rate the user.

[0057] Combining the above indicators into the immediate reward of reinforcement learning, , by adjusting the weights 、 and ,The algorithm can dynamically optimize the scheduling strategy to prioritize,meeting key indicators.

[0058] Dynamic optimization scheduling strategy. Specifically, after obtaining the description of the reward function based on the actual charging and swapping power, service time, and user feedback score, the dynamic optimization scheduling strategy is executed, including the following steps: 1. Initial weight setting: In the initial state, the weight coefficient 、 and The setting of can determine a reasonable initial value range based on historical data statistics and expert experience. , =0.3, =0.3.

[0059] 2. Contribution evaluation: After each scheduling task is completed, calculate the contribution of the completion rate, service time, and user feedback score of the task to the final scheduling effect. The contribution evaluation can be determined by comparing the deviation between the actual indicators and the historical average indicators. The calculation formula is: ; in, For this task The actual value of the indicator, is the historical average.

[0060] 3. Weight adaptive adjustment: Based on the contribution evaluation results, if the actual value of a certain indicator is significantly better than the historical average, it is considered that the indicator has a higher positive contribution in this task, and its weight coefficient is increased accordingly; conversely, if the actual value is significantly worse than the historical average, its weight coefficient is appropriately reduced. The weight adjustment formula is: ; in, To adjust the step size, is the weight adjustment amount, and .

[0061] 4. Weight adjustment mechanism: Establish a weight adaptive adjustment mechanism that can automatically make adaptive adjustments as the scheduling environment and task requirements change, without relying entirely on manual experience. This makes the scheduling strategy more scientific and reasonable to reflect the importance of different indicators in the actual scheduling process, thereby improving the performance and adaptability of the scheduling algorithm.

[0062] This embodiment combines the reinforcement learning algorithm and the particle swarm optimization algorithm and applies them to the intelligent dispatching system of idle energy storage vehicles. The dispatching algorithm is optimized based on historical dispatching data: ； in, Is the current state Take action of value; is the learning rate, which determines how quickly new information updates old information （ ） ; Is in state Take action The instant rewards obtained are obtained through the above; is the discount factor, which determines the importance of future rewards （ ） ; Take action The next state after .

[0063] In order to further improve the performance and adaptability of the scheduling strategy, a hybrid algorithm mechanism combining the particle swarm optimization (PSO) algorithm and reinforcement learning is introduced.

[0064] The parameters of the state-action value function (Q-value) in reinforcement learning are represented as particles in a particle swarm, with each particle representing a set of possible policy parameter combinations. During each iteration, particles in the swarm update their own speed and position based on their own fitness and the fitness information of other particles in the swarm, that is, adjust the policy parameter combination. The speed and position update formulas for particle swarm optimization are: ; in, is the inertia weight, and is the acceleration constant, and is a random number, is the optimal position of an individual, is the global optimal position.

[0065] Synergy: Through the intelligent search capabilities of the particle swarm, it is possible to quickly find more optimal policy parameter regions within a larger solution space, providing better initial policy parameters or parameter adjustment directions for reinforcement learning, thereby accelerating the convergence process of reinforcement learning and improving learning efficiency. At the same time, during the exploration process of reinforcement learning, when encountering complex and changing scheduling scenarios or falling into a local optimal solution, the particle swarm optimization auxiliary mechanism is triggered, leveraging the global search capabilities of the particle swarm to help reinforcement learning escape the local optimal solution and further optimize the scheduling strategy.

[0066] Advantages of the hybrid algorithm: This hybrid algorithm model that combines reinforcement learning and particle swarm optimization fully leverages the advantages of both algorithms. Reinforcement learning can continuously learn and adapt to dynamically changing scheduling needs based on environmental feedback, while particle swarm optimization optimizes and searches for strategy parameters from a global perspective. The synergistic effect of the two enables the scheduling strategy of the present invention to make optimal decisions more quickly and accurately when facing complex electric vehicle charging and discharging scheduling scenarios, effectively improving the overall performance and robustness of the scheduling system.

[0067] The dispatching system's economic benefits are evaluated. For road vehicle charging and swapping services, revenue is calculated based on the amount of electricity used and the service fee standard. For grid emergency power supply, revenue is calculated based on the amount of electricity delivered to the grid and the electricity price. Operating costs are also calculated, including vehicle driving costs, battery depreciation costs, and communication costs.

[0068] Economic benefit evaluation: For road vehicle charging and swapping services, the revenue is calculated based on the amount of charging and swapping and the service fee standard. The formula is: ; in, is the income from charging and swapping services, It is the amount of electricity used for charging and replacing batteries. It is the unit price of charging and swapping batteries.

[0069] For emergency power supply to the grid, the profit is calculated based on the amount of electricity delivered to the grid and the electricity price. The formula is: ; in is the revenue from supplying electricity to the grid, is the amount of electricity delivered to the grid. It is the electricity purchase price of the power grid.

[0070] Calculate operating costs, including vehicle driving costs, battery depreciation costs, communication costs, etc. The formula is: .

[0071] The final economic benefits are: , that is, the sum of the revenue generated by the charging and swapping services and the grid emergency power supply period, minus the operating costs.

[0072] After obtaining the description of economic benefit evaluation, the scheduling algorithm is updated as follows: 1. Economic benefit feedback: After each dispatch task is completed, in addition to calculating economic benefit indicators such as the charging and swapping service revenue, power supply revenue to the grid, and operating costs of this task, these economic benefit data are also incorporated into the optimization process of the dispatch algorithm as important feedback information.

[0073] 2. Comprehensive reward signal: The final economic benefit value is used as a key component of the reward function in the reinforcement learning algorithm. Together with indicators such as completion rate, service time, and user feedback score, it constitutes a comprehensive reward signal to evaluate the quality of scheduling decisions.

[0074] 3. Redefinition of the immediate reward function: In the state-action value function update formula of reinforcement learning, the immediate reward R is no longer simply derived from a linear combination of completion rate, service time, and user feedback score. Instead, an economic benefit indicator is introduced, and the immediate reward function is redefined as follows: ; in, Represent the weight coefficients of completion, service time, user feedback score and economic benefits respectively, and , The actual charging and swapping capacity, is the power demand, is the actual service time, is the maximum service time allowed, Rate the user, The final economic benefit value of this scheduling task (i.e. ).

[0075] 4. Parameter Correlation and Optimization: Build a correlation model between economic benefits and dispatch strategy parameters. Regularly analyze and mine the economic benefit data of historical dispatch tasks to explore economic benefit trends under different dispatch strategy parameter combinations. Based on the analysis results, make targeted adjustments and optimizations to key parameters in the dispatch algorithm (such as vehicle screening criteria, path planning weights, and charging and discharging strategy parameters) to further improve the dispatch system's economic performance.

[0076] 5. Strategy adjustment: In subsequent scheduling tasks, the algorithm will tend to select scheduling actions that can generate higher economic benefits while meeting requirements such as completion rate, service time, and user satisfaction, thereby achieving a balance between maximizing economic benefits and optimizing scheduling service quality.

[0077] This solution monitors vehicle status (battery level, location, and driving status) in real time to screen dispatchable energy storage vehicles and avoid idle resources. It also uses the A* algorithm to optimize routes, reduce idle mileage, improve dispatch efficiency, and reduce energy consumption. During peak electricity consumption periods, it dispatches idle vehicles to supply power to the grid, alleviating pressure on the grid and increasing the absorption rate of new energy.

[0078] During dispatch, the collected data such as GPS, battery status and road condition information can reuse the existing sensors on the vehicle, so as to quickly match the required vehicle with the energy storage vehicle and shorten the waiting time; the collected battery information automatically adjusts the plan according to battery compatibility, charging power and battery replacement requirements to ensure safety and efficiency; after the service is completed, reinforcement learning is used to continuously optimize the dispatch strategy to adapt to emergencies (such as traffic jams and demand surges).

[0079] The core goal of using reinforcement learning is to enable the dispatching algorithm to autonomously learn the optimal decision-making strategy, that is, to select the most reasonable action (such as which vehicle to dispatch, charging / swapping strategy, whether to participate in V2G discharge, etc.) under different environmental conditions (such as vehicle location, battery level, road conditions, grid load, etc.) to maximize long-term benefits (such as reducing operating costs, improving user satisfaction, and increasing grid peak-shaving benefits).

[0080] During reinforcement learning, the state Real-time vehicle data (location, battery level, temperature, driving status, etc.), grid demand (peak and valley hours, electricity price fluctuations), road condition information (congestion, accidents, weather), and user needs (location and urgency of vehicles with low battery) action For scheduling decisions: selecting which idle energy storage vehicle to provide service; path planning: dynamically adjusting the weight of the A* algorithm (such as shortest time vs. lowest energy consumption); charging and battery swapping strategies: charging power, battery swapping routes; V2G decision-making: whether to allow the vehicle to discharge to the grid.

[0081] By combining reinforcement learning with particle swarm optimization, the advantages of both algorithms are fully utilized. Reinforcement learning can continuously learn and adapt to dynamically changing scheduling needs based on environmental feedback, while particle swarm optimization optimizes and searches for strategy parameters from a global perspective. The two work together, enabling the updated scheduling strategy to make optimal decisions more quickly and accurately when facing complex electric vehicle charging and discharging scheduling scenarios, effectively improving the overall performance and robustness of the scheduling system.

[0082] The reward function includes positive rewards (encouraging behavior), successful completion of charging and battery swapping, high user satisfaction (such as short response time), grid peak-shaving benefits (high discharge electricity prices), energy saving (low idle driving rate), and negative rewards (punishing behavior): scheduling failure (vehicle failure to arrive), battery overcharge / over-discharge, delays caused by traffic congestion, and excessive operating costs (such as long-distance idle driving).

[0083] Completion (e.g., successful charging and swapping) directly impacts system reliability. Ignoring this aspect can lead algorithms to adopt low-risk, low-return strategies (e.g., dispatching only vehicles within short distances, leading to some demand not being met). User feedback (e.g., wait times and service experience) determines commercial viability. If algorithms prioritize grid revenue while ignoring user experience, long-term user churn will result.

[0084] By establishing a correlation model between economic benefits and dispatch strategy parameters, we regularly analyze and mine the economic benefit data of historical dispatch tasks to explore the changing trends of economic benefits under different dispatch strategy parameter combinations. Based on the analysis results, we make targeted adjustments and optimizations to key parameters in the dispatch algorithm (such as vehicle screening conditions, path planning weights, and charging and discharging strategy parameters), further improving the dispatch system's economic performance. In subsequent dispatch tasks, the algorithm will tend to select dispatch actions that generate higher economic benefits while meeting requirements such as completion rate, service time, and user satisfaction, thereby achieving a balance between maximizing economic benefits and optimizing dispatch service quality.

[0085] Therefore, the value function method is used to update the scheduling strategy. By maintaining the Q table, the long-term value (Q value) of each state-action pair is stored. After each scheduling service is completed, the function can be updated. This method is suitable for discrete action spaces (such as "dispatch car A or car B") and simplifies the maintenance of the optimization algorithm, reducing the cost of manual intervention.

[0086] As data accumulates, dispatch algorithms can identify hidden patterns (such as a surge in demand in a certain area during the morning rush hour). Furthermore, without the need for manual weighting, the algorithm can automatically balance multiple demands. It can also make reliable decisions in unexpected situations (such as traffic accidents and power grid failures) and adapt to uncertainty.

[0087] Example 2: The dispatch system for idle energy storage vehicles includes: A target location module is configured to: obtain a charging / swapping task or a power supply task to the power grid and determine a target location; An idle vehicle screening module is configured to: select idle energy storage vehicles based on the status of vehicles within a set range of the target location; determine the power supply capacity based on the current location, speed, battery status, mileage information and idle status information of the idle energy storage vehicles, and screen out the idle energy storage vehicles with the shortest distance to the target location; A path planning and scheduling module is configured to: obtain the optimal path between the idle energy storage vehicles and the target location based on the shortest path algorithm, and issue a scheduling instruction to the corresponding idle energy storage vehicles, so that the idle energy storage vehicles perform the corresponding tasks; The decision optimization module is configured to: after the task is completed, obtain the actual charging and swapping power, service time and user feedback score to form a reward function, save it to historical data, and update the state-action value function during the scheduling period based on the historical data.

[0088] Example 3: A computer program product includes computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the above-mentioned method for dispatching idle energy storage vehicles.

[0089] Example 4: An electronic device includes at least one processor and a memory connected to the processor, the memory is used to store a computer program; the processor is used to execute the computer program, so that the electronic device can implement the above-mentioned scheduling method for idle energy storage vehicles.

[0090] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for dispatching idle energy storage vehicles, characterized in that: The following steps are involved: Obtain charging and swapping tasks or power supply tasks to the power grid and determine the target location; Select idle energy storage vehicles based on the vehicle status within the set range of the target location area; Determine the power supply capacity based on the current location, speed, battery status, mileage information, and idle status information of the idle energy storage vehicles, and select the idle energy storage vehicles with the shortest distance to the target location; Based on the shortest path algorithm, the optimal path between the idle energy storage vehicle and the target location is obtained, and a dispatch instruction is issued to the corresponding idle energy storage vehicle, and the idle energy storage vehicle performs the corresponding task; After the task is completed, the actual charging and swapping power, service time and user feedback score are obtained to form a reward function and saved to historical data. The state-action value function during the scheduling period is updated based on the historical data.

2. The method for dispatching idle energy storage vehicles according to claim 1, characterized in that: Idle energy storage vehicles are electric vehicles that are equipped with energy storage systems and have no tasks to perform or no travel needs within a set time period.

3. The method for dispatching idle energy storage vehicles according to claim 1, characterized in that: Based on the shortest path algorithm, the optimal path between the idle energy storage vehicle and the target location is obtained, specifically: A graph model is constructed with road intersections and key locations as nodes and road segments as edges between nodes. Each node stores location information, and each edge is associated with road segment information. The starting node represents the current location of the idle energy storage vehicle, the final node represents the target location, and the intermediate nodes represent key locations in the road network; Define the actual cost from the starting node to the intermediate node. The actual cost represents the actual driving cost, which at least includes energy consumption cost and time cost; Heuristic estimated cost from the intermediate node to the final node, the estimated cost represents the estimated cost; Combining the number of road sections between nodes passed by idle energy storage vehicles, the distance between the sections, the corresponding energy consumption and time, and taking the minimum total cost as the goal, the optimal path between the idle energy storage vehicle and the target location is obtained.

4. The method for dispatching idle energy storage vehicles according to claim 1, characterized in that: The actual charging and swapping capacity, service time, and user feedback scores are obtained to form a reward function, specifically: The ratio of actual charging and swapping power to required power is the reward function of completion, as shown in the following formula: ； The reward function corresponding to the service time is shown below: ； If the service is overtime or not completed, a negative reward will be given. User feedback scores are mapped to a reward function as shown below: ； Instant Rewards ; in, 、 and is the weight coefficient; The actual amount of charge and swap; is the power demand; Rate the user.

5. The method for dispatching idle energy storage vehicles according to claim 1, characterized in that: The state-action value function during scheduling is updated based on historical data as shown in the following formula: ； in, Is the current state Take action of value; is the learning rate; Is in state Take action Instant rewards received; is the discount factor; Take action The next state after .

6. The method for dispatching idle energy storage vehicles according to claim 1, characterized in that: During the update of the state-action value function, the parameters of the Q value are represented as particles in the particle swarm. Each particle represents a set of possible strategy parameter combinations, and the strategy parameter combinations are updated through multiple iterations. In each iteration, each particle updates its own speed and position based on its own fitness and the fitness information of other particles in the swarm to obtain the adjusted strategy parameter combination.

7. The method for dispatching idle energy storage vehicles according to claim 1, characterized in that: During the update of the state-action value function, the revenue of the charging and swapping task, the revenue of the power transmission task to the grid, and the operating cost are calculated to obtain the revenue evaluation of this task, and the reward function is updated. The state-action value function is regularly updated using the updated reward function, specifically: ； in, Represent the weight coefficients of completion, service time, user feedback score and economic benefits respectively, and , The actual charging and swapping capacity, is the power demand, is the actual service time, is the maximum service time allowed, Rate the user, The final economic benefit value of this dispatching task is the sum of the income from the charging and swapping task and the income from the power supply task to the grid, minus the operating cost.

8. A dispatching system for idle energy storage vehicles, characterized in that: include: A target location module is configured to: obtain a charging / swapping task or a power supply task to the power grid and determine a target location; An idle vehicle screening module is configured to select idle energy storage vehicles based on the status of vehicles within a set range of the target location area; Determine the power supply capacity based on the current location, speed, battery status, mileage information, and idle status information of the idle energy storage vehicles, and select the idle energy storage vehicles with the shortest distance to the target location; A path planning and scheduling module is configured to: obtain the optimal path between the idle energy storage vehicles and the target location based on the shortest path algorithm, and issue a scheduling instruction to the corresponding idle energy storage vehicles, so that the idle energy storage vehicles perform the corresponding tasks; The decision optimization module is configured to: after the task is completed, obtain the actual charging and swapping power, service time and user feedback score to form a reward function, save it to historical data, and update the state-action value function during the scheduling period based on the historical data.

9. A computer program product, characterized in that The method comprises computer-readable instructions, which, when executed on an electronic device, enable the electronic device to implement the method for dispatching idle energy storage vehicles as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: The electronic device comprises at least one processor and a memory connected to the processor, the memory being used to store a computer program; the processor being used to execute the computer program, so that the electronic device can implement the method for dispatching idle energy storage vehicles as described in any one of claims 1 to 7.

Images