Shared bicycle dispatching variable VRP search method based on deep reinforcement learning driving

CN121936782BActive Publication Date: 2026-10-09UNIV OF ELECTRONICS SCI & TECH OF CHINA
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202511943638.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-12-22
Publication Date
2026-10-09
Estimated Expiration
2045-12-22

AI Technical Summary

Technical Problem

[0007]本发明的目的是提供一种基于深度强化学习驱动的共享单车调度可变VRP搜索方法,以解决现有技术在不同城市和时段的动态场景下自适应能力差的技术问题

Benefits of technology

本发明突破了传统调度模型中“需求必须完全满足”的刚性限制,显著提升了系统的净收益与核心服务水平。 针对现有技术在资源有限时因强制全量满足需求而导致无解或运输成本过高的问题,本发明通过构建可变VRP数学模型及引入实际操作量决策变量,允许算法根据边际效益智能地“取舍”。这一技术手段使得调度资源能够自动向高价值、高热度站点倾斜,在有限的时间和运力约束下,实现了系统总体净收益的最大化。这不仅大幅提升了重点区域的需求满足率,还有效增加了单车的周转效率,彻底解决了供需失衡场景下的调度难题。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121936782B_ABST
    Figure CN121936782B_ABST
Patent Text Reader

Abstract

The application discloses a shared bicycle scheduling variable VRP search method based on deep reinforcement learning driving, comprising the following steps: S1, a variable VRP mathematical model considering space-time resource constraints is constructed, the constraint conditions met by the variable VRP mathematical model include one or more of the following: variable traffic volume constraint, flow conservation constraint, vehicle load limit and time cost coupling constraint of travel time and operation time; S2, a deep reinforcement learning interactive environment based on ALNS is constructed; S3, a Double DQN intelligent agent is built and trained; S4, a double-layer nested search strategy is used to perform online topology-traffic volume nested search; S5, a constraint penalty term is introduced into the objective function, and softening and processing of multiple complex constraints are performed. The application has excellent generalization performance, can be directly extended and applied to various logistics scenes such as express terminal distribution, material emergency scheduling and urban garbage recycling, and has a broad industrial application prospect.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and in particular to a variable VRP search method for shared bicycle scheduling based on deep reinforcement learning. Background Technology

[0002] With the widespread adoption of bike-sharing systems, the distribution of bikes in cities exhibits significant spatial and temporal imbalances (such as the tidal effect during rush hours), leading to bike accumulation at some stations while shortages occur at others, severely impacting user experience. To address this issue, operators typically need to utilize limited fleets for bike rebalancing between stations.

[0003] Current shared bicycle dispatching and route planning technologies mainly include the following categories: 1. Heuristic algorithms based on the traditional Vehicle Routing Problem (VRP) model. Represented by genetic algorithms and simulated annealing. For example, a shared bicycle scheduling method based on a genetic algorithm models the scheduling problem as the traditional Capacity-Constrained Vehicle Routing Problem (CVRP), optimizing routes through chromosome encoding and crossover mutation. The drawback of existing technologies is that they typically assume that the demand at all stations must be "fully satisfied" (i.e., rigid demand). However, in actual operation, scheduling resources (number of vehicles, load capacity, operating hours) are often limited. When resources are insufficient to cover all demand, this rigid assumption can lead to the algorithm failing to find a feasible solution, or causing a sharp increase in transportation costs to meet the limited demand at peripheral stations, thus failing to maximize system efficiency with limited resources.

[0004] 2. Adaptive Large Neighborhood Search (ALNS) Algorithm. ALNS is currently the mainstream method for solving the VRP problem. It iteratively optimizes by removing some stations through disruptive operators and then reinserting them through repair operators. Existing ALNS algorithms typically rely on manually designed rules or simple historical statistics to adjust operator weights. The drawback of existing technologies is that this parameter adjustment mechanism based on human experience or simple rules has poor adaptability. Faced with complex and ever-changing scheduling scenarios involving different city layouts and different time periods (such as morning rush hour and off-peak hours), the algorithm struggles to automatically switch to the optimal search strategy, easily gets trapped in local optima, and relies on a tedious manual parameter tuning process.

[0005] 3. Deep Learning-Based Path Planning Methods. In recent years, deep learning (such as attention models) has been used to directly generate paths or assist traditional algorithms. The drawback of existing technologies is that current deep learning-based VRP methods primarily focus on optimizing "routing" while neglecting the fine-grained decision-making regarding "volume." In shared bicycle scheduling, the loading and unloading capacity of each station should be a variable decision variable (i.e., allowing for "partial satisfaction"), but existing technologies lack the ability to jointly optimize "path selection" and "capacity allocation," resulting in an inability to improve the overall service level of the system by sacrificing some secondary demands when resources are limited.

[0006] 4. Handling Mechanisms Under Multiple Constraints. Actual scheduling faces strict limitations regarding vehicle load, working time windows, and geofencing (regional division). The drawback of existing technologies is that traditional methods often require adding numerous penalty terms or complex repair logic when handling these multiple complex constraints, leading to a significant decrease in the algorithm's search efficiency and making it difficult to meet the timeliness requirements of real-time scheduling. Summary of the Invention

[0007] The purpose of this invention is to provide a variable VRP search method for shared bicycle scheduling based on deep reinforcement learning, so as to solve the technical problem of poor adaptability of existing technologies in dynamic scenarios of different cities and time periods.

[0008] This invention is achieved using the following technical solution: a variable VRP search method for shared bicycle scheduling based on deep reinforcement learning, comprising the following steps: S1: Construct a variable VRP mathematical model that considers spatiotemporal resource constraints. The constraints satisfied by the variable VRP mathematical model include one or more of the following: variable transport capacity constraints, flow conservation constraints, vehicle load limits, and time cost coupling constraints of travel time and operation time. S2: Construct a deep reinforcement learning interactive environment based on ALNS, including state space, action space, and reward function; S3: Build and train the Double DQN agent; S4: Employ a two-level nested search strategy to perform online topology-capacity nested search; S5: Introduce a constraint penalty term into the objective function to soften and process multiple complex constraints.

[0009] Furthermore, step S1 includes the following sub-steps: S11: Abstract the shared bicycle rebalancing scenario into a directed graph model. ,in This represents a collection of nodes that includes the repository and all potential task sites. The set of arcs representing the connecting nodes; S12: Define mixed decision variables: one type is binary path selection variables, used to decide vehicles. Has it passed through an arc? One type establishes the topology; the other type consists of continuous operational variables used for vehicle decision-making. On the site The actual number of installations and uninstalls determines the service depth; S13: Construct a mixed integer programming model with the goal of maximizing the system's net profit.

[0010] Furthermore, the objective function of the mixed-integer programming model includes: Service revenue: The sum of the products of the actual demand met at each site and the value of the demand per unit at that site; Transportation costs: fuel or electricity costs incurred due to vehicle travel distance, and fixed costs associated with vehicle startup; Penalties for breach of contract: Penalties for violating soft time windows or driving beyond the designated boundaries.

[0011] Furthermore, the state space is described by multi-dimensional feature vectors, specifically including one or more of the following: search process features, solution quality features, resource utilization features, and demand satisfaction features.

[0012] Furthermore, the action space is a set of heuristic operator pairs, specifically including: selecting which destruction operator and selecting which repair operator.

[0013] Furthermore, the reward function is an instant reward mechanism based on net profit increment. If the solution generated by the current action is better than the historical best solution, a large positive reward is given; if the solution is improved but does not break the best, a small positive reward is given; if the solution quality declines, zero reward or negative penalty is given according to the simulated annealing criterion, thereby guiding the agent to converge quickly to the high-quality solution space.

[0014] Furthermore, step S3 includes the following sub-steps: S31: Adopting the DDQN architecture, a deep neural network is built as the decision-making agent; S32: An offline strategy is used to train the agent. The training process includes: experience playback, exploration and utilization, and parameter updates.

[0015] Furthermore, step S4 specifically involves: employing a double-nested search strategy to decouple the complex mixed-integer programming problem. Outer layer: Controlled by a trained DRL agent, the agent selects the optimal destruction and repair operators in real time based on the current state, removes and re-inserts nodes in the current path scheme, dynamically adjusts the access order of vehicles, and generates candidate path skeletons. Inner layer: For each fixed path generated by the outer layer, run the greedy allocator algorithm to calculate the marginal revenue of each station on the path and prioritize the needs of high-value stations.

[0016] Furthermore, the constraint penalty includes: geofencing constraint processing, which uses the ray method to determine whether the coordinates of all stations in the vehicle path are located within the preset work area polygon. For stations that cross the boundary, the shortest Euclidean distance to the boundary is calculated, and a non-linear boundary penalty cost is applied, forcing the search algorithm to automatically correct the path to the compliant area.

[0017] Furthermore, the constraint penalty term includes: working time constraint processing, real-time accumulation of vehicle driving time and node operation time, and if the total time exceeds a preset threshold, a penalty term proportional to the timeout duration is added to the objective function.

[0018] The beneficial effects of this invention are as follows: This invention breaks through the rigid constraint of "demand must be fully satisfied" in traditional scheduling models, significantly improving the system's net benefit and core service level. Addressing the problem that existing technologies, when resources are limited, lead to unsolvable problems or excessively high transportation costs due to mandatory full demand satisfaction, this invention constructs a variable VRP mathematical model and introduces actual operational quantity decision variables, allowing the algorithm to intelligently "make trade-offs" based on marginal benefits. This technique enables scheduling resources to automatically tilt towards high-value, high-demand sites, maximizing the overall net benefit of the system under limited time and capacity constraints. This not only significantly improves the demand satisfaction rate in key areas but also effectively increases the turnover efficiency of individual vehicles, completely solving the scheduling problem in scenarios of supply-demand imbalance.

[0019] This invention achieves intelligent adaptive switching of search strategies, greatly improving solution efficiency and algorithm robustness. Addressing the shortcomings of traditional heuristic algorithms—relying on human experience to design rules, struggling to adapt to changing scenarios, and easily getting trapped in local optima—this invention utilizes a deep reinforcement learning (DRL) agent to replace fixed rules. The agent can perceive the search state in real time and dynamically adjust the combination strategy of destruction and repair operators. This innovation gives the algorithm extremely strong environmental adaptability, enabling it to converge quickly to high-quality solutions and significantly shortening the computation time for large-scale problems. Simultaneously, the algorithm's dependence on the initial solution is greatly reduced, exhibiting extremely high stability and ensuring the stable output of high-quality scheduling schemes across different time periods (such as peak and off-peak periods).

[0020] This invention enhances the ability to handle complex real-world constraints, ensuring high resource utilization and strong executability of scheduling schemes. Addressing the problem that existing methods often fail to implement routes due to neglecting the linear relationship between loading and unloading times or geofencing limitations, this invention proposes an online nested search framework of "topology-capacity" and a soft constraint penalty mechanism. By decoupling and optimizing route planning and capacity allocation, and introducing strict physical constraint processing, the generated schemes are highly consistent with actual operational scenarios. This significantly improves vehicle load utilization and working time utilization, effectively reduces vehicle idling and idle waste, and ensures the accurate execution of scheduling instructions in actual operations.

[0021] This invention brings significant comprehensive economic benefits and broad social value. By aiming to maximize the system's net revenue, this method increases vehicle operating revenue while reducing operating costs such as fuel, labor, and maintenance. At the social level, optimized route planning reduces unnecessary mileage, helps alleviate urban traffic congestion, reduces carbon emissions, and improves the travel experience for citizens. Furthermore, the model architecture of this method has excellent generalization performance and can be directly extended to various logistics scenarios such as last-mile delivery, emergency material dispatching, and urban waste recycling, demonstrating broad prospects for industrial applications. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0023] Figure 1 This is a flowchart of the present invention; Figure 2 This is a schematic diagram of the invention; Figure 3 This is a schematic diagram of the path planning. Figure 4 This is a diagram illustrating the statistical indicators of the overall effect. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0025] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0026] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0027] See Figures 1 to 4 A variable VRP search method for shared bicycle scheduling based on deep reinforcement learning includes the following steps: Step S1: Construct a variable VRP mathematical model; Step S2: Construct a deep reinforcement learning environment; Step S3: Build and train the DRL agent; Step S4: Perform an online nested search; Step S5: Handling multiple constraints.

[0028] In this embodiment, step S1 specifically involves: First, abstracting the shared bicycle scheduling problem into a graph model. ,in For a set of nodes, Representing the car park, Represents a collection of task sites. This represents the set of arcs connecting nodes. To accurately describe the scheduling scenario, the following key parameters are defined: Site Unit service rewards, value range Yuan / vehicle, with a preferred value of 2.0 yuan / vehicle. Site The ideal demand, Indicates the need to pick up the car. Indicates the need to return the car, with a range of values. Vehicles. :node To the node Distance, range of values rice. :node To the node Travel time, range of values Second. Fixed parking time for each service, with a range of values. The preferred value is 60 seconds. Unit handling time, range of values The optimal value is 3 seconds per vehicle. Maximum capacity of the vehicle, range of values The preferred value is 50 vehicles. Maximum operating time of the vehicle, range of values The preferred value is 3600 seconds. Unit distance transportation cost, range of values Yuan / meter, with a preferred value of 0.1 yuan / meter. :vehicle The set of polygon coordinates for the work area. :big A constant used for linearization constraints, with a range of values. The preferred value is .

[0029] To overcome the rigid requirements of traditional VRP, the following decision variables are defined: : A 0-1 integer variable representing a vehicle Has it passed through an arc? , used to determine the path topology. Continuous variable, representing vehicles On the site The actual number of operations (positive for picking up a vehicle and negative for returning it) is used to determine the service depth. Continuous variable, representing vehicles Reaching the node The time. Continuous variable, representing vehicles Leave node Load capacity at that time. : Integer variable, representing the vehicle Access Node The order is used to eliminate sub-loops.

[0030] Construct a mixed-integer programming model with the objective of maximizing the system's net profit, and set the objective function as follows: as follows: ; Among them, penalties for crossing regional boundaries Defined as: ; in, Indicates site To the vehicle Work area The shortest distance, Penalty cost per unit distance for exceeding the boundary.

[0031] In addition, the model must satisfy the following complete constraints: Variable capacity constraint: The actual number of operations must not exceed the absolute value of demand, and operations only occur when a vehicle visits the station. ; Flow conservation constraint: ensuring balance at vehicle entry and exit nodes. ; Parking lot constraints: Vehicles must depart from and eventually return to the parking lot. ; ; Load limit constraint: Ensures that the vehicle's load at any node is non-negative and does not exceed its capacity, and updates dynamically based on the amount of operation. ; ; ; Time cost coupling constraints: Considering travel time, fixed parking time, and variable service time related to operation volume: ; ; ; Subloop constraint elimination: Eliminating subloops using the MTZ method: ; ; Vehicle number constraint: The total number of vehicles used shall not exceed the maximum fleet size limit. : .

[0032] In this embodiment, step S2 specifically involves modeling the iterative process of the Adaptive Large Neighborhood Search (ALNS) as a Markov Decision Process (MDP). State space The state vector contains 4 sets of features, with a total dimension of approximately 15-20, used to comprehensively describe the current search state: Search process characteristics: including the current iteration progress Simulated annealing temperature And the number of consecutive iterations that stall without improvement. Solution quality characteristics: including the difference between the current solution and the historical best solution. ,past The variance of the objective function in each iteration and the diversity of solutions (Hamming distance). Resource utilization characteristics: including the average remaining capacity rate of all vehicles. Average remaining time rate And the standard deviation of resource utilization. Demand satisfaction characteristics: including the average demand satisfaction rate across all sites. The demand satisfaction rate of the top 20% of high-value sites and the distribution ratio of different satisfaction rate intervals.

[0033] Action space Defined as selecting a pair (destruction operators) Repair operator ), total Combinations. Destructive operator set. This includes random removal and worst-case removal (where the removal unit yields a lower benefit). The three categories of removal methods are: lowest-rated sites, cluster removal (removing geographically clustered sites), and low-satisfaction removal (removing sites with a satisfaction rate lower than [a certain percentage]). (sites). Number of removals Set to 3-5. Repair operator set. This includes Greedy Insertion (selecting the position with the lowest insertion cost) and Regret-K Insertion (selecting the site with the highest regret value). And high-yield insertion (preferential insertion of unit yield) (Large sites).

[0034] reward function Designed as a multi-level reward function to guide the agent: ; Among them, the threshold was significantly improved. .

[0035] In this embodiment, step S3 specifically involves: using a Double DQN architecture as the agent. The input layer dimension adapts to the state space (15-20 dimensions); it contains 3 fully connected hidden layers, each with 128 neurons, using the ReLU activation function and setting Dropout to 0.2 to prevent overfitting; the output layer contains 12 neurons, corresponding to the Q values ​​of 12 action combinations, and uses a linear activation function. The training parameters and process involve using Adam as the optimizer and setting the learning rate to 1e-4. The experience replay pool has a capacity of 10,000, and the batch size is 64. The target network is updated every 100 steps. Strategy, The initial value is 1.0, the decay rate is 0.995, and the minimum value is 0.1. The number of training rounds is set to 1000-5000, with 1000 iterations per round.

[0036] The specific training process is as follows: First, initialize the network parameters and the experience replay buffer; then, use a greedy algorithm to generate the initial solution. In each iteration, the state is extracted. ,pass Strategy selection action Receive a reward after performing the action. and new status and will experience Store the data in the playback pool; randomly sample batches of data from the playback pool and calculate the target Q value. and the current Q value By minimizing the mean square error loss Update the main network parameters; periodically update the target network parameters and Values; save the main network parameters after training.

[0037] In this embodiment, step S4 specifically involves: the main framework of the algorithm is Adaptive Large Neighborhood Search (ALNS), forming a two-layer nested search structure: the outer layer is controlled by a DRL agent to manage topological perturbations, and the inner layer embeds computational optimization. The algorithm flow is as follows, and the online solution algorithm parameters are set as follows: maximum number of iterations. Simulated annealing initial temperature Cooling rate Termination temperature The algorithm execution steps are as follows: Initialization: Load the pre-trained DQN model and generate initial solutions. Initialize the optimal solution ,temperature and iterative counter ; Intelligent decision-making: Extracting the current state The intelligent agent outputs actions ; Topological perturbation (outer loop): Applying a disruptive operator Remove Each site, application repair operator Reinsert the site to generate a temporary solution for the fixed-path topology. ; Adaptive capacity allocation (inner loop): For For each path in the algorithm, the greedy allocator algorithm is called to calculate the optimal number of operations. And update the net profit of the path to obtain a new solution. ; Simulated annealing acceptance: Calculate the difference in the objective function .like If a new solution is accepted, the optimal solution is attempted to be updated; otherwise, the solution is determined by probability. Accept the new interpretation; Updates and records: Temperature reduction Calculate rewards Store the experience in the replay pool and proceed to the next iteration.

[0038] Greedy allocator algorithm, for fixed paths The steps of the greedy allocator algorithm are as follows: Initialization: Set all paths to the specified initialization value. Current load Current time Revenue per unit The sites are sorted in descending order to get a list. ; Forward computation (capacity constraint): Starting from the parking lot, perform a forward traversal and compute the capacity of each station. Limited by demand and vehicle capacity Maximum feasible operation range ; Backward computation (time constraint): Starting from the return point of the parking lot, perform a backward traversal based on the remaining working time. Calculate the maximum feasible number of operations for each site, limited by time. ; Greedy allocation: by Iterate through the sites sequentially, finding the intersection of capacity constraints, time constraints, and demand constraints. Greedy distribution And update the current load in real time. and time ; Adjust: Fine-tune the allocation results to ensure all hard constraints are met, and return the optimized result. Net income from the path.

[0039] In this embodiment, step S5 specifically involves: handling multiple complex constraints by introducing a soft constraint penalty term into the objective function and a hard constraint check into the greedy allocator algorithm. Geofencing constraint: added to the objective function. Item. Using the ray casting method to determine if a station is within a polygon. If the boundary is crossed, a penalty proportional to the distance will be imposed. This forces vehicles to operate within a designated area. Working time constraint: In the greedy sub-allocation algorithm, the number of operations is strictly limited through a "backward computation" step to ensure... If timeouts are allowed in the initial stages of the search, a timeout penalty term can be added to the objective function (e.g., ...). This guides the solution to converge toward the feasible region. Weight constraint: This is strictly guaranteed during the "forward computation" step of the greedy allocator algorithm. This ensures that the vehicle is not overloaded at any time.

[0040] For the foregoing embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to this application.

[0041] The above embodiments describe the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Modifications and variations made by those skilled in the art without departing from the spirit and scope of the invention should be within the protection scope of the appended claims.

Claims

1. A variable VRP search method for shared bicycle scheduling based on deep reinforcement learning, characterized in that, Includes the following steps: S1: Construct a variable VRP mathematical model considering spatiotemporal resource constraints. The constraints satisfied by the variable VRP mathematical model include one or more of the following: variable transport capacity constraints, flow conservation constraints, vehicle load limits, and time cost coupling constraints between travel time and operation time. Step S1 includes the following sub-steps: S11: Abstract the shared bicycle rebalancing scenario into a directed graph model. ,in This represents a collection of nodes that includes the repository and all potential task sites. The set of arcs representing the connecting nodes; S12: Define mixed decision variables: one type is binary path selection variables, used to decide on vehicles. Has it passed through an arc? One type establishes the topology; the other type consists of continuous operational variables used for vehicle decision-making. On the site The actual number of installations and uninstalls determines the service depth; S13: Construct a mixed-integer programming model with the goal of maximizing the system's net profit; S2: Construct a deep reinforcement learning interactive environment based on ALNS, including state space, action space, and reward function; S3: Build and train the DDQN agent; S4: Execute online topology-capacity nested search using a double-nested search strategy; Step S4 specifically involves decoupling the complex mixed-integer programming problem using a double-nested search strategy. Outer layer: Controlled by a trained DRL agent, the agent selects the optimal destruction and repair operators in real time based on the current state, removes and re-inserts nodes in the current path scheme, dynamically adjusts the access order of vehicles, and generates candidate path skeletons. Inner layer: For each fixed path generated by the outer layer, run the greedy allocator algorithm to calculate the marginal revenue of each station on the path and prioritize the needs of high-value stations. S5: Introduce constraint penalty terms into the objective function to soften and process multiple complex constraints.

2. The variable VRP search method for shared bicycle scheduling based on deep reinforcement learning as described in claim 1, characterized in that, The objective function of the mixed integer programming model includes: Service revenue: The sum of the products of the actual demand met at each site and the value of the demand per unit at that site; Transportation costs: fuel or electricity consumption costs incurred due to vehicle travel distance, and fixed costs associated with vehicle startup; Penalties for breach of contract: Penalties for violating soft time windows or driving beyond the designated boundaries.

3. The variable VRP search method for shared bicycle scheduling based on deep reinforcement learning as described in claim 1, characterized in that, The state space is described by multi-dimensional feature vectors, specifically including one or more of the following: search process features, solution quality features, resource utilization features, and demand satisfaction features.

4. The variable VRP search method for shared bicycle scheduling based on deep reinforcement learning as described in claim 1, characterized in that, The action space is a set of heuristic operator pairs, specifically including: which destruction operator to choose and which repair operator to choose.

5. The variable VRP search method for shared bicycle scheduling based on deep reinforcement learning as described in claim 1, characterized in that, The reward function is an instant reward mechanism based on net profit increment. If the solution generated by the current action is better than the historical best solution, a large positive reward is given; if the solution is improved but does not break the best, a small positive reward is given; if the solution quality declines, zero reward or negative penalty is given according to the simulated annealing criterion, thereby guiding the agent to converge quickly to the high-quality solution space.

6. The variable VRP search method for shared bicycle scheduling based on deep reinforcement learning as described in claim 1, characterized in that, Step S3 includes the following sub-steps: S31: Adopting the DDQN architecture, a deep neural network is built as the decision-making agent; S32: An offline strategy is used to train the agent. The training process includes: experience playback, exploration and utilization, and parameter updates.

7. The variable VRP search method for shared bicycle scheduling based on deep reinforcement learning as described in claim 1, characterized in that, The constraint penalty includes: geofencing constraint processing, which uses the ray method to determine whether the coordinates of all stations in the vehicle path are within the preset work area polygon. For stations that cross the boundary, the shortest Euclidean distance to the boundary is calculated, and a non-linear boundary penalty cost is applied, forcing the search algorithm to automatically correct the path to the compliant area.

8. The variable VRP search method for shared bicycle scheduling based on deep reinforcement learning as described in claim 1, characterized in that, The constraint penalty includes: working time constraint processing, real-time accumulation of vehicle driving time and node operation time, and if the total time exceeds a preset threshold, a penalty term proportional to the timeout duration is added to the objective function.

Citation Information

Patent Citations

  • Shared bicycle multi-target dynamic scheduling method based on double-matrix variable neighborhood search

    CN120996403A

  • Multi-constraint vehicle path planning method based on deep reinforcement learning

    CN121146645A