Express delivery vehicle scheduling optimization method and system with soft time window

Abstract

The invention discloses an express delivery vehicle scheduling optimization method and system with a soft time window, and relates to the technical field of vehicle scheduling, and the system comprises a CVRPSTW modeling and analysis module, an algorithm solution CVRPSTW module, and an experiment setting and result analysis module. The CVRPSTW modeling and analysis module constructs an express delivery vehicle scheduling problem model; the algorithm solution CVRPSTW module is used for respectively inputting node feature embedding and position feature embedding into an encoder; and the experiment setting and result analysis module is used for generating instances which can be input into the reinforcement learning network and verifying the capability of generalizing the CT model trained by the random instances to real world instances. According to the express delivery vehicle scheduling optimization method and system with the soft time window, excellent results can be obtained on data sets of multiple scales, the implementation effectiveness and application potential of the method are proved, and an efficient optimization scheme is provided for the vehicle scheduling problem in express delivery logistics.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle scheduling, and in particular to an express vehicle scheduling optimization method and system with a soft time window. Background Art

[0002] With the rapid development of e-commerce and the sharp increase in logistics demand, the express delivery industry has become an important part of the modern economy. The efficiency and punctuality of express delivery services directly affect customer satisfaction, while also placing higher demands on the operating costs and service quality of logistics companies. During the express delivery process, customers usually have clear requirements for the delivery time of their packages. This time window constraint makes the vehicle scheduling problem more complicated and poses severe challenges to vehicle path planning and scheduling optimization.

[0003] At present, traditional vehicle scheduling methods often cannot effectively handle time window constraints and dynamically changing delivery demands, resulting in low delivery efficiency, increased costs and decreased customer satisfaction. Therefore, an express vehicle scheduling optimization method and system with soft time windows are proposed to solve the above problems. Summary of the Invention

[0004] The main purpose of the present invention is to provide an express vehicle scheduling optimization method and system with soft time windows to solve the problems raised in the above background.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is: a method and system for optimizing express vehicle scheduling with soft time windows, including a CVRPSTW modeling and analysis module, an algorithm solution module for CVRPSTW, and an experimental setting and result analysis module;

[0006] The CVRPSTW modeling and analysis module constructs a courier vehicle scheduling problem model and performs constraint data analysis on the total delivery cost by minimizing transportation cost and penalty cost;

[0007] The algorithm solves the CVRPSTW module based on the encoding method of separating node features and position features. The node feature embedding and position feature embedding are input into the encoder respectively, and processed through a multi-layer collaborative attention mechanism to generate a fused feature representation.

[0008] The experimental setup and result analysis module includes data generation instructions, experimental and parameter settings, and real-time verification. It is used to generate instances that can be input into the reinforcement learning network for algorithm verification and analysis, calculate the relative deviation rate of each algorithm, evaluate the model hyperparameters in real time, and verify the ability of the CT model trained on random instances to generalize to real-world instances.

[0009] The CVRPSTW modeling and analysis module includes model construction and model analysis. The model construction is used to construct a model of the express vehicle scheduling problem. The optimization goal of the express vehicle scheduling problem is to minimize the total delivery cost. The minimization of the total delivery cost consists of two parts: transportation cost D(δ) and penalty cost U(δ);

[0010] Constraint data analysis is used to constrain the express vehicle scheduling problem in real time based on transportation costs, penalty costs, lead time when vehicles serve customers, late time when vehicles serve customers, completion time of vehicles serving customers, each customer point is served and only served once, the carrying capacity of each vehicle must not exceed the maximum rated load, and variable value constraint models.

[0011] The node feature code is used to represent the static information of each customer point, including location coordinates, demand volume, and time window.

[0012] The position feature coding is used to represent the access sequence information of the customer points in the path, and the access sequence of the customer points is encoded using cyclic position coding.

[0013] The specific method of processing through the multi-layer collaborative attention mechanism is as follows:

[0014] Take two sets of embeddings NFEs and PFEs as input;

[0015] The cross-aspect reference attention mechanism is used to fuse the self-attention correlations of the two sets of embeddings pairwise to achieve the sharing of position features and node features;

[0016] The AT layer is followed by the encoder's FFN layer, which converts the second step and As input, FFN h and FFN g The parameters are different, the layer uses a layer with dimension d FF = 64 hidden sublayers and ReLU activation function to calculate node projections.

[0017] The multi-layer collaborative attention mechanism also includes a decoder strategy. The decoder consists of a max pooling sublayer, an MHC sublayer, and an FFN sublayer, each of which plays a vital role in the decoding process.

[0018] The algorithm for solving the CVRPSTW module also includes using Advantage Function to guide the real-time update of the vehicle scheduling optimal strategy.

[0019] The data generation instructions are used to assign values ​​to node features for each category before validating the multi-classification problem, generating instances that can be input into the reinforcement learning network for algorithm validation and analysis;

[0020] The experiment and parameter settings are used to measure the algorithm performance by calculating the relative deviation rate of each algorithm, and the calculation formula is: where f A (s) represents the target value of algorithm A on example s, f Best (s) represents the optimal objective function value of all algorithms on example s.

[0021] The real-time validation is to compare the CT model with seven algorithms, including LKH, OR-Tools, ACO, HGA SIH

[100] , Ptr Net, AM

[99] and MDAM[]. The experimental results report the gap between the total cost and the optimal solution and the average computation time of each algorithm under two problem sizes.

[0022] The learning curves of the constructed express vehicle scheduling problem model algorithm and the AM algorithm are compared, and the ability of the CT model trained on random instances to generalize to real-world instances is verified in real time through experiments.

[0023] A method for optimizing express vehicle scheduling with a soft time window includes the following steps:

[0024] Step 1: Build a model to assist the Transformer attention network reinforcement learning algorithm in solving the problem;

[0025] Step 2: Adopt a coding method based on the separation of node features and location features to capture the spatial location, demand, time window, and access sequence information of the customer point, and obtain node feature coding and location feature coding;

[0026] Step 3: Use collaborative multi-head attention to replace the multi-head attention in the original Transformer to promote collaboration and enable information exchange between the two embeddings;

[0027] Step 4: Use the cross-aspect reference attention mechanism to fuse the correlations obtained from the two embedded information pairwise to achieve the sharing of position features and node features;

[0028] Step 5: Set the decoder strategy. The decoder design consists of a max pooling sublayer, an MHC sublayer, and an FFN sublayer.

[0029] Step 6: A reinforcement learning-driven path adjustment strategy, based on a local search strategy using deep reinforcement learning, automatically optimizes the solution starting from the initial solution and improves the previous path at each step;

[0030] Step 7: Assist in verifying the effectiveness of the Transformer attention network reinforcement learning algorithm.

[0031] The present invention has the following beneficial effects:

[0032] 1. In this invention, by limiting vehicle resources, the system has a limited number of homogeneous vehicles in the express delivery center. These vehicles will depart at the same time and return to the delivery center after completing their delivery tasks, making vehicle delivery scheduling more convenient;

[0033] 2. In this invention, by setting delivery uniqueness for each vehicle, each vehicle can only travel between the customer point and the delivery center once, and each customer point can only be visited once, so as to meet the customer's delivery needs and ensure the timeliness and safety of customer delivery needs;

[0034] 3. In the present invention, by setting a load limit on vehicles, the cargo volume of each vehicle during transportation shall not exceed the maximum carrying capacity of the vehicle, thereby ensuring the safety and reliability of vehicle scheduling optimization;

[0035] 4. In this invention, by setting the continuity of the route, after visiting a customer point, the vehicle must return to the distribution center or go to the next customer point. It cannot visit multiple customer points continuously without returning to the distribution center, thus avoiding route confusion during the vehicle scheduling optimization process.

[0036] 5. In this invention, by simplifying the time of the vehicle, the type of package has no significant impact on the loading and unloading time, achieving the effect of the same processing time for all packages, and preventing the impact of different package types on express delivery efficiency;

[0037] 6. In the present invention, by setting a fixed speed limit for vehicles, the driving speed of all vehicles is fixed and is not affected by road conditions or other external factors. This ensures the driving safety of vehicles during express delivery and reduces the risk of vehicle accidents.

[0038] 7. In the present invention, by setting a time window constraint for the express delivery vehicle, when the customer has clear requirements for the delivery time of the package, the vehicle can arrive within the specified time window, thereby improving customer satisfaction with the service. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a schematic diagram of the overall process of an express vehicle scheduling optimization method and system with soft time windows of the present invention;

[0040] Figure 2 This is a schematic diagram of the reinforcement learning algorithm flow of the CT model of the express vehicle scheduling optimization method and system with soft time windows of the present invention;

[0041] Figure 3 A schematic diagram showing the training convergence comparison between the CT model and other models of the express vehicle scheduling optimization method with soft time windows of the present invention;

[0042] Figure 4 This is a schematic diagram showing the comparison results of the CT model of the express vehicle scheduling optimization method and system with soft time windows of the present invention and other methods on the Solomon test set;

[0043] Figure 5 A schematic diagram of an optimal algorithm for an optimal vehicle scheduling strategy of an express vehicle scheduling optimization method and system with a soft time window according to the present invention;

[0044] Figure 6 This is a schematic diagram comparing the experimental results of the express vehicle scheduling optimization method with a soft time window and the CT model of the system on CVRPSTW20, CVRPSTW50 and CVRPSTW100. DETAILED DESCRIPTION

[0045] In order to make the technical means, creative features, objectives and effects achieved by the present invention easier to understand, the present invention is further described below in conjunction with specific implementation methods.

[0046] Please refer to Figures 1-6 Shown: A method and system for optimizing express vehicle scheduling with soft time windows, including a CVRPSTW modeling and analysis module, an algorithm for solving CVRPSTW, and an experimental setup and result analysis module;

[0047] The CVRPSTW modeling and analysis module builds a model for express vehicle scheduling problems and performs constraint data analysis on the total delivery cost by minimizing transportation costs and penalty costs.

[0048] The algorithm solves the CVRPSTW module based on the encoding method of separating node features and position features. The node feature embedding and position feature embedding are input into the encoder respectively, and processed through a multi-layer collaborative attention mechanism to generate a fused feature representation.

[0049] The experimental setup and result analysis module includes data generation instructions, experimental and parameter settings, and real-time verification. It is used to generate instances that can be input into the reinforcement learning network for algorithm verification and analysis, calculate the relative deviation rate of each algorithm, evaluate the model hyperparameters in real time, and verify the ability of the CT model trained on random instances to generalize to real-world instances.

[0050] The CVRPSTW modeling and analysis module includes model construction and model analysis. Model construction is used to construct a model for the express vehicle scheduling problem. The optimization goal of the express vehicle scheduling problem is to minimize the total delivery cost. Minimizing the total delivery cost consists of two parts: transportation cost D(δ) and penalty cost U(δ). The formula for minimizing the total delivery cost is as follows:

[0051] minD(δ)+U(δ);

[0052] Constraint data analysis is used to constrain the express delivery vehicle scheduling problem in real time based on transportation costs, penalty costs, lead time when vehicles serve customers, late time when vehicles serve customers, completion time of vehicles serving customers, each customer point is served and served only once, the carrying capacity of each vehicle must not exceed the maximum rated load, and the variable value constraint model. The specific constraint model is as follows:

[0053]

[0054] The variable table in the constraint model is represented as follows:

[0055]

[0056] The parameters in the constraint model are expressed as follows:

[0057]

[0058] Node feature coding is used to represent the static information of each customer point, including location coordinates, demand volume, and time window. Node feature coding is achieved through the following steps:

[0059] Set the position coordinates (x i ,y i ) is mapped to a high-dimensional space through linear transformation to generate a position feature vector;

[0060] Demand at the customer point d i Mapped into demand feature vector through linear layer;

[0061] The time window of the customer point (e i ,l i ) are mapped into the feature vectors of the earliest arrival time and the latest arrival time respectively through the linear layer;

[0062]

[0063] in, Represents vector concatenation operation;

[0064] Position encoding is used to represent the order in which customers visit points, and two-dimensional encoding is used to represent feasible solutions. Since warehouses can be visited multiple times, the path length of the solution to the CVRPSTW problem in this paper may be greater than N+1. Even for the same problem instance, different solutions may contain different numbers of subroutes, resulting in different path lengths. For example, for the CVRPSTW problem with four customers, the solutions δ0 = {0, 1, 2, 0, 4, 3, 0} and δ1 = {0, 1, 2, 4, 3, 0} are both feasible solutions, but with different path lengths. The model's neural network cannot handle this dimensional inconsistency. Therefore, this paper unifies the dimensionality of the feasible solutions by adding multiple virtual warehouses to the end of the initial solution. The number of these virtual warehouses can be regarded as the maximum number of available vehicles, thus avoiding the problem of inconsistent path lengths and facilitating batch training. Furthermore, the addition of virtual warehouses enables the model to automatically learn the number and length of subpaths in the solution, allowing the model to not only learn how to optimize the order of each node in the path but also handle multiple subpaths in the solution.

[0065] Position feature coding is used to represent the access sequence information of customer points in the path. Circular position coding is used to encode the access sequence of customer points. Position feature coding is implemented through the following steps:

[0066] Set the access order, the access order p of each customer point in the path i Mapped to position feature vector through cyclic position encoding;

[0067] Cyclic position coding: Cyclic position coding generates periodic features through sine and cosine functions to capture the relative position relationship between customer points in real time;

[0068] The cyclic position encoding used in this model is an improvement on Gray codes. Gray codes have cyclic properties (the "1110" in the last column is adjacent to the "1111" in the first column) and adjacency similarity properties (any codes in adjacent columns differ by only one digit). Both of these are required for cyclic sequences. Therefore, the cyclic position encoding of this model maintains these properties and follows the following rules:

[0069] Each number contains a period with reflection symmetry, for example, '10|001' in the lowest number;

[0070] The higher the value, the longer the period, where a periodic function with a period of 4πωd (induced by the modulus) is used to generate a basic symmetric pattern;

[0071]

[0072] in, means that N nodes are linearly spaced in the generated pattern, with an angular frequency ω d As the size decreases, the wavelength Varies within the range.

[0073] Node information x i Represents the coordinates, node requirements and time window of node i, and location information p i It represents the visit order of customers within a sub-route. During the encoding process, node features and position features are processed through different embedding layers to generate node feature embedding and position feature embedding respectively. These two feature embeddings are fused in the subsequent encoder and information interaction is achieved through the collaborative attention mechanism. This separate encoding method can effectively avoid information interference between node features and position features, and improve the learning ability of the model.

[0074] The specific method of processing through the multi-layer collaborative attention mechanism is as follows:

[0075] Taking two sets of embeddings NFEs and PFEs as input, we compute the self-attention correlations from these two sets of embeddings, where and are all trainable parameters and calculate and Q and V, and then use the soft max function to get and Further enhancement is done to obtain and

[0076]

[0077] The cross-aspect reference attention mechanism is used to fuse the self-attention correlations of the two sets of embeddings pairwise to achieve the sharing of position features and node features;

[0078] The AT layer is followed by the encoder's FFN layer, which converts the second step and As input, FFN h and FFN g The parameters are different, the layer uses a layer with dimension d FF = 64 hidden sub-layers and ReLU activation function are used to calculate node projections. The FFN layer performs linear transformation, ReLU activation and normalization operations.

[0079] The multi-layer collaborative attention mechanism also includes a decoder strategy. The decoder consists of a max pooling sublayer, an MHC sublayer, and an FFN sublayer. Each sublayer plays a vital role in the decoding process. The specific description of the decoder strategy is as follows:

[0080] The outputs from the encoder are aggregated into each respective representation through a max pooling sublayer, where , These are all trainable parameters;

[0081]

[0082] Step 2: The attention correlation of each max pooling layer result pair is then calculated through the MHC sublayer;

[0083] Step 3: The output is further processed by the feedforward network and the Mask softmax layer to produce a probability distribution of size N×N, which is used as the action distribution for node sorting in the solution.

[0084] The algorithm for solving the CVRPSTW module also includes using the Advantage Function to guide the real-time update of the optimal vehicle scheduling strategy. The specific steps are as follows:

[0085] Step 1: The actor interacts with the environment to generate a series of state-action pairs. At each step, the actor selects an action based on the current policy and obtains the next state and immediate reward through interaction with the environment.

[0086] Step 2: The Critic network evaluates the policy output in the current state and calculates the value function of each state. The Critic estimates the advantage of the current policy by comparing it with the actual reward.

[0087] Step 3: Use the n-step advantage function to calculate the advantage of the current strategy and speed up the training process;

[0088] Step 4: During the training process, the policy network is updated by optimizing the PPO objective function. The PPO algorithm uses a shear policy optimization to ensure the stability of the training process by limiting the amplitude of the policy update.

[0089] Step 5: Use the Adam optimizer to update the parameters of the policy network and the critic network, and adjust the learning rate according to the gradient size of different parameters;

[0090] Step 6: By continuously interacting with the environment and updating the policy, the actor and critic network gradually learn the optimal policy. In each training iteration, the actor adjusts the policy based on the critic evaluation results, ultimately achieving the optimal policy for vehicle scheduling.

[0091] The data generation instructions are used to assign node features to each category before verifying the multi-classification problem, generate instances that can be input into the reinforcement learning network, and are used for algorithm verification and analysis. For the CVRPSTW problem, this paper follows the existing convention and randomly generates instances of three sizes, namely N = 20, 50 and 100, and names them CVRPSTW20, CVRPSTW50 and CVRPSTW100. The algorithm studied in this paper automatically generates instances, where the coordinates of each node are uniformly distributed in the unit Euclidean plane [0,1]×[0,1], and the demand d of customer i is d. i Randomly sampling in the range of {1, ..., 9} and normalizing according to the number of customers, the normalized demand of customer i is expressed as The early time window of the customer time window is randomly generated uniformly from the [0,1] interval. The [0,1] interval is analogous to 12 hours in a day. The customer's service time window is set to 2 hours, so that the difference between the early and late time windows is 0.16. The capacity D of the vehicles used in CVRPSTW20, 50, and 100 are 30, 40, and 50 respectively. The maximum number of available vehicles is 10, 20, and 20 respectively. The early arrival penalty weight coefficient and the late arrival penalty weight coefficient are 0.1 and 0.5 respectively;

[0092] The experiment and parameter settings are used to measure the algorithm performance by calculating the relative deviation rate of each algorithm. The calculation formula is: where f A (s) represents the target value of algorithm A on example s, f Best (s) represents the optimal objective function value of all algorithms on the example s. In the network model training part, the number of epochs is set to E = 100, and 256 instances are trained in each epoch. The batch size is set to B = 256 during the experiment. The Adam optimizer is selected as the training optimizer. In the testing phase, the trained model is tested using verification data, that is, after completing each training cycle epoch, the model is verified according to the size of B = 100 to evaluate the model performance. During the test, the greedy strategy is used to select the path node, that is, the node with the largest attention value is selected. In addition, this study also conducted experiments using the classic example on Solomon

[91] to verify the generalization performance of the CT model. The initial experiments using three path adjustment strategies, 2-opt, Swap and Insert, showed that 2-opt performed the best (Insert is better than Swap).

[0093] The real-time verification is to compare the CT model with seven algorithms, including LKH, OR-Tools, ACO, HGASIH

[100] , Ptr Net, AM

[99] and MDAM[]. The experimental results report the gap between the total cost and the optimal solution and the average calculation time of various algorithms under two problem scales. The selection of a suitable model iteration step size T has an important impact on the convergence and final performance of the algorithm. In order to comprehensively evaluate the performance of the model under different training step sizes, the CT model is directly compared with the seven algorithms mentioned above in the experiment. The experimental results report the total cost, the gap between the optimal solution and the average calculation time of various algorithms under two problem scales. Figure 6 As shown;

[0094] The learning curves of the constructed express vehicle scheduling problem model algorithm and the AM algorithm are compared. The ability of the CT model trained with random instances to generalize to real-world instances is verified in real time by experiments. In order to detect the convergence of the model, this paper compares the learning curves of the proposed algorithm and the AM algorithm. It is generally believed that when training the same problem, if a model can achieve the same learning effect as the target algorithm in a shorter training cycle, or achieve the same learning efficiency with fewer instances, then the model is considered to have better convergence, such as Figure 3 As shown in Figure 2, the training cost of all scale problems decreases rapidly in the initial stage of training. As the number of training rounds increases, the network of the model is continuously updated and the rate of decrease of the target value begins to slow down, which means that the model begins to approach the local minimum of the VRPSTW problem. Figure 3 All the curves in the figure show the superior performance of rapid decrease of target value in the first few training cycles. As the training continues, the rate of cost decrease slows down, indicating that the model parameters begin to be adjusted more carefully to capture more subtle features and regularities in the data, and then gradually stabilize. At the same time, the target value of the CT model algorithm decreases faster than that of the AM algorithm during optimization, indicating that the CT model algorithm has higher sample efficiency than the AM algorithm, thus verifying that considering structural problems helps to improve the convergence of the algorithm. When the problem scale is large, the sample efficiency advantage of the algorithm is more significant. Figure 4 As shown in Figure 2, experiments were also conducted on the classic example in the Solomon

[91] example. The model was trained for 100 epochs using random instances of CVRPSTW20 and CVRPSTW50, and the optimal solution was recorded for each instance 10 times. Figure 4The results of the AM model and the CT model in this paper are compared with the known optimal solution. The experimental results show that in most cases, the CT model surpasses the AM model in controlling the number of subpaths and target values, and is close to the known optimal solution, verifying the ability of the CT model trained on random instances to generalize to real-world instances.

[0095] In the present invention, a method and system for optimizing express vehicle scheduling with a soft time window are provided. When the system is in operation, a model is first constructed to assist the Transformer attention network reinforcement learning algorithm in solving the problem, so that the system has a limited number of homogeneous vehicles in the express delivery center. These vehicles will depart at the same time and return to the delivery center after completing the delivery task, thereby increasing the convenience of vehicle delivery scheduling. A coding method based on the separation of node features and position features is adopted to capture the spatial location, demand, time window and access sequence information of customer points, and obtain node feature coding and position feature coding, so that each vehicle can only travel between the customer point and the delivery center once, and each customer point can only be visited once, so as to meet the customer's delivery needs and ensure The timeliness and safety of customer delivery needs are guaranteed; the collaborative multi-head attention is used to replace the multi-head attention in the original Transformer to promote collaboration and realize information exchange between the two embeddings, so that the cargo volume of each vehicle during transportation shall not exceed the maximum carrying capacity of the vehicle, ensuring the safety and reliability of vehicle scheduling optimization; the cross-aspect reference attention mechanism is used to fuse the correlations obtained from the two embedding information pairwise to realize the sharing of location features and node features, so that after visiting a customer point, the vehicle must return to the distribution center or go to the next customer point, and cannot visit multiple customer points continuously without returning to the distribution center, avoiding path confusion in the vehicle scheduling optimization process; set the decoder strategy, the decoder design is based on the max It consists of a pooling sublayer, an MHC sublayer, and a FFN sublayer, so that the type of package has no significant impact on the loading and unloading time, achieving the same processing time for all packages and preventing different package types from affecting the efficiency of express delivery; the reinforcement learning-driven path adjustment strategy, based on the local search strategy of deep reinforcement learning, automatically optimizes the solution starting from the initial solution and improves the previous path at each step, so that the driving speed of all vehicles is a fixed value and is not affected by road conditions or other external factors, so that the driving safety of vehicles during express delivery is guaranteed and the risk of vehicle accidents is reduced; it assists in verifying the effectiveness of the Transformer attention network reinforcement learning algorithm, so that when customers have clear requirements for the delivery time of the package, the vehicle can arrive within the specified time window, thereby improving customer satisfaction with the service and improving the feasibility of the express vehicle scheduling optimization method with soft time windows.

[0096] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A delivery vehicle dispatch optimization system with soft time windows, characterized in that: The system includes a CVRPSTW modeling and analysis module, an algorithm solution CVRPSTW module, and an experiment setting and result analysis module; The CVRPSTW modeling and analysis module constructs a courier vehicle scheduling problem model and performs constraint data analysis on the total delivery cost by minimizing transportation cost and penalty cost; The algorithm solves the CVRPSTW module based on the encoding method of separating node features and position features. The node feature embedding and position feature embedding are input into the encoder respectively, and processed through a multi-layer collaborative attention mechanism to generate a fused feature representation. The experimental setup and result analysis module includes data generation instructions, experimental and parameter settings, and real-time verification. It is used to generate instances that can be input into the reinforcement learning network for algorithm verification and analysis, calculate the relative deviation rate of each algorithm, evaluate the model hyperparameters in real time, and verify the ability of the CT model trained on random instances to generalize to real-world instances.

2. The system according to claim 1, wherein: The CVRPSTW modeling and analysis module includes model construction and model analysis. Model construction is used to construct a model of the express vehicle scheduling problem. The optimization goal of the express vehicle scheduling problem is to minimize the total delivery cost. Minimizing the total delivery cost consists of two parts: transportation cost D(δ) and penalty cost U(δ). The formula for minimizing the total delivery cost is as follows: minD(δ)+U(δ); Constraint data analysis is used to constrain the express vehicle scheduling problem in real time based on transportation costs, penalty costs, lead time when vehicles serve customers, late time when vehicles serve customers, completion time of vehicles serving customers, each customer point is served and only served once, the carrying capacity of each vehicle must not exceed the maximum rated load, and variable value constraint models.

3. The system according to claim 1, wherein: The node feature coding is used to represent the static information of each customer point, including location coordinates, demand volume, and time window. The node feature coding is implemented by the following steps: Set the position coordinates (x i ,y i ) is mapped to a high-dimensional space through linear transformation to generate a position feature vector; Demand at the customer point d i Mapped into demand feature vector through linear layer; The time window of the customer point (e i ,l i ) are mapped into the feature vectors of the earliest arrival time and the latest arrival time respectively through the linear layer; in, Represents vector concatenation operation; Position coding is used to indicate the order in which customer points are visited.

4. The system according to claim 1, wherein: The position feature coding is used to represent the access sequence information of the customer points in the path. The access sequence of the customer points is encoded using cyclic position coding. The position feature coding is implemented by the following steps: Set the access order, the access order p of each customer point in the path i Mapped to position feature vector through cyclic position encoding; Cyclic position coding,Cyclic position coding generates periodic features through sine and cosine functions,,capturing the relative position relationship between customer points in real,time.

5. The system according to claim 1, wherein: The specific method of processing through the multi-layer collaborative attention mechanism is as follows: Taking two sets of embeddings NFEs and PFEs as input, we compute the self-attention correlations from these two sets of embeddings, where and are all trainable parameters and calculate and Q and V, and then use the softmax function to get and Further enhancement is done to obtain and The cross-aspect reference attention mechanism is used to fuse the self-attention correlations of the two sets of embeddings pairwise to achieve the sharing of position features and node features; The AT layer is followed by the encoder's FFN layer, which converts the second step and As input, FFN h and FFN g The parameters are different, the layer uses a layer with dimension d FF = 64 hidden sublayers and ReLU activation function to calculate node projections.

6. The system according to claim 5, characterized in that: The multi-layer collaborative attention mechanism also includes a decoder strategy. The decoder consists of a max pooling sublayer, an MHC sublayer, and an FFN sublayer. Each sublayer plays a vital role in the decoding process. The specific description of the decoder strategy is as follows: The outputs from the encoder are aggregated into each respective representation through a max pooling sublayer, where , These are all trainable parameters; Step 2: The attention correlation of each max pooling layer result pair is then calculated through the MHC sublayer; Step 3: The output is further processed by the feedforward network and the Mask softmax layer to produce a probability distribution of size N×N, which is used as the action distribution for node sorting in the solution.

7. The system according to claim 1, wherein: The algorithm for solving the CVRPSTW module also includes using the Advantage Function to guide the real-time update of the optimal vehicle scheduling strategy. The specific steps are as follows: Step 1: The actor interacts with the environment to generate a series of state-action pairs. At each step, the actor selects an action based on the current policy and obtains the next state and immediate reward through interaction with the environment. Step 2: The Critic network evaluates the policy output in the current state and calculates the value function of each state. The Critic estimates the advantage of the current policy by comparing it with the actual reward. Step 3: Use the n-step advantage function to calculate the advantage of the current strategy and speed up the training process; Step 4: During the training process, the policy network is updated by optimizing the PPO objective function. The PPO algorithm uses a shear policy optimization to ensure the stability of the training process by limiting the amplitude of the policy update. Step 5: Use the Adam optimizer to update the parameters of the policy network and the critic network, and adjust the learning rate according to the gradient size of different parameters; Step 6: By continuously interacting with the environment and updating the policy, the actor and critic network gradually learn the optimal policy. In each training iteration, the actor adjusts the policy based on the results of the critic evaluation, ultimately achieving the optimal policy representation for vehicle scheduling.

8. The system according to claim 1, wherein: The data generation instructions are used to assign values ​​to node features for each category before validating the multi-classification problem, generating instances that can be input into the reinforcement learning network for algorithm validation and analysis; The experiment and parameter settings are used to measure the algorithm performance by calculating the relative deviation rate of each algorithm, and the calculation formula is: where f A (s) represents the target value of algorithm A on example s, f Best (s) represents the optimal objective function value of all algorithms on example s.

9. The system according to claim 8, characterized in that: The real-time validation is to compare the CT model with seven algorithms, including LKH, OR-Tools, ACO, HGA SIH[100], Ptr Net, AM[99] and MDAM[]. The experimental results report the gap between the total cost and the optimal solution and the average computation time of each algorithm under two problem sizes. The learning curves of the constructed express vehicle scheduling problem model algorithm and the AM algorithm are compared, and the ability of the CT model trained on random instances to generalize to real-world instances is verified in real time through experiments.

10. The express delivery vehicle scheduling optimization method with soft time windows according to any one of claims 1 to 9, characterized in that: The following steps are involved: Step 1: Build a model to assist the Transformer attention network reinforcement learning algorithm in solving the problem; Step 2: Adopt a coding method based on the separation of node features and location features to capture the spatial location, demand, time window, and access sequence information of the customer point, and obtain node feature coding and location feature coding; Step 3: Use collaborative multi-head attention to replace the multi-head attention in the original Transformer to promote collaboration and enable information exchange between the two embeddings; Step 4: Use the cross-aspect reference attention mechanism to fuse the correlations obtained from the two embedded information pairwise to achieve the sharing of position features and node features; Step 5: Set the decoder strategy. The decoder design consists of a max pooling sublayer, an MHC sublayer, and an FFN sublayer. Step 6: A reinforcement learning-driven path adjustment strategy, based on a local search strategy using deep reinforcement learning, automatically optimizes the solution starting from the initial solution and improves the previous path at each step; Step 7: Assist in verifying the effectiveness of the Transformer attention network reinforcement learning algorithm.

Images