Vehicle path planning method and device

Abstract

The present invention provides a vehicle routing method and apparatus, comprising: a data acquisition step, a model loading step, and a routing generation step. In the routing generation step, a routing strategy search model is utilized to generate and output a routing strategy. The routing strategy search model employs a single simulated vehicle to search through all customer sites, and a sequence of sites, in which the simulated vehicle departs from a starting site, passes through one or more customer sites, and returns to the starting site, is used as the delivery route for a single delivery vehicle, thereby obtaining routing strategies for multiple delivery vehicles. The vehicle routing method and apparatus provided by the present invention, through a serial decision search mechanism, can improve the performance and efficiency of solving time-windowed vehicle-to-remote delivery (VRP) problems.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle routing problem (VRP), and in particular to a vehicle routing planning method and device. Background Art

[0002] The vehicle routing planning problem (VRP) with time windows has a wide range of real-world applications, such as bus planning and freight transportation. Existing solutions to this problem mainly include traditional methods and reinforcement learning-based methods.

[0003] Among them, traditional methods specifically include precise algorithms and heuristic methods. Precise algorithms are inefficient and have no practical value; the performance of heuristic methods relies on sophisticated manual design and is not efficient enough.

[0004] Reinforcement learning-based methods struggle to maintain high-performance search for target optimization while simultaneously considering multiple constraints. In this project, these constraints include time window constraints, vehicle load limits, and vehicle number limits.

[0005] For example, a Chinese patent application (CN110147901A) uses a pointer network to directly generate a solution to the VRP problem. However, when considering multiple constraints simultaneously, the model becomes difficult to train and suffers from a significant performance degradation due to the substantial increase in the number of states, action spaces, and model size. Therefore, it is difficult to apply to large-scale VRP problems.

[0006] For example, another Chinese patent application (CN106251009A) uses an ant colony optimization algorithm to solve the VRP problem and generate a solution. However, it has defects such as slow generation speed and lack of real-time response capability when dealing with large-scale data distribution.

[0007] In addition, existing technologies usually use neural networks for path search, and the search performance is very low under multiple constraints.

[0008] Therefore, there is an urgent need for a VRP method that can be applied under multiple constraints and has both high performance and high efficiency to solve the vehicle path planning problem. Summary of the Invention

[0009] The technical problem to be solved by the embodiments of the present invention is to provide a vehicle path planning method and device, which can improve the performance and efficiency of path planning.

[0010] To solve the above technical problems, an embodiment of the present invention provides a vehicle path planning method, comprising:

[0011] A data acquisition step, obtaining the originating station for route planning, the maximum load of the simulated vehicle, the geographical locations of multiple customer stations, the delivery demand and service time window of each customer station as input data;

[0012] A model loading step, loading a pre-trained path strategy search model, and initializing the global time and the remaining capacity and geographic location of the simulated vehicle;

[0013] The path generation step uses the path strategy search model to generate and output a path strategy, wherein the path strategy search model uses a single simulated vehicle to search among all customer sites, and uses a sequence of sites in which the simulated vehicle departs from the starting site, passes through one or more customer sites, and returns to the starting site as the delivery path of a delivery vehicle, thereby obtaining the path strategy for multiple delivery vehicles.

[0014] Optionally, the using the path strategy search model to generate and output a path strategy includes:

[0015] The first judgment step is to judge whether there is an unvisited customer site in the input data, and if so, enter the site screening step, otherwise enter the policy output step;

[0016] The site screening step includes calculating the arrival time of the simulated vehicle at each unvisited customer site based on the geographical location of the simulated vehicle and the geographical location of the unvisited customer site, and screening customer sites whose arrival time is no later than the end time of the service time window of the customer site as candidate sites;

[0017] The second judgment step is to judge whether the candidate site exists, if so, to enter the probability calculation step, otherwise to enter the reset step;

[0018] The probability calculation step calculates the probability of the simulated vehicle going to each candidate station, selects the candidate station with the largest selection probability as the current target station, and determines whether the remaining capacity of the simulated vehicle meets the delivery demand of the current target station. If so, the state update step is entered; otherwise, the reset step is entered;

[0019] The resetting step resets the global time, returns the simulated vehicle to the originating station, and then enters the station screening step;

[0020] The state updating step updates the geographical location of the simulated vehicle to the geographical location of the current target site, and updates the global time, the remaining capacity of the simulated vehicle, and the access status of the customer site, and then returns to the first judgment step;

[0021] The strategy output step obtains a total site sequence including the order in which each site is visited, and takes the site sequence starting from the originating site, passing through one or more customer sites, and returning to the originating site as a site sequence. The total site sequence is divided into at least one site sequence and then output, wherein each site sequence corresponds to the delivery path of a delivery vehicle.

[0022] Optionally, the calculation of the selection probability value of the simulated vehicle going to each candidate site includes:

[0023] Generate a first feature expression for each candidate site based on the geographic location and delivery demand of each candidate site; calculate the expected waiting time after the simulated vehicle arrives at each candidate site, and generate a second feature expression for each candidate site based on the expected waiting time; generate a site feature expression for each candidate site based on the first and second feature expressions; and generate a global feature based on the site feature expressions of all candidate sites;

[0024] Using the global feature as an index, an attention mechanism is used to generate a selection probability value for each candidate site.

[0025] Optionally, before the data input step, the method further includes:

[0026] The training step is to obtain the path strategy search model through training.

[0027] Optionally, the training step specifically includes:

[0028] The first step is to obtain a training set, wherein the training set includes multiple sets of training data, each set of training data includes an originating station, a maximum load capacity of a simulated vehicle, geographical locations of multiple customer stations, a delivery demand of each customer station, and a service time window;

[0029] The second step is to initialize a reference model of the path strategy search model;

[0030] The third step is to select a set of unused training data and perform a round of training using the selected training data to train the path strategy search model, wherein at the beginning of each round of training, the path strategy search model uses the current reference model; during each round of training, the model parameters of the path strategy search model are updated using the training data, and when the performance improvement of the updated path strategy search model relative to the current reference model meets a preset condition, the reference model is updated according to the updated path strategy search model;

[0031] The fourth step is to determine whether the preset network training end conditions are met at the end of each round of training. If so, the training of the path strategy search model is terminated, and the current reference model is output as the path strategy search model obtained by the final training; otherwise, return to the third step.

[0032] Optionally, performing a round of training using the selected training data to train the path strategy search model includes:

[0033] The first sub-step is to initialize the path strategy search model according to the current reference model;

[0034] The second sub-step is to use the selected training data as input data, execute the path generation step, and obtain the path strategy corresponding to the training data;

[0035] The third sub-step is to calculate the reward function value according to the path strategy corresponding to the training data;

[0036] A fourth sub-step is to update the model parameters of the path strategy search model according to the reward function value;

[0037] The fifth sub-step is to determine whether the performance improvement of the updated path strategy search model relative to the current reference model meets the preset conditions. If so, update the reference model according to the updated path strategy search model and return to the second sub-step. Otherwise, end the current round of training.

[0038] Optionally, calculating the reward function value according to the path strategy corresponding to the training data includes:

[0039] generating a main item according to the total distance of the path strategy corresponding to the training data;

[0040] Generate a penalty term corresponding to the number of vehicles based on the distance between the last two stations in each station sequence in the routing strategy corresponding to the training data;

[0041] The reward function value is calculated based on the main term and the penalty term.

[0042] Optionally, determining whether the performance improvement of the updated path strategy search model relative to the current reference model meets a preset condition includes:

[0043] Input a set of verification data as input data into the updated path strategy search model to obtain a first path strategy corresponding to the verification data; input the verification data into the reference model to obtain a second path strategy corresponding to the verification data; calculate the paired t-test results of the alternative probability value distribution corresponding to the first path strategy and the alternative probability value distribution corresponding to the second path strategy;

[0044] According to whether the paired t-test result is less than a preset threshold, it is determined whether the performance improvement degree of the updated path strategy search model relative to the current reference model meets a preset condition.

[0045] Optionally, the network training termination condition is: the performance improvement of the updated path strategy search model obtained after M consecutive rounds of training relative to the current reference model does not meet a preset condition; and M is a preset integer greater than 2.

[0046] An embodiment of the present invention further provides a vehicle path planning device, comprising:

[0047] A data acquisition module is used to obtain, as input data, the originating station for route planning, the maximum load capacity of the simulated vehicle, the geographical locations of multiple customer stations, the delivery demand and service time window of each customer station;

[0048] A model loading module is used to load a pre-trained path strategy search model and initialize the global time and the remaining capacity and geographic location of the simulated vehicle;

[0049] A path generation module is used to generate and output a path strategy using the path strategy search model, wherein the path strategy search model uses a single simulated vehicle to search among all customer sites, and uses a sequence of sites in which the simulated vehicle departs from the originating site, passes through one or more customer sites, and returns to the originating site as the delivery path of a delivery vehicle, thereby obtaining path strategies for multiple delivery vehicles.

[0050] Optionally, the path generation module includes:

[0051] A first judgment module is used to judge whether there is an unvisited customer site in the input data, and if so, enter the site screening module, otherwise enter the policy output module;

[0052] The site screening module is configured to calculate the arrival time of the simulated vehicle at each unvisited customer site based on the geographical location of the simulated vehicle and the geographical locations of the unvisited customer sites, and screen customer sites whose arrival time is no later than the end time of the service time window of the customer site as candidate sites;

[0053] The second judgment module is used to judge whether the candidate site exists, and if so, enter the probability calculation module, otherwise enter the reset module;

[0054] The probability calculation module is used to calculate the selection probability value of the simulated vehicle to each candidate station, select the candidate station with the largest selection probability value as the current target station, and determine whether the remaining capacity of the simulated vehicle meets the delivery demand of the current target station. If so, enter the state update module; otherwise, enter the reset module;

[0055] The reset module is used to reset the global time and return the simulated vehicle to the starting station and then enter the station screening module;

[0056] The state updating module is used to update the geographical location of the simulated vehicle to the geographical location of the current target site, and update the global time, the remaining capacity of the simulated vehicle and the access status of the customer site, and then return to the first judgment module;

[0057] The strategy output module is used to obtain a total site sequence including the order in which each site is visited, and to take the site sequence that starts from the originating site, passes through one or more customer sites, and returns to the originating site as a site sequence segment. The total site sequence is divided into at least one site sequence segment and then output, wherein each site sequence segment corresponds to the delivery path of a delivery vehicle.

[0058] Optionally, the device further includes:

[0059] The training module is used to train and obtain the path strategy search model.

[0060] Optionally, the training module specifically includes:

[0061] The first module is used to obtain a training set, wherein the training set includes multiple sets of training data, each set of training data includes an originating station, a maximum load capacity of a simulated vehicle, geographical locations of multiple customer stations, a delivery demand of each customer station, and a service time window;

[0062] The second module is used to initialize a reference model of a path strategy search model;

[0063] a third module, configured to select a set of unused training data and perform a round of training using the selected training data to train the path strategy search model, wherein at the beginning of each round of training, the path strategy search model adopts a current reference model; during each round of training, the model parameters of the path strategy search model are updated using the training data; and when the performance improvement of the updated path strategy search model relative to the current reference model satisfies a preset condition, the reference model is updated according to the updated path strategy search model;

[0064] The fourth module is used to determine whether the preset network training end conditions are met at the end of each round of training. If so, the training of the path strategy search model is ended and the current reference model is output as the path strategy search model obtained by the final training; otherwise, the third module is returned.

[0065] Optionally, the third module includes:

[0066] A first submodule is configured to initialize the path strategy search model according to the current reference model;

[0067] The second submodule is configured to use the selected training data as input data, execute the path generation module, and obtain a path strategy corresponding to the training data;

[0068] A third submodule is used to calculate a reward function value according to the path strategy corresponding to the training data;

[0069] A fourth submodule is configured to update the model parameters of the path strategy search model according to the reward function value;

[0070] The fifth submodule is used to determine whether the performance improvement of the updated path strategy search model relative to the current reference model meets the preset conditions. If so, the reference model is updated according to the updated path strategy search model and returns to the second submodule. Otherwise, the current round of training is terminated.

[0071] Optionally, the third submodule is specifically configured to:

[0072] generating a main item according to the total distance of the path strategy corresponding to the training data;

[0073] Generate a penalty term corresponding to the number of vehicles based on the distance between the last two stations in each station sequence in the routing strategy corresponding to the training data;

[0074] The reward function value is calculated based on the main term and the penalty term.

[0075] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the vehicle path planning method described above are implemented.

[0076] Compared to existing technologies, the vehicle path planning method and apparatus provided in embodiments of the present invention utilize a single simulated vehicle to search in global space, concentrating multiple conditional constraints on a single vehicle for simulation. This significantly reduces the difficulty of the search and ultimately assigns each segment of the vehicle's path to different vehicles in a real-world production environment, achieving equivalent implementation of the goal and improving the performance and efficiency of path search. Furthermore, in the reward function, embodiments of the present invention not only retain the total distance as an optimization term, but also, given that the number of vehicles is equivalent to the number of times a simulated vehicle returns to the origin, a weighted penalty is applied to all paths that return to the origin in the final function. This allows the reward function to achieve dual control over both the total distance and the number of vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0078] Figure 1 A schematic flow chart of a vehicle path planning method according to an embodiment of the present invention;

[0079] Figure 2 A flow chart of generating a path strategy in a vehicle path planning method according to an embodiment of the present invention;

[0080] Figure 3 This is an overall flow chart of training the path strategy search model according to an embodiment of the present invention;

[0081] Figure 4 A flowchart of each round of training of the path strategy search model provided by an embodiment of the present invention;

[0082] Figure 5 A schematic structural diagram of a vehicle path planning device provided by an embodiment of the present invention;

[0083] Figure 6 Another structural schematic diagram of the vehicle path planning device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0084] To make the technical problems, technical solutions, and advantages to be solved by the present invention more apparent, a detailed description will be given below with reference to the accompanying drawings and specific embodiments. In the following description, specific details such as specific configurations and components are provided solely to facilitate a comprehensive understanding of the embodiments of the present invention. Therefore, it should be clear to those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. In addition, for the sake of clarity and brevity, descriptions of known functions and configurations have been omitted.

[0085] It should be understood that references throughout this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic associated with the embodiment is included in at least one embodiment of the present invention. Therefore, the appearances of "in one embodiment" or "in an embodiment" throughout this specification do not necessarily refer to the same embodiment. Furthermore, these particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0086] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0087] As described in the background art, in the existing vehicle path planning methods, neural networks often have problems such as low performance and efficiency, or even impracticality, when searching for the optimal path under multiple constraints. To solve at least one of the above problems, an embodiment of the present invention provides a vehicle path planning method, such as Figure 1 As shown, the method includes:

[0088] Step 101 , obtaining the originating site for route planning, the maximum load capacity of the simulated vehicle, the geographical locations of multiple customer sites, the delivery demand and the service time window of each customer site as input data.

[0089] Here, the embodiment of the present invention performs vehicle route planning, which requires inputting the starting station to be planned, the maximum load capacity of the simulated vehicle (such as maximum capacity, maximum load, etc.), the geographical location of multiple customer stations, and the distribution requirements and service time window information of each customer station. The geographical location can be represented by longitude and latitude coordinates, and the distribution requirement information can be delivery service and / or pickup service and the corresponding cargo load capacity requirements (such as volume requirements, load requirements, etc.). The service time window of the customer station, that is, the customer station can only be served within this time window. If the time of arrival at the customer station is later than the end time of the time window, the customer station cannot be served; if the time of arrival at the customer station is earlier than the start time of the time window, it is necessary to wait until it enters the range of the time window. In addition, when performing route planning, the embodiment of the present invention can also input more information, such as road congestion information, vehicle quantity limit and other conditions.

[0090] In this embodiment of the present invention, there may be multiple delivery vehicles. Each delivery vehicle departs from a departure point and follows a planned route to various customer sites, serving each customer site in sequence, before ultimately returning to the departure point. Any customer site may require delivery and / or pickup services. Specifically, when delivering goods to a customer site, the delivery vehicle loads the goods at the departure point, transports them to the customer site, and then unloads them. When picking up goods from a customer site, the delivery vehicle loads the corresponding goods at the customer site, transports them to the departure point, and then unloads them. To simplify the search process, this embodiment of the present invention employs a serial decision search mechanism for path search. Specifically, a single simulated vehicle simulates the delivery of multiple delivery vehicles. It is assumed that each delivery vehicle has the same specifications (i.e., the same maximum load capacity) and that the specifications of the simulated vehicle and the delivery vehicle are identical. Each delivery trip performed by the simulated vehicle corresponds to an actual delivery trip by the delivery vehicle. Here, the sequence of stops made by the simulated vehicle after departing from the departure point, passing through one or more customer sites, and returning to the departure point is considered a stop sequence, corresponding to a delivery trip. When there are a large number of customer sites and / or a large amount of cargo, the simulated vehicle may need to perform multiple deliveries to complete service for all customer sites.

[0091] Step 102 : Load a pre-trained path strategy search model and initialize the global time and the remaining capacity and geographic location of the simulated vehicle.

[0092] Here, the embodiment of the present invention pre-trains a path strategy search model, loads the model in step 102, and initializes the global time and the remaining capacity and geographic location of the simulated vehicle. The global time is initialized to 0 or other initial values. At the beginning of each delivery, the global time needs to be initialized (i.e., reset to the initial value) to represent the starting time of a delivery trip when the vehicle departs from the starting station. The initial value of the remaining capacity of the simulated vehicle is the maximum load capacity of the vehicle, and the initial value of the geographic location is the geographic location of the starting station. At the beginning of each delivery, the remaining capacity and geographic location of the simulated vehicle need to be initialized.

[0093] Step 103: Utilize the path strategy search model to generate and output a path strategy.

[0094] Here, the path strategy search model uses a single simulated vehicle to search among all customer sites, and takes a sequence of sites in which the simulated vehicle departs from the originating site, passes through one or more customer sites, and returns to the originating site as the actual delivery path of a delivery vehicle. In this way, each site sequence corresponds to the actual delivery path of a delivery vehicle, thereby obtaining the delivery paths of multiple delivery vehicles, that is, the final path strategy.

[0095] Through the above steps, the embodiment of the present invention uses a simulated vehicle to perform path search. After each delivery is completed, it returns to the originating site and continues the next delivery to serve the remaining customer sites. Therefore, through the above-mentioned serial decision search mechanism, a single vehicle is used to search in the global space, which can greatly reduce the search difficulty. Ultimately, the delivery paths of the simulated vehicle are assigned to different delivery vehicles in the actual production environment, achieving an equivalent implementation effect of the target.

[0096] The following describes in detail how the path strategy search model is used to generate and output the path strategy in step 103 .

[0097] In the embodiment of the present invention, the path strategy search model is used to perform the following steps to obtain the path strategy, specifically, Figure 2 As shown,

[0098] Step 1031 , determining whether there is an unvisited customer site in the input data, if so, proceeding to step 1032 , otherwise proceeding to step 1038 .

[0099] Step 1032, based on the geographic location of the simulated vehicle and the geographic location of the unvisited customer sites, calculate the arrival time of the simulated vehicle at each unvisited customer site, and select customer sites whose arrival time is no later than the deadline of the service time window of the customer site as alternative sites.

[0100] Step 1033 , determine whether the candidate site exists. If so, proceed to step 1034 , otherwise proceed to step 1036 .

[0101] Step 1034 , calculating the selection probability values ​​of the simulated vehicle going to each candidate site, and selecting the candidate site with the largest selection probability value as the current target site.

[0102] Here, when calculating the selection probability value, the embodiment of the present invention can generate a first feature expression for each candidate site based on the geographical location and delivery demand of each candidate site, and can specifically generate the first feature expression using a neural network layer;

[0103] Then, the expected waiting time after the simulated vehicle arrives at each alternative site is calculated, and the second characteristic expression of each alternative site is generated based on the expected waiting time; here, when the time when the simulated vehicle arrives at a certain alternative site is within the service time window of the site, the expected waiting time is 0; if the time when the simulated vehicle arrives at a certain alternative site is before the start time of the service time window of the site, the expected waiting time is the difference between the start time and the arrival time.

[0104] Then, based on the first feature expression and the second feature expression, a site feature expression of each candidate site is generated. For example, by vector concatenation, the first feature expression and the second feature expression are concatenated into a vector as the site feature expression of the candidate site.

[0105] Then, a global feature is generated based on the site feature expressions of all candidate sites. For example, the average value of the site feature expressions of all candidate sites is calculated to obtain the global feature.

[0106] Then, the global feature is used as an index and an attention mechanism is used to generate a selection probability value for each candidate site.

[0107] Step 1035 , determine whether the remaining capacity of the simulated vehicle meets the delivery demand of the current target site. If so, proceed to step 1037 , otherwise, proceed to step 1036 .

[0108] Here, when delivering goods to the current target site or picking up goods at the current target site, whether the delivery requirements of the current target site are met is determined based on whether the volume of the delivered goods / picked up goods is less than or equal to the remaining capacity of the simulated vehicle. When delivering goods to the current target site and also picking up goods at the current target site, the delivery requirements of the current target site are determined to be met only if the volume of the delivered goods is less than or equal to the remaining capacity of the simulated vehicle, and the volume of the picked up goods is less than or equal to the remaining capacity of the simulated vehicle; otherwise, the delivery requirements of the current target site are determined to be met.

[0109] Step 1036, reset the global time, return the simulated vehicle to the starting station and enter step 1032.

[0110] Step 1037 , updates the geographic location of the simulated vehicle to the geographic location of the current target site, and updates the global time, the remaining capacity of the simulated vehicle, and the access status of the customer site, and then returns to step 1031 .

[0111] Here, the remaining capacity of the simulated vehicle is updated. The updated remaining capacity is: the remaining capacity before the update minus the capacity of the goods picked up minus the capacity of the goods delivered. In addition, the visit status of the current target station is updated to visited.

[0112] Step 1038: Obtain a total site sequence including the order in which each site is visited, and take the site sequence that starts from the originating site, passes through one or more customer sites, and returns to the originating site as a site sequence segment. Divide the total site sequence into at least one site sequence segment and output it, wherein each site sequence segment corresponds to the delivery path of a delivery vehicle.

[0113] Through the above steps, the embodiment of the present invention uses the trained path strategy search model to generate the delivery path of each delivery vehicle.

[0114] Before the above step 101, the embodiment of the present invention can pre-train a path strategy search model, such as Figure 3 As shown, the specific training steps include:

[0115] Step 301: Obtain a training set, which includes multiple sets of training data. Each set of training data includes a departure station, a maximum load capacity of a simulated vehicle, geographic locations of multiple customer stations, a delivery demand of each customer station, and a service time window.

[0116] Step 302: Initialize a reference model of a path strategy search model.

[0117] Step 303: Select a set of unused training data, and perform a round of training using the selected training data to train the path strategy search model, wherein at the beginning of each round of training, the path strategy search model adopts the current reference model; during each round of training, the model parameters of the path strategy search model are updated using the training data, and when the performance of the updated path strategy search model is improved relative to the current reference model, the reference model is updated according to the updated path strategy search model.

[0118] Step 304: At the end of each round of training, determine whether the preset network training end condition is met. If so, proceed to step 305; otherwise, return to step 303.

[0119] Here, the network training termination condition is: after M consecutive rounds of training, the performance improvement of the updated path strategy search model relative to the current reference model does not meet a preset condition; M is a preset integer greater than 2, such as 10. For details on determining whether the performance improvement meets the preset condition, please refer to the description of step 3035 below.

[0120] Step 305 , ending the training of the path strategy search model, and outputting the current reference model as the path strategy search model obtained through final training.

[0121] Through the above iterative training process, the path strategy search model is finally trained and obtained.

[0122] In the above step 303, a round of training is performed using a set of selected training data to train the path strategy search model, such as Figure 4 As shown, specifically including:

[0123] Step 3031: Initialize the path strategy search model according to the current reference model.

[0124] Step 3032: Use the selected training data as input data and execute Figure 2 The path generation step shown obtains the path strategy corresponding to the training data.

[0125] Step 3033: Calculate the reward function value according to the path strategy corresponding to the training data.

[0126] In order to achieve dual control of the total path cost and the number of delivery vehicles, when calculating the reward function value in the above step 3033, the embodiment of the present invention generates a main term of the reward function value based on the total distance of the path strategy corresponding to the training data; generates a penalty term corresponding to the number of vehicles based on the distance between the last two stations in each station sequence in the path strategy corresponding to the training data; and then, calculates the reward function value based on the main term and the penalty term.

[0127] Here, the main term represents the cost corresponding to the total distance, which is generally positively correlated with the total length of the distance. The penalty term represents the cost corresponding to the number of delivery vehicles and is generally positively correlated with the number of vehicles. In the above calculation, the distance between the last two stations in each station sequence can be expressed as the sum of the products of a preset penalty coefficient alpha. Since the last station in each station sequence is the starting station, the distance between the last two stations refers to the distance between the previous station and the starting station before the vehicle returns to the starting station.

[0128] Step 3034: Update the model parameters of the path strategy search model according to the reward function value.

[0129] Step 3035 , determining whether the performance improvement of the updated path strategy search model relative to the current reference model meets a preset condition; if so, proceed to step 3036 ; otherwise, proceed to step 3037 .

[0130] Step 3036: Update the reference model according to the updated path strategy search model, and return to step 3032.

[0131] Step 3037, end the current round of training.

[0132] Here, determining whether the performance improvement of the updated routing strategy search model relative to the current reference model does not meet the preset conditions can be achieved through verification data. Specifically, a set of verification data (the verification data includes the originating station to be routed, the maximum load capacity of the simulated vehicle, the geographical location of multiple customer stations, the delivery requirements and service time windows of each customer station) can be used as input data and input into the updated routing strategy search model to obtain a first routing strategy corresponding to the verification data; the verification data is input into the reference model to obtain a second routing strategy corresponding to the verification data; a paired t-test result is calculated for the alternative probability value distribution corresponding to the first routing strategy and the alternative probability value distribution corresponding to the second routing strategy; and then, based on whether the paired t-test result is less than a preset threshold, determining whether the performance improvement of the updated routing strategy search model relative to the current reference model meets the preset conditions. For example, if it is less than the preset threshold, then the performance improvement is determined to meet the preset conditions; otherwise, the performance improvement is determined to not meet the preset conditions.

[0133] It can be seen from the above steps that the embodiment of the present invention not only retains the total distance as an optimization item in the reward function, but also, considering that the number of vehicles is equivalent to the number of times (number of trips) that the simulated vehicles return to the starting station, a weighted penalty is applied to all paths returning to the starting station in the final function, so that the reward function achieves dual control over the total distance and the number of vehicles.

[0134] The above describes the method of the embodiment of the present invention. It can be seen that compared with the prior art, the embodiment of the present invention has the following characteristics:

[0135] 1) Utilize a serial decision-making search mechanism. A single vehicle is used to search the global space. Simulating multiple constraints on a single vehicle significantly reduces the search complexity. Ultimately, each segment of the vehicle's path is assigned to different vehicles in the actual production environment, achieving equivalent results.

[0136] 2) A dual reward function design is adopted. In the reward function, not only does the total distance be optimized, but given that the number of vehicles is equivalent to the number of times a simulated vehicle returns to the starting station, a weighted penalty is applied to all paths that return to the starting station in the final function. This allows the reward function to achieve dual control over the total distance and the number of vehicles.

[0137] 3) Use semi-greedy search: During the search process, customer stations whose estimated arrival time exceeds the trailing edge of their time window are directly screened out; for stations whose estimated arrival time is ahead of the leading edge of the time window, vehicles are allowed to wait, but the waiting time is taken into account in the station decision-making.

[0138] 4) In the training phase, the embodiments of the present invention use a large amount of training data to help improve the performance of the model, ultimately obtaining a neural network with excellent parameter adjustment and a corresponding parameter set; in the application phase, the model structure can be directly used to input the adjusted network parameters to directly generate a corresponding solution for the specific data distribution.

[0139] Based on the above vehicle path planning method, an embodiment of the present invention further provides a device for implementing the above method.

[0140] Please refer to Figure 5 , an embodiment of the present invention provides a vehicle path planning device 50, comprising:

[0141] The data acquisition module 51 is used to obtain, as input data, the originating station for route planning, the maximum load capacity of the simulated vehicle, the geographical locations of multiple customer stations, the delivery requirements and service time windows of each customer station;

[0142] A model loading module 52 is used to load a pre-trained path strategy search model and initialize the global time and the remaining capacity and geographic location of the simulated vehicle;

[0143] The path generation module 53 is used to generate and output a path strategy using the path strategy search model, wherein the path strategy search model uses a single simulated vehicle to search among all customer sites, and uses a sequence of sites in which the simulated vehicle departs from the starting site, passes through one or more customer sites, and returns to the starting site as the delivery path of a delivery vehicle, thereby obtaining a path strategy for multiple delivery vehicles.

[0144] Optionally, the path generation module includes:

[0145] A first judgment module is used to judge whether there is an unvisited customer site in the input data, and if so, enter the site screening module, otherwise enter the policy output module;

[0146] The site screening module is configured to calculate the arrival time of the simulated vehicle at each unvisited customer site based on the geographical location of the simulated vehicle and the geographical locations of the unvisited customer sites, and screen customer sites whose arrival time is no later than the end time of the service time window of the customer site as candidate sites;

[0147] The second judgment module is used to judge whether the candidate site exists, and if so, enter the probability calculation module, otherwise enter the reset module;

[0148] The probability calculation module is used to calculate the selection probability value of the simulated vehicle to each candidate station, select the candidate station with the largest selection probability value as the current target station, and determine whether the remaining capacity of the simulated vehicle meets the delivery demand of the current target station. If so, enter the state update module; otherwise, enter the reset module;

[0149] The reset module is used to reset the global time and return the simulated vehicle to the starting station and then enter the station screening module;

[0150] The state updating module is used to update the geographical location of the simulated vehicle to the geographical location of the current target site, and update the global time, the remaining capacity of the simulated vehicle and the access status of the customer site, and then return to the first judgment module;

[0151] The strategy output module is used to obtain a total site sequence including the order in which each site is visited, and to take the site sequence that starts from the originating site, passes through one or more customer sites, and returns to the originating site as a site sequence segment. The total site sequence is divided into at least one site sequence segment and then output, wherein each site sequence segment corresponds to the delivery path of a delivery vehicle.

[0152] Optionally, the probability calculation module specifically includes:

[0153] A first calculation module is configured to generate a first characteristic expression for each candidate station based on the geographical location and delivery demand of each candidate station; calculate an expected waiting time after the simulated vehicle arrives at each candidate station, and generate a second characteristic expression for each candidate station based on the expected waiting time; generate a site characteristic expression for each candidate station based on the first characteristic expression and the second characteristic expression; and generate a global feature based on the site characteristic expressions of all candidate stations;

[0154] The second calculation module is used to generate a selection probability value for each candidate site using the global feature as an index and an attention mechanism.

[0155] Optionally, the vehicle path planning device further includes:

[0156] The training module is used to train and obtain the path strategy search model.

[0157] Optionally, the training module specifically includes:

[0158] The first module is used to obtain a training set, wherein the training set includes multiple sets of training data, each set of training data includes an originating station, a maximum load capacity of a simulated vehicle, geographical locations of multiple customer stations, a delivery demand of each customer station, and a service time window;

[0159] The second module is used to initialize a reference model of a path strategy search model;

[0160] a third module, configured to select a set of unused training data and perform a round of training using the selected training data to train the path strategy search model, wherein at the beginning of each round of training, the path strategy search model adopts a current reference model; during each round of training, the model parameters of the path strategy search model are updated using the training data; and when the performance improvement of the updated path strategy search model relative to the current reference model satisfies a preset condition, the reference model is updated according to the updated path strategy search model;

[0161] The fourth module is used to determine whether the preset network training end conditions are met at the end of each round of training. If so, the training of the path strategy search model is ended and the current reference model is output as the path strategy search model obtained by the final training; otherwise, the third module is returned.

[0162] Optionally, the third module includes:

[0163] A first submodule is configured to initialize the path strategy search model according to the current reference model;

[0164] The second submodule is configured to use the selected training data as input data, execute the path generation module, and obtain a path strategy corresponding to the training data;

[0165] A third submodule is used to calculate a reward function value according to the path strategy corresponding to the training data;

[0166] A fourth submodule is configured to update the model parameters of the path strategy search model according to the reward function value;

[0167] The fifth submodule is used to determine whether the performance improvement of the updated path strategy search model relative to the current reference model meets the preset conditions. If so, the reference model is updated according to the updated path strategy search model and returns to the second submodule. Otherwise, the current round of training is terminated.

[0168] Optionally, the third submodule is specifically configured to:

[0169] generating a main item according to the total distance of the path strategy corresponding to the training data;

[0170] Generate a penalty term corresponding to the number of vehicles based on the distance between the last two stations in each station sequence in the routing strategy corresponding to the training data;

[0171] The reward function value is calculated based on the main term and the penalty term.

[0172] Optionally, the fifth submodule is specifically configured to:

[0173] Input a set of verification data as input data into the updated path strategy search model to obtain a first path strategy corresponding to the verification data; input the verification data into the reference model to obtain a second path strategy corresponding to the verification data; calculate the paired t-test results of the alternative probability value distribution corresponding to the first path strategy and the alternative probability value distribution corresponding to the second path strategy;

[0174] According to whether the paired t-test result is less than a preset threshold, it is determined whether the performance improvement degree of the updated path strategy search model relative to the current reference model meets a preset condition.

[0175] Optionally, the network training termination condition is: the performance improvement of the updated path strategy search model obtained after M consecutive rounds of training relative to the current reference model does not meet a preset condition; and M is a preset integer greater than 2.

[0176] Through the above device, the embodiment of the present invention can improve the performance and efficiency of path planning in vehicle path planning.

[0177] like Figure 6 As shown, an embodiment of the present invention further provides another vehicle path planning device 60, which specifically includes a processor 61, a memory 62, a bus system 63, a receiver 64, and a transmitter 65. The processor 61, the memory 62, the receiver 64, and the transmitter 65 are connected via the bus system 63. The memory 62 is used to store instructions, and the processor 61 is used to execute the instructions stored in the memory 62 to control the receiver 64 to receive signals and control the transmitter 65 to send signals.

[0178] The processor 61 is configured to read the program in the memory and execute the following process:

[0179] A data acquisition step, obtaining the originating station for route planning, the maximum load of the simulated vehicle, the geographical locations of multiple customer stations, the delivery demand and service time window of each customer station as input data;

[0180] A model loading step, loading a pre-trained path strategy search model, and initializing the global time and the remaining capacity and geographic location of the simulated vehicle;

[0181] The path generation step uses the path strategy search model to generate and output a path strategy, wherein the path strategy search model uses a single simulated vehicle to search among all customer sites, and uses a sequence of sites in which the simulated vehicle departs from the starting site, passes through one or more customer sites, and returns to the starting site as the delivery path of a delivery vehicle, thereby obtaining the path strategy for multiple delivery vehicles.

[0182] It should be understood that in the embodiment of the present invention, the processor 61 may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0183] The memory 62 may include a read-only memory and a random access memory, and provides instructions and data to the processor 61. A portion of the memory 62 may also include a non-volatile random access memory. For example, the memory 62 may also store information about the device type.

[0184] In addition to the data bus, the bus system 63 may also include a power bus, a control bus, a status signal bus, etc. However, for the sake of clarity, all the buses are labeled as the bus system 63 in the figure.

[0185] During implementation, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor 61 or by instructions in the form of software. The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 62, and the processor 61 reads the information in the memory 62 and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it will not be described in detail here.

[0186] According to at least one embodiment of the present invention, when the program is executed by the processor 61, the following steps may also be implemented:

[0187] The first judgment step is to judge whether there is an unvisited customer site in the input data, and if so, enter the site screening step, otherwise enter the policy output step;

[0188] The site screening step includes calculating the arrival time of the simulated vehicle at each unvisited customer site based on the geographical location of the simulated vehicle and the geographical location of the unvisited customer site, and screening customer sites whose arrival time is no later than the end time of the service time window of the customer site as candidate sites;

[0189] The second judgment step is to judge whether the candidate site exists, if so, to enter the probability calculation step, otherwise to enter the reset step;

[0190] The probability calculation step calculates the probability of the simulated vehicle going to each candidate station, selects the candidate station with the largest selection probability as the current target station, and determines whether the remaining capacity of the simulated vehicle meets the delivery demand of the current target station. If so, the state update step is entered; otherwise, the reset step is entered;

[0191] The resetting step resets the global time, returns the simulated vehicle to the originating station, and then enters the station screening step;

[0192] The state updating step updates the geographical location of the simulated vehicle to the geographical location of the current target site, and updates the global time, the remaining capacity of the simulated vehicle, and the access status of the customer site, and then returns to the first judgment step;

[0193] The strategy output step obtains a total site sequence including the order in which each site is visited, and takes the site sequence starting from the originating site, passing through one or more customer sites, and returning to the originating site as a site sequence. The total site sequence is divided into at least one site sequence and then output, wherein each site sequence corresponds to the delivery path of a delivery vehicle.

[0194] According to at least one embodiment of the present invention, when the program is executed by the processor 61, the following steps may also be implemented:

[0195] Generate a first feature expression for each candidate site based on the geographic location and delivery demand of each candidate site; calculate the expected waiting time after the simulated vehicle arrives at each candidate site, and generate a second feature expression for each candidate site based on the expected waiting time; generate a site feature expression for each candidate site based on the first and second feature expressions; and generate a global feature based on the site feature expressions of all candidate sites;

[0196] Using the global feature as an index, an attention mechanism is used to generate a selection probability value for each candidate site.

[0197] According to at least one embodiment of the present invention, when the program is executed by the processor 61, the following steps may also be implemented:

[0198] The training step is to obtain the path strategy search model through training.

[0199] According to at least one embodiment of the present invention, when the program is executed by the processor 61, the following steps may also be implemented:

[0200] The first step is to obtain a training set, wherein the training set includes multiple sets of training data, each set of training data includes an originating station, a maximum load capacity of a simulated vehicle, geographical locations of multiple customer stations, a delivery demand of each customer station, and a service time window;

[0201] The second step is to initialize a reference model of the path strategy search model;

[0202] The third step is to select a set of unused training data and perform a round of training using the selected training data to train the path strategy search model, wherein at the beginning of each round of training, the path strategy search model uses the current reference model; during each round of training, the model parameters of the path strategy search model are updated using the training data, and when the performance improvement of the updated path strategy search model relative to the current reference model meets a preset condition, the reference model is updated according to the updated path strategy search model;

[0203] The fourth step is to determine whether the preset network training end conditions are met at the end of each round of training. If so, the training of the path strategy search model is terminated, and the current reference model is output as the path strategy search model obtained by the final training; otherwise, return to the third step.

[0204] According to at least one embodiment of the present invention, when the program is executed by the processor 61, the following steps may also be implemented:

[0205] The first sub-step is to initialize the path strategy search model according to the current reference model;

[0206] The second sub-step is to use the selected training data as input data, execute the path generation step, and obtain the path strategy corresponding to the training data;

[0207] The third sub-step is to calculate the reward function value according to the path strategy corresponding to the training data;

[0208] A fourth sub-step is to update the model parameters of the path strategy search model according to the reward function value;

[0209] The fifth sub-step is to determine whether the performance improvement of the updated path strategy search model relative to the current reference model meets the preset conditions. If so, update the reference model according to the updated path strategy search model and return to the second sub-step. Otherwise, end the current round of training.

[0210] According to at least one embodiment of the present invention, when the program is executed by the processor 61, the following steps may also be implemented:

[0211] generating a main item according to the total distance of the path strategy corresponding to the training data;

[0212] Generate a penalty term corresponding to the number of vehicles based on the distance between the last two stations in each station sequence in the routing strategy corresponding to the training data;

[0213] The reward function value is calculated based on the main term and the penalty term.

[0214] According to at least one embodiment of the present invention, when the program is executed by the processor 61, the following steps may also be implemented:

[0215] Input a set of verification data as input data into the updated path strategy search model to obtain a first path strategy corresponding to the verification data; input the verification data into the reference model to obtain a second path strategy corresponding to the verification data; calculate the paired t-test results of the alternative probability value distribution corresponding to the first path strategy and the alternative probability value distribution corresponding to the second path strategy;

[0216] According to whether the paired t-test result is less than a preset threshold, it is determined whether the performance improvement degree of the updated path strategy search model relative to the current reference model meets a preset condition.

[0217] According to at least one embodiment of the present invention, the network training termination condition is: the performance improvement of the updated path strategy search model obtained after M consecutive rounds of training relative to the current reference model does not meet the preset conditions; M is a preset integer greater than 2.

[0218] When the program is executed by the processor, it can achieve Figures 1 to 4 All implementations of the vehicle path planning method shown can achieve the same technical effect. To avoid repetition, they will not be described here.

[0219] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0220] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0221] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0222] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the objectives of the embodiments of the present invention.

[0223] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0224] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0225] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A vehicle path planning method, characterized in that: include: A data acquisition step, obtaining the originating station for route planning, the maximum load of the simulated vehicle, the geographical locations of multiple customer stations, the delivery demand and service time window of each customer station as input data; A model loading step, loading a pre-trained path strategy search model, and initializing the global time and the remaining capacity and geographic location of the simulated vehicle; a route generation step, using the route strategy search model to generate and output a route strategy, wherein the route strategy search model uses a single simulated vehicle to search through all customer sites, and uses a sequence of sites in which the simulated vehicle departs from a starting site, passes through one or more customer sites, and returns to the starting site as a delivery route for a delivery vehicle, thereby obtaining route strategies for multiple delivery vehicles; The step of using the path strategy search model to generate and output a path strategy includes: The first judgment step is to judge whether there is an unvisited customer site in the input data, and if so, enter the site screening step, otherwise enter the policy output step; The site screening step includes calculating the arrival time of the simulated vehicle at each unvisited customer site based on the geographical location of the simulated vehicle and the geographical location of the unvisited customer site, and screening customer sites whose arrival time is no later than the end time of the service time window of the customer site as candidate sites; The second judgment step is to judge whether the candidate site exists, if so, to enter the probability calculation step, otherwise to enter the reset step; The probability calculation step calculates the probability of the simulated vehicle going to each candidate station, selects the candidate station with the largest selection probability as the current target station, and determines whether the remaining capacity of the simulated vehicle meets the delivery demand of the current target station. If so, the state update step is entered; otherwise, the reset step is entered; The resetting step resets the global time, returns the simulated vehicle to the originating station, and then enters the station screening step; The state updating step updates the geographical location of the simulated vehicle to the geographical location of the current target site, and updates the global time, the remaining capacity of the simulated vehicle, and the access status of the customer site, and then returns to the first judgment step; The strategy output step obtains a total site sequence including the order in which each site is visited, and defines a site sequence starting from the originating site, passing through one or more customer sites, and returning to the originating site as a site sequence segment. The total site sequence is divided into at least one site sequence segment and then output, wherein each site sequence segment corresponds to a delivery route for a delivery vehicle; The calculation of the probability of the simulated vehicle going to each candidate station includes: Generate a first feature expression for each candidate site based on the geographic location and delivery demand of each candidate site; calculate the expected waiting time after the simulated vehicle arrives at each candidate site, and generate a second feature expression for each candidate site based on the expected waiting time; generate a site feature expression for each candidate site based on the first and second feature expressions; and generate a global feature based on the site feature expressions of all candidate sites; Using the global feature as an index, an attention mechanism is used to generate a selection probability value for each candidate site.

2. The method according to claim 1, wherein Before the data input step, the method further includes: The training step is to obtain the path strategy search model through training.

3. The method according to claim 2, wherein The training steps specifically include: The first step is to obtain a training set, wherein the training set includes multiple sets of training data, each set of training data includes an originating station, a maximum load capacity of a simulated vehicle, geographical locations of multiple customer stations, a delivery demand of each customer station, and a service time window; The second step is to initialize a reference model of the path strategy search model; The third step is to select a set of unused training data and perform a round of training using the selected training data to train the path strategy search model, wherein at the beginning of each round of training, the path strategy search model uses the current reference model; during each round of training, the model parameters of the path strategy search model are updated using the training data, and when the performance improvement of the updated path strategy search model relative to the current reference model meets a preset condition, the reference model is updated according to the updated path strategy search model; The fourth step is to determine whether the preset network training end conditions are met at the end of each round of training. If so, the training of the path strategy search model is terminated, and the current reference model is output as the path strategy search model obtained by the final training; otherwise, return to the third step.

4. The method according to claim 3, wherein A round of training is performed using the selected training data to train the path strategy search model, including: The first sub-step is to initialize the path strategy search model according to the current reference model; The second sub-step is to use the selected training data as input data, execute the path generation step, and obtain the path strategy corresponding to the training data; The third sub-step is to calculate the reward function value according to the path strategy corresponding to the training data; A fourth sub-step is to update the model parameters of the path strategy search model according to the reward function value; The fifth sub-step is to determine whether the performance improvement of the updated path strategy search model relative to the current reference model meets the preset conditions. If so, update the reference model according to the updated path strategy search model and return to the second sub-step. Otherwise, end the current round of training.

5. The method according to claim 4, wherein The calculating the reward function value according to the path strategy corresponding to the training data includes: generating a main item according to the total distance of the path strategy corresponding to the training data; Generate a penalty term corresponding to the number of vehicles based on the distance between the last two stations in each station sequence in the routing strategy corresponding to the training data; The reward function value is calculated based on the main term and the penalty term.

6. The method according to claim 4, wherein The determining whether the performance improvement of the updated path strategy search model relative to the current reference model meets a preset condition includes: Input a set of verification data as input data into the updated path strategy search model to obtain a first path strategy corresponding to the verification data; input the verification data into the reference model to obtain a second path strategy corresponding to the verification data; calculate the paired t-test results of the alternative probability value distribution corresponding to the first path strategy and the alternative probability value distribution corresponding to the second path strategy; According to whether the paired t-test result is less than a preset threshold, it is determined whether the performance improvement degree of the updated path strategy search model relative to the current reference model meets a preset condition.

7. The method according to claim 3, wherein The network training termination condition is: the updated path strategy search model obtained after M consecutive rounds of training does not meet the preset condition for the performance improvement relative to the current reference model; M is a preset integer greater than 2.

8. A vehicle path planning device, characterized in that: include: A data acquisition module is used to obtain, as input data, the originating station for route planning, the maximum load capacity of the simulated vehicle, the geographical locations of multiple customer stations, the delivery demand and service time window of each customer station; A model loading module is used to load a pre-trained path strategy search model and initialize the global time and the remaining capacity and geographic location of the simulated vehicle; a route generation module, configured to generate and output a route strategy using the route strategy search model, wherein the route strategy search model uses a single simulated vehicle to search through all customer sites, and uses a sequence of sites, in which the simulated vehicle departs from a starting site, passes through one or more customer sites, and returns to the starting site, as a delivery route for a delivery vehicle, thereby obtaining route strategies for multiple delivery vehicles; Wherein, the path generation module includes: A first judgment module is used to judge whether there is an unvisited customer site in the input data, and if so, enter the site screening module, otherwise enter the policy output module; The site screening module is configured to calculate the arrival time of the simulated vehicle at each unvisited customer site based on the geographical location of the simulated vehicle and the geographical locations of the unvisited customer sites, and screen customer sites whose arrival time is no later than the end time of the service time window of the customer site as candidate sites; The second judgment module is used to judge whether the candidate site exists, and if so, enter the probability calculation module, otherwise enter the reset module; The probability calculation module is used to calculate the selection probability value of the simulated vehicle to each candidate station, select the candidate station with the largest selection probability value as the current target station, and determine whether the remaining capacity of the simulated vehicle meets the delivery demand of the current target station. If so, enter the state update module; otherwise, enter the reset module; The reset module is used to reset the global time and return the simulated vehicle to the starting station and then enter the station screening module; The state updating module is used to update the geographical location of the simulated vehicle to the geographical location of the current target site, and update the global time, the remaining capacity of the simulated vehicle and the access status of the customer site, and then return to the first judgment module; The strategy output module is configured to obtain a total site sequence including the order in which each site is visited, and to divide the site sequence starting from the originating site, passing through one or more customer sites, and returning to the originating site into at least one site sequence segment, and then output the segmented site sequence, wherein each segment of the site sequence corresponds to a delivery route for a delivery vehicle; The calculation of the probability of the simulated vehicle going to each candidate station includes: Generate a first feature expression for each candidate site based on the geographic location and delivery demand of each candidate site; calculate the expected waiting time after the simulated vehicle arrives at each candidate site, and generate a second feature expression for each candidate site based on the expected waiting time; generate a site feature expression for each candidate site based on the first and second feature expressions; and generate a global feature based on the site feature expressions of all candidate sites; Using the global feature as an index, an attention mechanism is used to generate a selection probability value for each candidate site.

9. The vehicle path planning device according to claim 8, characterized in that: Also includes: The training module is used to train and obtain the path strategy search model.

10. The vehicle path planning device according to claim 9, characterized in that: The training module specifically includes: The first module is used to obtain a training set, wherein the training set includes multiple sets of training data, each set of training data includes an originating station, a maximum load capacity of a simulated vehicle, geographical locations of multiple customer stations, a delivery demand of each customer station, and a service time window; The second module is used to initialize a reference model of a path strategy search model; a third module, configured to select a set of unused training data and perform a round of training using the selected training data to train the path strategy search model, wherein at the beginning of each round of training, the path strategy search model adopts a current reference model; during each round of training, the model parameters of the path strategy search model are updated using the training data; and when the performance improvement of the updated path strategy search model relative to the current reference model satisfies a preset condition, the reference model is updated according to the updated path strategy search model; The fourth module is used to determine whether the preset network training end conditions are met at the end of each round of training. If so, the training of the path strategy search model is ended and the current reference model is output as the path strategy search model obtained by the final training; otherwise, the third module is returned.

11. The vehicle path planning device according to claim 10, wherein: The third module includes: A first submodule is configured to initialize the path strategy search model according to the current reference model; The second submodule is configured to use the selected training data as input data, execute the path generation module, and obtain a path strategy corresponding to the training data; A third submodule is used to calculate a reward function value according to the path strategy corresponding to the training data; A fourth submodule is configured to update the model parameters of the path strategy search model according to the reward function value; The fifth submodule is used to determine whether the performance improvement of the updated path strategy search model relative to the current reference model meets the preset conditions. If so, the reference model is updated according to the updated path strategy search model and returns to the second submodule. Otherwise, the current round of training is terminated.

12. The vehicle path planning device according to claim 11, wherein: The third submodule is specifically used for: generating a main item according to the total distance of the path strategy corresponding to the training data; Generate a penalty term corresponding to the number of vehicles based on the distance between the last two stations in each station sequence in the routing strategy corresponding to the training data; The reward function value is calculated based on the main term and the penalty term.

Images