Path decision method for optimizing punctual arrival probability in combination with Decision Transform and GPG

Abstract

The invention discloses a path decision-making method for optimizing the punctual arrival probability in combination with a Decision Transform and a GPG (General Purpose Graphics). The method comprises the following steps: acquiring a vehicle initial path data set; constructing a vehicle path decision model based on a Decision Transform, and obtaining probability distribution of vehicle path node selection by using the vehicle initial path data set and the vehicle path decision model to generate a vehicle prediction path; and calculating a strategy gradient by taking the punctual arrival probability as an optimization target based on the vehicle predicted path and the GPG, carrying out back propagation training on the vehicle path decision model by utilizing the strategy gradient, and carrying out vehicle path decision by utilizing the vehicle path decision model. According to the method, the problem that spatial correlation modeling is insufficient in the prior art can be solved, the accuracy and stability of path decision making are improved, meanwhile, dependence on priori knowledge can be reduced in the optimization decision making process, and then the punctual arrival probability of optimization path decision making can be achieved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a path decision method for optimizing on-time arrival probability by combining Decision Transformer and GPG (Generalized Policy Gradient Algorithm). Background Art

[0002] Reliable navigation in random traffic networks has long been a cornerstone of intelligent transportation systems. Among the many reliable navigation objectives, optimizing the probability of on-time arrival has become a widely used key objective, particularly in various time-sensitive navigation scenarios.

[0003] Traditional routing decision-making methods can be categorized as data-driven or model-driven. Data-driven methods typically assume that routing times follow a Markov process, meaning that future routing times are unaffected by past ones. However, this assumption is difficult to establish in practice, resulting in poor adaptability to complex road networks. Model-driven methods typically require assumptions about the distribution of routing times and rely on strong prior knowledge, making it difficult to accurately describe the true distribution of routing times and lacking sufficient spatial correlation modeling. Furthermore, neither data-driven nor model-driven methods effectively incorporate the dynamically changing spatiotemporal characteristics of routing decisions, resulting in a low probability of on-time arrival. Summary of the Invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a path decision method that combines Decision Transformer and GPG to optimize the probability of on-time arrival.

[0005] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0006] A path decision method for optimizing on-time arrival probability by combining Decision Transformer and GPG is characterized by comprising the following steps:

[0007] S1. Obtaining the initial vehicle path dataset based on the road network;

[0008] S2. Build a vehicle routing decision model based on Decision Transformer, use the vehicle initial routing dataset and the vehicle routing decision model to obtain the probability distribution of vehicle routing node selection, and generate the vehicle predicted route based on the probability distribution of vehicle routing node selection;

[0009] S3. Based on the vehicle predicted path and GPG, the policy gradient is calculated with the on-time arrival probability as the optimization goal. The vehicle path decision model is trained through backpropagation using the policy gradient, and the vehicle path decision model after backpropagation training is used to make vehicle path decisions.

[0010] Furthermore, in step S1 , the vehicle initial path data set includes vehicle path node data, vehicle node passing time data, vehicle remaining time budget data, and vehicle current node data.

[0011] Furthermore, in step S2, the vehicle path decision model includes an attention embedding layer, a linear embedding layer, an encoder and a decoder; the input end of the attention embedding layer serves as the first data input end, the output end of the attention embedding layer is connected to the input end of the encoder, the output end of the encoder is connected to the first input end of the decoder, the input end of the linear embedding layer serves as the second data input end, the output end of the linear embedding layer is connected to the second input end of the decoder, and the output end of the decoder serves as the output of the vehicle path decision model; the data input to the first data input end is the vehicle path node data and the vehicle node passing time data; the data input to the second data input end is the vehicle remaining time budget data and the vehicle current node data.

[0012] Furthermore, the expression for data processing in the attention embedding layer is:

[0013] P′=EmbeddingP(P),

[0014] P=[p1,p2,...,p L ],

[0015] R′=W R R+b R ,

[0016] R=[r1,r2,...,r L ],

[0017] X fusion =Attention(P′,R′,R′)

[0018] Where: P' is the embedded data of the node passed by the vehicle path, EmbeddingP is the node lookup table passed by the vehicle path, P is the node data passed by the vehicle path, p1 is the first node passed by the vehicle path, p2 is the second node passed by the vehicle path, p L is the Lth node that the vehicle path passes through, L is the number of nodes that the vehicle path passes through, R′ is the embedded data of the vehicle node passing time, W R is the weight coefficient of the vehicle node passing time, R is the vehicle node passing time data, b R is the bias coefficient of the vehicle node passing time, r1 is the vehicle passing time at the first node, r2 is the vehicle passing time at the second node, and r L is the passing time of the vehicle at the Lth node, Xfusion is the fusion feature of the vehicle, and Attention is the standard self-attention mechanism.

[0019] Furthermore, the Encoder includes 4 layers of TransformerBlock. The expression for data processing in the first layer of TransformerBlock of the Encoder is:

[0020] X out1 =TransformerBlock(X in1 + R′),

[0021] X in1 =Dropout(X fusion )+Embedding(P′)

[0022] Where: X out1 is the output of the first layer TransformerBlock of Encoder, X in1 is the input of the first layer TransformerBlock of Encoder, R′ is the time embedding data of vehicle nodes, Dropout is the regularization process, X fusion is the fusion feature of the vehicle, Embedding is the position encoding process, and P′ is the embedded data of the nodes passed by the vehicle path;

[0023] The output of the first layer TransformerBlock of Encoder is used as the input of the second layer TransformerBlock of Encoder, and the above processing is carried out in sequence. The output X of the fourth layer TransformerBlock of Encoder out4 As the output X of Encoder enc .

[0024] Furthermore, the expression for data processing in the linear embedding layer is:

[0025] p s ′=EmbeddingS(p s ),

[0026] B′=W B B+b B

[0027] Where: p s ′ is the embedding data of the vehicle’s current node, EmbeddingS is the vehicle’s current node lookup table, and p s is the vehicle’s current node data, B′ is the vehicle’s remaining time budget embedded data, and W Bis the weight coefficient of the vehicle's remaining time budget, B is the vehicle's remaining time budget data, and b B Bias factor for the vehicle's remaining time budget.

[0028] Furthermore, the Decoder consists of 4 layers of TransformerBlock and prediction module;

[0029] The expression for data processing in the first layer TransformerBlock of Decoder is:

[0030] X out1 ′=TransformerBlock(X in1 ′,X enc ,X enc ),

[0031] X in1 ′=Dropout(B′+p s ′)

[0032] Where: X out1 ′ is the output of the first layer TransformerBlock of Decoder, X enc is the output of Encoder, X in1 ′ is the input of the first layer TransformerBlock of Decoder, Dropout is the regularization process, p s ′ is the vehicle’s current node embedded data, and B′ is the vehicle’s remaining time budget embedded data;

[0033] The output of the first layer TransformerBlock of the Decoder is used as the input of the second layer TransformerBlock of the Decoder. The above processing is carried out in sequence. The output X of the fourth layer TransformerBlock of the Decoder out4 As the input of the Decoder prediction module;

[0034] The expression for data processing in the Decoder prediction module is:

[0035] y=softmax(fc(X out4 ′))

[0036] Where: y is the probability of vehicle path node selection, softmax is the softmax activation function, fc is the fully connected layer, X out4 ′ is the output of the fourth layer TransformerBlock of Decoder.

[0037] Furthermore, in step S3, based on the predicted vehicle path and GPG, the policy gradient is calculated with the on-time arrival probability as the optimization target, including the following steps:

[0038] A1. Calculate the required time based on the vehicle's predicted path;

[0039] A2. Based on the predicted route duration of the vehicle, the objective function of the vehicle routing decision model is constructed with the probability of on-time arrival as the optimization goal;

[0040] A3. Based on the objective function of the vehicle path decision model and GPG, an initial policy gradient model is constructed. The Monte Carlo sampling method and the fixed numerical baseline method are used to update the initial policy gradient model to obtain the final policy gradient model. The policy gradient is calculated using the final policy gradient model.

[0041] Furthermore, in step A3, the final policy gradient model is expressed as:

[0042]

[0043] in: is the policy gradient, M is the number of vehicle predicted paths generated by the vehicle path decision model through the Monte Carlo sampling method, j is the predicted path number generated by the vehicle path decision model through the Monte Carlo sampling method, For the condition R(τ (j) )≤T indicator function, indicating that when the condition R(τ (j) )≤T is true, and the value is 1 when the condition R(τ (j) )≤T is false, the value is 0, R(τ (j) ) is the vehicle predicted path τ (j) The required time, τ (j) is the jth vehicle predicted path generated by the Monte Carlo sampling method of the vehicle path decision model, T is the target time, b is the baseline value, and k is the vehicle predicted path τ (j) The node number in H j Predict the path τ for the vehicle (j) The number of nodes in is the logarithmic policy gradient, The current state of the vehicle is Execute action when The corresponding strategy, To select the execution action corresponding to the k+1th node at the kth node in the jth vehicle prediction path, is the state of the kth node in the predicted path of the jth vehicle.

[0044] Furthermore, the expression of the node status in the vehicle prediction path is:

[0045] s k =(p s，k ′,y k+1 , B k ′,S k ),

[0046] S k =(p k-1 ′, r k ′,S k-1 ,a k-1 )

[0047] Where: s k is the state of the kth node in the vehicle’s predicted path, p s，k ′ is the vehicle’s current node embedding data at the kth node in the vehicle’s predicted path, y k+1 The probability of selecting the k+1th node at the kth node for the vehicle prediction path, B k ′ is the vehicle remaining time budget embedding data at the kth node in the vehicle prediction path, S k is the vehicle history data at the kth node in the vehicle prediction path, p k-1 ′ is the k-1th node embedding data of the vehicle’s predicted path, r k ′ is the embedded data of the vehicle’s passing time at the kth node in the vehicle’s predicted path, S k-1 is the vehicle history data at the k-1th node in the vehicle prediction path, a k-1 Select the execution action corresponding to the kth node at the k-1th node in the vehicle's predicted path.

[0048] The present invention has the following beneficial effects:

[0049] (1) By constructing a vehicle routing decision model based on the Decision Transformer, the present invention can leverage the Decision Transformer's ability to handle long-term temporal dependencies, effectively modeling the dynamic changes in routing time, addressing the problem of insufficient spatial correlation modeling in the prior art, and thereby improving the accuracy and stability of routing decisions.

[0050] (2) The present invention calculates the policy gradient based on the vehicle predicted path and GPG, takes the on-time arrival probability as the optimization target, uses the policy gradient to perform back-propagation training on the vehicle routing decision model, and uses the vehicle routing decision model trained by back-propagation to make vehicle routing decisions. This process can reduce the dependence on prior knowledge, thereby achieving the on-time arrival probability of the optimized routing decision;

[0051] (3) The present invention updates the initial policy gradient model by adopting the Monte Carlo sampling method and the fixed numerical baseline method, which can reduce the variance of the policy update during the reinforcement learning process and improve the training convergence. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A flowchart of the routing decision method that combines Decision Transformer and GPG to optimize the probability of on-time arrival;

[0053] Figure 2 Schematic diagram of the data processing and back propagation training process of the path decision model of the present invention. DETAILED DESCRIPTION

[0054] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0055] like Figure 1 As shown in FIG, the path decision method for optimizing the probability of on-time arrival by combining Decision Transformer and GPG includes steps S1-S3, which are as follows:

[0056] S1. Obtain the initial vehicle path dataset based on the road network.

[0057] In an optional embodiment of the present invention, the vehicle initial path data set includes vehicle path node data, vehicle node passing time data, vehicle remaining time budget data and vehicle current node data.

[0058] The road network in the present invention is a given road network with spatial correlation. The present invention simulates the road network using the Monte Carlo sampling method to obtain an initial vehicle path dataset. By using the Monte Carlo sampling method to simulate the road network and obtain the initial vehicle path dataset, the present invention avoids making distribution assumptions about vehicle node transit times. This eliminates the need for a mathematical model for path transit time distribution, thereby improving the present invention's adaptability to different road network structures.

[0059] S2. Build a vehicle path decision model based on Decision Transformer, use the vehicle initial path dataset and the vehicle path decision model to obtain the probability distribution of vehicle path node selection, and generate the vehicle predicted path based on the probability distribution of vehicle path node selection.

[0060] In an optional embodiment of the present invention, the vehicle path decision model includes an attention embedding layer, a linear embedding layer, an encoder and a decoder, such as Figure 2 As shown; the input end of the attention embedding layer serves as the first data input end, the output end of the attention embedding layer is connected to the input end of the Encoder, the output end of the Encoder is connected to the first input end of the Decoder, the input end of the linear embedding layer serves as the second data input end, the output end of the linear embedding layer is connected to the second input end of the Decoder, and the output end of the Decoder serves as the output of the vehicle path decision model; the data input to the first data input end are the vehicle path node data and the vehicle node passing time data; the data input to the second data input end are the vehicle remaining time budget data and the vehicle current node data.

[0061] The attention embedding layer is used to convert the vehicle path passing node data into vehicle path passing node embedding data through a lookup table, convert the vehicle node passing time data into vehicle node passing time embedding data through a linear transformation, and fuse the vehicle path passing node embedding data and the vehicle node passing time embedding data through the attention mechanism to obtain the fusion features of the vehicle; the Encoder is used to extract the road network features of the vehicle fusion features to output the deep features of the road network spatial correlation; the linear embedding layer is used to convert the vehicle current node data into vehicle current node embedding data through a lookup table, and convert the vehicle remaining time budget data into vehicle remaining time budget embedding data through a linear transformation; the Decoder is used to process the input deep features of the road network spatial correlation, vehicle current node embedding data and vehicle remaining time budget embedding data through the self-attention mechanism to obtain the probability distribution of vehicle path node selection.

[0062] The expression for data processing in the attention embedding layer is:

[0063] P′=EmbeddingP(P),

[0064] P=[p1,p2,...,p L ],

[0065] R′=W R R+b R ,

[0066] R=[r1,r2,...,r L ],

[0067] X fusion =Attention(P′,R′,R′)

[0068] Where: P' is the embedded data of the node passed by the vehicle path, EmbeddingP is the node lookup table passed by the vehicle path, P is the node data passed by the vehicle path, p1 is the first node passed by the vehicle path, p2 is the second node passed by the vehicle path, p L is the Lth node that the vehicle path passes through, L is the number of nodes that the vehicle path passes through, R′ is the embedded data of the vehicle node passing time, W R is the weight coefficient of the vehicle node passing time, R is the vehicle node passing time data, b R is the bias coefficient of the vehicle node passing time, r1 is the vehicle passing time at the first node, r2 is the vehicle passing time at the second node, and r L is the passing time of the vehicle at the Lth node, X fusion The fusion feature of the vehicle. Attention is a standard self-attention mechanism that can capture the correlation information between path and time to obtain fusion features. The dimension of the attention head is 8 and the number of layers is 2.

[0069] The Encoder consists of 4 layers of TransformerBlock. The expression for data processing in the first layer of TransformerBlock of the Encoder is:

[0070] X out1 =TransformerBlock(X in1 + R′),

[0071] X in1 =Dropout(X fusion )+Embedding(P′)

[0072] Where: X out1 is the output of the first layer TransformerBlock of Encoder. TransformerBlock includes self-attention mechanism and FFN (Feed Forward Network), and uses residual connection and layer normalization to stabilize training. in1 is the input of the first layer TransformerBlock of Encoder, R′ is the time embedding data of vehicle nodes, Dropout is the regularization process, X fusion is the fusion feature of the vehicle, Embedding is the position encoding process, and P′ is the embedded data of the node passed by the vehicle path.

[0073] The output of the first layer TransformerBlock of Encoder is used as the input of the second layer TransformerBlock of Encoder, and the above processing is carried out in sequence. The output X of the fourth layer TransformerBlock of Encoder out4 As the output X of Encoder enc , X enc It is the deep feature of the spatial correlation of the road network.

[0074] The expression for data processing in the linear embedding layer is:

[0075] p s ′=EmbeddingS(p s ),

[0076] B′=W B B+b B

[0077] Where: p s ′ is the embedding data of the vehicle’s current node, EmbeddingS is the vehicle’s current node lookup table, and p s is the vehicle’s current node data, B′ is the vehicle’s remaining time budget embedded data, and W B is the weight coefficient of the vehicle's remaining time budget, B is the vehicle's remaining time budget data, and b B Bias factor for the vehicle's remaining time budget.

[0078] The Decoder consists of 4 layers of TransformerBlock and a prediction module.

[0079] The expression for data processing in the first layer TransformerBlock of Decoder is:

[0080] X out1 ′=TransformerBlock(X in1 ′,X enc ,X enc ),

[0081] X in1 ′=Dropout(B′+p s ′)

[0082] Where: X out1 ′ is the output of the first layer TransformerBlock of Decoder. TransformerBlock includes self-attention mechanism and FFN (Feed Forward Network), and uses residual connection and layer normalization to stabilize training. encis the output of Encoder, X in1 ′ is the input of the first layer TransformerBlock of Decoder, Dropout is the regularization process, p s ′ is the embedded data of the vehicle’s current node, and B′ is the embedded data of the vehicle’s remaining time budget.

[0083] The output of the first layer TransformerBlock of the Decoder is used as the input of the second layer TransformerBlock of the Decoder. The above processing is carried out in sequence. The output X of the fourth layer TransformerBlock of the Decoder out4 As the input of the Decoder's prediction module.

[0084] The expression for data processing in the Decoder prediction module is:

[0085] y=softmax(fc(X out4 ′))

[0086] Where: y is the probability of vehicle path node selection, softmax is the softmax activation function, fc is the fully connected layer, X out4 ′ is the output of the fourth layer TransformerBlock of Decoder.

[0087] S3. Based on the vehicle predicted path and GPG, the policy gradient is calculated with the on-time arrival probability as the optimization goal. The vehicle path decision model is trained through backpropagation using the policy gradient, and the vehicle path decision model after backpropagation training is used to make vehicle path decisions.

[0088] In an optional embodiment of the present invention, the present invention calculates a policy gradient based on the vehicle predicted path and GPG with the on-time arrival probability as the optimization target, including the following steps:

[0089] A1. Calculate the required time based on the vehicle's predicted path.

[0090] A2. Based on the predicted route duration of the vehicle, the objective function of the vehicle routing decision model is constructed with the probability of on-time arrival as the optimization goal.

[0091] The objective function of the vehicle routing decision model is expressed as:

[0092] J s (π θ )=P[R(τ)≤T]

[0093] Among them: J s (π θ ) is the path decision model π θThe objective function of θ is the weight parameter of the path decision model, θ=(W R , b R , W B , b B ), W R is the weight coefficient of vehicle node passing time, b R is the bias coefficient of the vehicle node passing time, W B is the weight coefficient of the vehicle's remaining time budget, b B is the bias coefficient of the vehicle's remaining time budget, P[R(τ)≤T] is the probability that the time required for the vehicle's predicted path R(τ) is less than or equal to the target time T, and τ is the vehicle's predicted path.

[0094] A3. Based on the objective function of the vehicle path decision model and GPG, an initial policy gradient model is constructed. The Monte Carlo sampling method and the fixed numerical baseline method are used to update the initial policy gradient model to obtain the final policy gradient model. The policy gradient is calculated using the final policy gradient model.

[0095] The initial policy gradient model expression is:

[0096]

[0097] The present invention uses the Monte Carlo sampling method to update the initial policy gradient model. The expression is:

[0098]

[0099] The final expression of the policy gradient model is:

[0100]

[0101] in: is the policy gradient, M is the number of vehicle predicted paths generated by the vehicle path decision model through the Monte Carlo sampling method, j is the predicted path number generated by the vehicle path decision model through the Monte Carlo sampling method, For the condition R(τ (j) )<T indicator function, indicating that when the condition R(τ (j) )<T is true, the value is 1, when the condition R(τ (j) )<T is false, the value is 0, R(τ (j) ) is the vehicle predicted path τ (j) The required time, τ (j) is the jth vehicle predicted path generated by the Monte Carlo sampling method of the vehicle path decision model, T is the target time, b is the baseline value, and k is the vehicle predicted path τ (j) The node number in H j Predict the path τ for the vehicle(j) The number of nodes in is the logarithmic policy gradient, The current state of the vehicle is Execute action when The corresponding strategy, To select the execution action corresponding to the k+1th node at the kth node in the jth vehicle prediction path, is the state of the kth node in the predicted path of the jth vehicle.

[0102] The expression of the node status in the vehicle prediction path is:

[0103] s k =(p s，k ′,y k+1 , B k ′,S k ),

[0104] S k =(p k-1 ′, r k ′,S k-1 ,a k-1 )

[0105] Where: s k is the state of the kth node in the vehicle’s predicted path, p s，k ′ is the vehicle’s current node embedding data at the kth node in the vehicle’s predicted path, y k+1 The probability of selecting the k+1th node at the kth node for the vehicle prediction path, B k ′ is the vehicle remaining time budget embedding data at the kth node in the vehicle prediction path, S k is the vehicle history data at the kth node in the vehicle prediction path, p k-1 ′ is the k-1th node embedding data of the vehicle’s predicted path, r k ′ is the embedded data of the vehicle’s passing time at the kth node in the vehicle’s predicted path, S k-1 is the vehicle history data at the k-1th node in the vehicle prediction path, a k-1 Select the execution action corresponding to the kth node at the k-1th node in the vehicle's predicted path.

[0106] The expression for back propagation training of the vehicle path decision model using policy gradient in the present invention is:

[0107]

[0108] Where: θ′ is the weight parameter of the path decision model after back-propagation training, and α is the learning rate.

[0109] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0110] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0111] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0112] Specific embodiments are used in the present invention to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

[0113] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and it should be understood that the scope of protection of the present invention is not limited to such specific descriptions and embodiments. Those skilled in the art can make various other specific variations and combinations based on the technical teachings disclosed in the present invention without departing from the essence of the present invention, and such variations and combinations are still within the scope of protection of the present invention.

Claims

1. A routing decision method combining Decision Transformer and GPG to optimize on-time arrival probability is characterized by: The following steps are involved: S1. Obtaining the initial vehicle path dataset based on the road network; S2. Build a vehicle routing decision model based on Decision Transformer, use the vehicle initial routing dataset and the vehicle routing decision model to obtain the probability distribution of vehicle routing node selection, and generate the vehicle predicted route based on the probability distribution of vehicle routing node selection; S3. Based on the vehicle predicted path and GPG, the policy gradient is calculated with the on-time arrival probability as the optimization goal. The vehicle path decision model is trained through backpropagation using the policy gradient, and the vehicle path decision model after backpropagation training is used to make vehicle path decisions.

2. The path decision method for optimizing on-time arrival probability by combining Decision Transformer and GPG according to claim 1, characterized in that: In step S1 , the vehicle initial path data set includes vehicle path node data, vehicle node passing time data, vehicle remaining time budget data, and vehicle current node data.

3. The path decision method for optimizing on-time arrival probability by combining Decision Transformer and GPG according to claim 1, characterized in that: In step S2, the vehicle path decision model includes an attention embedding layer, a linear embedding layer, an encoder and a decoder; the input end of the attention embedding layer serves as the first data input end, the output end of the attention embedding layer is connected to the input end of the encoder, the output end of the encoder is connected to the first input end of the decoder, the input end of the linear embedding layer serves as the second data input end, the output end of the linear embedding layer is connected to the second input end of the decoder, and the output end of the decoder serves as the output of the vehicle path decision model; the data input to the first data input end is the vehicle path node data and the vehicle node passing time data; the data input to the second data input end is the vehicle remaining time budget data and the vehicle current node data.

4. The path decision method for optimizing on-time arrival probability by combining Decision Transformer and GPG according to claim 3, characterized in that: The expression for data processing in the attention embedding layer is: P′=EmbeddingP(P), P=[p1,p2,...,p L ], R′=W R R+b R , R＝[r1,r2,...,r L ]， X fusion =Attention(P′,R′,R′) Where: P' is the embedded data of the node passed by the vehicle path, EmbeddingP is the node lookup table passed by the vehicle path, P is the node data passed by the vehicle path, p1 is the first node passed by the vehicle path, p2 is the second node passed by the vehicle path, p L is the Lth node that the vehicle path passes through, L is the number of nodes that the vehicle path passes through, R′ is the embedded data of the vehicle node passing time, W R is the weight coefficient of the vehicle node passing time, R is the vehicle node passing time data, b R is the bias coefficient of the vehicle node passing time, r1 is the vehicle passing time at the first node, r2 is the vehicle passing time at the second node, and r L is the passing time of the vehicle at the Lth node, X fusion is the fusion feature of the vehicle, and Attention is the standard self-attention mechanism.

5. The path decision method for optimizing on-time arrival probability by combining Decision Transformer and GPG according to claim 3, characterized in that: The Encoder consists of 4 layers of TransformerBlock. The expression for data processing in the first layer of TransformerBlock of the Encoder is: X out1 =TransformerBlock(X in1 +R′), X in1 =Dropout(X fusion )+Embedding(P′) Where: X out1 is the output of the first layer TransformerBlock of Encoder, X in1 is the input of the first layer TransformerBlock of Encoder, R′ is the time embedding data of vehicle nodes, Dropout is the regularization process, X fusion is the fusion feature of the vehicle, Embedding is the position encoding process, and P′ is the embedded data of the nodes passed by the vehicle path; The output of the first layer TransformerBlock of Encoder is used as the input of the second layer TransformerBlock of Encoder, and the above processing is carried out in sequence. The output X of the fourth layer TransformerBlock of Encoder out4 As the output X of Encoder enc .

6. The path decision method for optimizing on-time arrival probability by combining Decision Transformer and GPG according to claim 3, characterized in that: The expression for data processing in the linear embedding layer is: p s ′=EmbeddingS(p s ), B′=W B B+b B Where: p s ′ is the embedding data of the vehicle’s current node, EmbeddingS is the vehicle’s current node lookup table, and p s is the vehicle’s current node data, B′ is the vehicle’s remaining time budget embedded data, and W B is the weight coefficient of the vehicle's remaining time budget, B is the vehicle's remaining time budget data, and b B Bias factor for the vehicle's remaining time budget.

7. The path decision method for optimizing on-time arrival probability by combining Decision Transformer and GPG according to claim 3, characterized in that: Decoder includes 4 layers of TransformerBlock and prediction module; The expression for data processing in the first layer TransformerBlock of Decoder is: X out1 ′=TransformerBlock(X in1 ′,X enc ,X enc ), X in1 ′=Dropout(B′+p s ′) Where: X out1 ′ is the output of the first layer TransformerBlock of Decoder, X enc is the output of Encoder, X in1 ′ is the input of the first layer TransformerBlock of Decoder, Dropout is the regularization process, p s ′ is the vehicle’s current node embedded data, and B′ is the vehicle’s remaining time budget embedded data; The output of the first layer TransformerBlock of the Decoder is used as the input of the second layer TransformerBlock of the Decoder. The above processing is carried out in sequence. The output X of the fourth layer TransformerBlock of the Decoder out4 As the input of the Decoder prediction module; The expression for data processing in the prediction module of Decoder is: y=softmax(fc(X out4 ′)) Where: y is the probability of vehicle path node selection, softmax is the softmax activation function, fc is the fully connected layer, X out4 ′ is the output of the fourth layer TransformerBlock of Decoder.

8. The path decision method for optimizing on-time arrival probability by combining Decision Transformer and GPG according to claim 1, characterized in that: In step S3, based on the vehicle's predicted path and GPG, the policy gradient is calculated with the on-time arrival probability as the optimization target, including the following steps: A1. Calculate the required time based on the vehicle's predicted path; A2. Based on the predicted route duration of the vehicle, the objective function of the vehicle routing decision model is constructed with the probability of on-time arrival as the optimization goal; A3. Based on the objective function of the vehicle path decision model and GPG, an initial policy gradient model is constructed. The Monte Carlo sampling method and the fixed numerical baseline method are used to update the initial policy gradient model to obtain the final policy gradient model. The policy gradient is calculated using the final policy gradient model.

9. The path decision method for optimizing on-time arrival probability by combining Decision Transformer and GPG according to claim 8, characterized in that: In step A3, the final policy gradient model is expressed as: in: is the policy gradient, M is the number of vehicle predicted paths generated by the vehicle path decision model through the Monte Carlo sampling method, j is the predicted path number generated by the vehicle path decision model through the Monte Carlo sampling method, For the condition R(τ (j) )<T indicator function, indicating that when the condition R(τ (j) )<T is true, the value is 1, when the condition R(τ (j) )≤T is false, the value is 0, R(τ (j) ) is the vehicle predicted path τ (j) The required time, τ (j) is the jth vehicle predicted path generated by the Monte Carlo sampling method of the vehicle path decision model, T is the target time, b is the baseline value, and k is the vehicle predicted path τ (j) The node number in H j Predict the path τ for the vehicle (j) The number of nodes in is the logarithmic policy gradient, The current state of the vehicle is Execute action when The corresponding strategy, To select the execution action corresponding to the k+1th node at the kth node in the jth vehicle prediction path, is the state of the kth node in the predicted path of the jth vehicle.

10. The path decision method for optimizing on-time arrival probability by combining Decision Transformer and GPG according to claim 9, characterized in that: The expression of the node status in the vehicle prediction path is: s k =(p s，k ′,y k+1 ,B k ′,S k ), S k =(p k-1 ′,r k ′,S k-1 ,a k-1 ) Where: s k is the state of the kth node in the vehicle’s predicted path, p s，k ′ is the vehicle’s current node embedding data at the kth node in the vehicle’s predicted path, y k+1 The probability of selecting the k+1th node at the kth node for the vehicle prediction path, B k ′ is the vehicle remaining time budget embedding data at the kth node in the vehicle prediction path, S k is the vehicle history data at the kth node in the vehicle prediction path, p k-1 ′ is the k-1th node embedding data of the vehicle’s predicted path, r k ′ is the embedded data of the vehicle’s passing time at the kth node in the vehicle’s predicted path, S k-1 is the vehicle history data at the k-1th node in the vehicle prediction path, a k-1 Select the execution action corresponding to the kth node at the k-1th node in the vehicle's predicted path.

Images