Strip mine truck trajectory prediction method based on graph prompt tuning space-time pre-training

Abstract

The invention discloses a strip mine truck trajectory prediction method based on graph prompt tuning space-time pre-training. The strip mine truck trajectory prediction method comprises the steps of obtaining a historical GPS trajectory data set and a road network data set of a strip mine truck; the preprocessed historical GPS track data set is input into a Transform model, and the historical GPS track data set is input into the Transform model; converting the road network data set and the encoded historical GPS trajectory data set into vectors through a graph structure conversion module, inputting the vectors as input items into a graph attention neural network, and performing space-time pre-training to obtain a strip mine truck trajectory prediction model; creating a prompt memory pool, and performing knowledge-oriented prompt optimization on the strip mine truck trajectory prediction model by using the prompt memory pool; and predicting the running track of the strip mine truck by using the optimized strip mine truck track prediction model. The method has sufficient generalization ability, the problems of zero samples and small samples can be solved, a surface mine dispatching system is optimized, the productivity of a surface mine is improved, and the operation cost is reduced.

Description

Technical Field

[0001] The invention relates to the field of open pit truck scheduling, and in particular to an open pit truck trajectory prediction method with graph prompting and spatiotemporal pre-training. Background Art

[0002] Open-pit mining trucks are responsible for the main material transportation tasks in the mine transportation process. The safety, efficiency and cost control of their operation directly affect the benefits and safety of the entire mining operation. The driving trajectory prediction of open-pit mining trucks is of great significance for optimizing transportation scheduling, reducing energy consumption, reducing safety accidents and improving overall production efficiency.

[0003] Traditional trajectory prediction methods mostly rely on simple pattern matching using historical data sets, which makes it difficult to effectively capture the spatiotemporal variation characteristics in dynamic environments, thus affecting prediction accuracy. With the development of deep learning and spatiotemporal data analysis methods in trajectory prediction, the automation and intelligence of mining has become an important trend in the development of the industry. In particular, spatiotemporal pre-training models driven by large-scale data sets have shown great potential. These models can capture the complex spatiotemporal dependencies in the driving trajectories of open-pit mine trucks by pre-training on large-scale spatiotemporal data sets. However, given the complex environment and data sparsity of open-pit mines, it is difficult to achieve the expected results by simply relying on pre-training models. In addition, the existing open-pit mine truck trajectory prediction methods have low accuracy, cannot adapt to few-sample and zero-sample predictions, and cannot capture spatiotemporal dependencies well for accurate predictions. Summary of the invention

[0004] Purpose of the invention: In view of the above problems, the purpose of the present invention is to provide an open-pit mine truck trajectory prediction method with graph-prompted optimized spatiotemporal pre-training.

[0005] Technical solution: A method for predicting open pit truck trajectories by using graph prompts to optimize spatiotemporal pre-training of open pit truck trajectories of the present invention comprises the following steps:

[0006] Step 1, obtaining the historical GPS trajectory dataset and road network dataset of the open pit mine truck, and preprocessing the historical GPS trajectory dataset;

[0007] Step 2: Input the preprocessed historical GPS trajectory data set into the Transformer model to obtain the positional relationship between trajectory points;

[0008] Step 3: The road network dataset and the encoded historical GPS trajectory dataset are converted into vectors through the graph structure conversion module, and the vectors are input into the graph attention neural network as input items, and spatiotemporal pre-training is performed to obtain the open-pit mine truck trajectory prediction model;

[0009] Step 4, creating a prompt memory pool, and using the prompt memory pool to perform knowledge-guided prompt tuning on the open-pit mine truck trajectory prediction model;

[0010] Step 5: Use the optimized open-pit mine truck trajectory prediction model to predict the driving trajectory of the open-pit mine truck.

[0011] Further, step 1 comprises:

[0012] The historical GPS trajectory dataset is represented as T = {T1, T2, ..., T i , ..., T N}, where N represents the trajectory number, represents the jth point of the trajectory with trajectory number i, n represents the trajectory point number, represents the time, longitude, latitude and vehicle status of the jth point of the track numbered i; the open-pit mine road network dataset is represented as S = {S1, S2, ..., S k , ..., S N},in represents each line node of the road network numbered k, Indicates the longitude, latitude and road status of the jth node of the kth road network;

[0013] The historical GPS trajectory dataset is cleaned, including deleting redundant data that deviates from the actual route and deleting invalid stationary points or drifting points; and the preprocessed historical GPS trajectory dataset is saved in Json format.

[0014] Further, step 2 includes:

[0015] The preprocessed historical GPS trajectory data set is input into the position coding layer for sine position coding and cosine position coding to obtain the data set L = {L1, L2, ..., L i , ..., L n}, L i represents the i-th trajectory data after encoding; the position encoding formulas are:

[0016]

[0017]

[0018] In the formula, pos represents the index position of the element in the queue, i represents the dimension index of the encoded encoding vector, and d represents the dimension of the input vector;

[0019] The data set L is transformed linearly to generate the query matrix Q and key matrix K. The calculation formulas are:

[0020] Q=LW Q

[0021] K=LW K

[0022] Where W Q and W K is a trainable weight matrix;

[0023] According to the query matrix Q and the key matrix K, the dot product of each track point in the single-challenge track is calculated to obtain the attention score matrix A. The calculation formula is:

[0024]

[0025] In the formula, is the scaling factor, T represents the transpose;

[0026] Use the Softmax function to normalize the attention score matrix A and convert it into the attention weight matrix W. The calculation formula is:

[0027] W = Softmax(A).

[0028] Further, step 3 includes:

[0029] Construct a graph attention neural network model G = (L, S, W) and calculate the attention coefficient e of each trajectory point in the graph attention neural network model ij , the calculation expression is:

[0030] e ij =LeakyReLU(α T [Wh i ||Wh j ])

[0031] In the formula, LeakyReLU is the activation function, α T is a trainable attention vector, h i and h i are the feature vectors of trajectory point i and trajectory point j respectively, and || represents the vector concatenation operation.

[0032] Further, step 4 includes:

[0033] Taking the expertise of open pit truck trajectory selection as input, a memory pool M of open pit truck trajectory route selection knowledge with key-value pair structure is constructed. i , ..., M n}, where M i ={F, V}, where F is the vehicle state and road state conditions that affect the driving trajectory of open pit trucks, and V is the selected path information;

[0034] The current vehicle state and road state conditions of the trajectory point are taken as input x, and the Euclidean distance formula is used to calculate the similarity between the vehicle state and road state conditions x of the current input open-pit mine truck driving trajectory and the key F in the memory pool to obtain the most relevant key-value pair output M i , the calculation expression is:

[0035]

[0036] In the formula, Dist(x, F) represents the similarity, x i represents the conditions of the current vehicle state and road state including the trajectory point, F i is the condition key in the memory pool;

[0037] Output the most relevant key-value pair M i As the prompt vector P, the prompt vector P is combined with the attention coefficient e of the trajectory node ij Deep fusion, dynamically adjusts the attention parameters of the graph attention neural network model, expressed as:

[0038] e ij ′=λ·e ij +(1-λ)·P

[0039] Where λ is the adaptive coefficient;

[0040] The attention coefficients of all adjacent nodes are normalized and the calculation formula is:

[0041]

[0042] In the formula, α ij is the attention coefficient of trajectory point i to trajectory point j, indicating the influence weight of trajectory point j on trajectory point i, ∑ k∈N(i) exp(e ik ′) is the normalization factor of the attention scores corresponding to all neighboring trajectory points k of trajectory point i;

[0043] Using all the attention coefficients α ij The attention weight matrix B is combined into an attention weight matrix, which is used to adjust the attention scores between trajectory points and road network nodes to obtain the final spatiotemporal prediction model of open-pit mine truck trajectories.

[0044] Further, step 5 includes:

[0045] The real-time GPS trajectory dataset and the road network dataset are input into the final spatiotemporal prediction model of open-pit mine truck trajectory to obtain the predicted driving trajectory of the open-pit mine truck.

[0046] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0047] 1. The present invention uses sine position coding, cosine position coding and Transfomer's self-attention module to ensure the correctness of the relative relationship between the positions of each trajectory point in a single trajectory, so that the final open-pit mine truck trajectory prediction model can not only consider the influence of adjacent trajectory nodes on the path selection of the current trajectory point, but also consider the influence of distant nodes on the path selection of the current trajectory point, thereby improving the continuity and accuracy of trajectory prediction;

[0048] 2. The graph structure conversion module in the present invention can dynamically select the most relevant neighbor nodes for information transmission according to the current state of the truck, thereby improving the prediction accuracy and response speed of the model;

[0049] 3. The present invention uses prompt tuning, allowing the model to adjust its prediction strategy through a small number of task-related prompts, thereby reducing its dependence on a large amount of labeled data; combined with pre-training, the open-pit mine truck trajectory prediction model can maintain good prediction performance when facing zero-sample and few-sample predictions, reducing the risk of model overfitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A flowchart of an open pit truck trajectory prediction method with graph-cued tuning of spatiotemporal pre-training;

[0051] Figure 2 A flowchart of a method for predicting open pit truck trajectories by using graph-cued tuning of spatiotemporal pre-training;

[0052] Figure 3 Process diagram for knowledge-based cue vector generation. DETAILED DESCRIPTION

[0053] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0054] The present embodiment describes a method for predicting open pit truck trajectories by optimizing spatiotemporal pre-training with a graph prompt, and the flow chart is as follows: Figure 1 to Figure 2 As shown, the method at least includes the following steps 1 to 5.

[0055] Step 1: Obtain the historical GPS trajectory dataset and road network dataset of open-pit mine trucks, and preprocess the historical GPS trajectory dataset.

[0056] Specifically, step 1 includes:

[0057] The historical GPS trajectory dataset is represented as T = {T1, T2, ..., T i , ..., T N}, where N represents the trajectory number, represents the jth point of the trajectory with trajectory number i, n represents the trajectory point number, represents the time, longitude, latitude and vehicle status of the jth point of the track numbered i; the open-pit mine road network dataset is represented as S = {S1, S2, ..., S k , ..., S N},in represents each line node of the road network numbered k, Indicates the longitude, latitude and road status of the jth node of the kth road network;

[0058] The historical GPS trajectory data set is cleaned, including deleting redundant data that deviates from the actual route, deleting invalid static points or drifting points; and the preprocessed historical GPS trajectory data set is saved in Json format. Redundant data refers to the trajectory route that deviates from the road network, invalid data refers to repeated static trajectory points, and drift point data refers to the trajectory points that are recorded and do not match the actual driving trajectory points.

[0059] Step 2: Input the preprocessed historical GPS trajectory dataset into the Transformer model to obtain the positional relationship between trajectory points.

[0060] The Transformer model consists of a position encoding layer, an attention score layer, and a softmax layer.

[0061] Specifically, step 2 includes:

[0062] The preprocessed historical GPS trajectory data set is input into the position coding layer for sine position coding and cosine position coding to obtain the data set L = {L1, L2, ..., L i , ..., L n}, L i represents the encoded i-th trajectory data, and the dataset L is used as the encoded training dataset of the position dependency relationship of each trajectory point in a single trajectory; the position encoding formulas are:

[0063]

[0064]

[0065] In the formula, pos represents the index position of the element in the queue, i represents the dimension index of the encoded encoding vector, and d represents the dimension of the input vector;

[0066] The data set L is transformed linearly to generate the query matrix Q and key matrix K. The calculation formulas are:

[0067] Q=LW Q

[0068] K=LW K

[0069] Where W Q and W K is a trainable weight matrix, which is used to generate the query matrix Q and the key matrix K, with dimensions of d×d respectively. k , d is the feature dimension of each trajectory point, d k is the feature dimension after dimensionality reduction, which is 1.

[0070] According to the query matrix Q and the key matrix K, the dot product of each track point in the single-challenge track is calculated to obtain the attention score matrix A. The calculation formula is:

[0071]

[0072] In the formula, is a scaling factor used to prevent the dot product result from being too large, causing the gradient to disappear or explode; T represents transpose;

[0073] Use the Softmax function to normalize the attention score matrix A and convert it into the attention weight matrix W. The calculation formula is:

[0074] W = Softmax(A).

[0075] Step 3: The road network dataset and the encoded historical GPS trajectory dataset are converted into vectors through the graph structure conversion module, and the vectors are input into the graph attention neural network as input items, and spatiotemporal pre-training is performed to obtain the open-pit mine truck trajectory prediction model.

[0076] Specifically, step 3 includes:

[0077] Construct a graph attention neural network model G = (L, S, W), where L is the node set of the graph, representing the GPS trajectory points of the open-pit mine truck, S is the edge set of the graph composed of the road network dataset, representing the road network structure, and calculate the attention coefficient e of each trajectory point in the graph attention neural network model ij , the calculation expression is:

[0078] e ij =LeakyReLU(α T [Wh i ||Wh j ])

[0079] Where LeakyReLU is the activation function. Using the LeakyReLU activation function can prevent the model from falling into linear mapping and enhance the expressiveness of the model; α T is a trainable attention vector used to calculate the attention score, h i and h jare the feature vectors of trajectory point i and trajectory point j respectively, and || represents the vector concatenation operation.

[0080] Step 4: Create a prompt memory pool and use the prompt memory pool to perform knowledge-guided prompt tuning on the open-pit mine truck trajectory prediction model.

[0081] Specifically, Figure 3 In the diagram of the knowledge-oriented hint vector generation process shown in FIG. 1 , step 4 includes:

[0082] Taking the expertise of open pit truck trajectory selection as input, a memory pool M of open pit truck trajectory route selection knowledge with key-value pair structure is constructed. i , ..., M n}, where M i ={F, V}, where F is the vehicle state and road state conditions that affect the driving trajectory of open pit trucks, and V is the selected path information;

[0083] The current vehicle state and road state conditions of the trajectory point are taken as input x, and the Euclidean distance formula is used to calculate the similarity between the vehicle state and road state conditions x of the current input open-pit mine truck driving trajectory and the key F in the memory pool to obtain the most relevant key-value pair output M i , the calculation expression is:

[0084]

[0085] In the formula, Dist(x, F) represents the similarity, the smaller the value, the higher the similarity; i represents the conditions of the current vehicle state and road state including the trajectory point, F i is the condition key in the memory pool;

[0086] Output the most relevant key-value pair M i As the prompt vector P, the prompt vector P is combined with the attention coefficient e of the trajectory node ij Deep fusion dynamically adjusts the attention parameters of the graph attention neural network model to improve the versatility and accuracy of the model, solves the problem that the existing model performs poorly in small sample and zero sample prediction tasks, and obtains a spatiotemporal prediction model for open-pit mine truck trajectories with higher prediction accuracy and greater versatility; the expression of the attention coefficient after deep fusion is:

[0087] e ij ′=λ·e ij +(1-λ)·P

[0088] Where λ is the adaptive coefficient, which is used to control the influence of the prompt vector;

[0089] The attention coefficients of all adjacent nodes are normalized and the calculation formula is:

[0090]

[0091] In the formula, α ij is the attention coefficient of trajectory point i to trajectory point j, indicating the influence weight of trajectory point j on trajectory point i, ∑ k∈N(i) exp(e ik ′) is the normalization factor of the attention scores corresponding to all neighboring trajectory points k of trajectory point i; thus ensuring that each trajectory point p i The sum of the attention coefficients of its adjacent nodes is 1, so it can be used as the weight for weighted summation.

[0092] Using all the attention coefficients α ij The attention weight matrix B is combined into an attention weight matrix, which is used to adjust the attention scores between trajectory points and road network nodes, thereby affecting the calculation of the attention coefficient, so that the model can capture the spatiotemporal dependencies between trajectory points and road network nodes, and obtain the final spatiotemporal prediction model for open-pit mine truck trajectories.

[0093] By combining domain expertise and generating knowledge-guided prompts for open-pit mine truck trajectory selection, the open-pit mine truck trajectory prediction model can be effectively tuned. At the same time, the model's ability to perceive the probability and location of mine truck failures can also be enhanced, thereby achieving intelligent collaboration between mine truck transportation and maintenance vehicles and improving prediction accuracy and overall efficiency.

[0094] Step 5: Use the optimized open-pit mine truck trajectory prediction model to predict the driving trajectory of the open-pit mine truck.

[0095] Step 5 includes:

[0096] The real-time GPS trajectory dataset and the road network dataset are input into the final spatiotemporal prediction model of open-pit mine truck trajectory to obtain the predicted driving trajectory of the open-pit mine truck.

Claims

1. A method for predicting open pit truck trajectories by using graph-prompted optimization and spatiotemporal pre-training, characterized in that: The steps include: Step 1, obtaining the historical GPS trajectory dataset and road network dataset of the open pit mine truck, and preprocessing the historical GPS trajectory dataset; Step 2: Input the preprocessed historical GPS trajectory data set into the Transformer model to obtain the positional relationship between trajectory points; Step 3: The road network dataset and the encoded historical GPS trajectory dataset are converted into vectors through the graph structure conversion module, and the vectors are input into the graph attention neural network as input items, and spatiotemporal pre-training is performed to obtain the open-pit mine truck trajectory prediction model; Step 4, creating a prompt memory pool, and using the prompt memory pool to perform knowledge-guided prompt tuning on the open-pit mine truck trajectory prediction model; Step 5: Use the optimized open-pit mine truck trajectory prediction model to predict the driving trajectory of the open-pit mine truck.

2. According to claim 1, a method for predicting open-pit mine truck trajectories by using image prompts to optimize spatiotemporal pre-training is characterized in that: Step 1 includes: The historical GPS trajectory dataset is represented as T = {T1, T2, ..., T i ,…,T N }, where N represents the trajectory number, represents the jth point of the trajectory numbered i, n represents the number of the last trajectory point, represents the time, longitude, latitude and vehicle status of the jth point of the track numbered i; the open-pit mine road network dataset is represented as S = {S1, S2, ..., S k ,…,S N },in represents each line node of the road network numbered k, Indicates the longitude, latitude and road status of the jth node of the kth road network; The historical GPS trajectory dataset is cleaned, including deleting redundant data that deviates from the actual route and deleting invalid stationary points or drifting points; and the preprocessed historical GPS trajectory dataset is saved in Json format.

3. The open-pit mine truck trajectory prediction method of image prompt tuning spatiotemporal pre-training according to claim 2 is characterized in that: Step 2 includes: The preprocessed historical GPS trajectory data set is input into the position coding layer for sine position coding and cosine position coding to obtain the data set L = {L1, L2, …, L i ,…,L n }, L i represents the i-th trajectory data after encoding; the position encoding formulas are: In the formula, pos represents the index position of the element in the queue, i represents the dimension index of the encoded encoding vector, and d represents the dimension of the input vector; The data set L is transformed linearly to generate the query matrix Q and key matrix K. The calculation formulas are: Q=LW Q K=LW K Where W Q and W K is a trainable weight matrix; According to the query matrix Q and the key matrix K, the dot product of each track point in the single-challenge track is calculated to obtain the attention score matrix A. The calculation formula is: In the formula, is the scaling factor, T represents the transpose; Use the Softmax function to normalize the attention score matrix A and convert it into the attention weight matrix W. The calculation formula is: W = Softmax(A).

4. The method for predicting open pit truck trajectories by using image prompts to optimize spatiotemporal pre-training according to claim 3 is characterized in that: Step 3 includes: Construct a graph attention neural network model G = (L, S, W) and calculate the attention coefficient e of each trajectory point in the graph attention neural network model ij , the calculation expression is: and ij =LeakyReLU(α T [Wh i ∥Wh j ]) In the formula, LeakyReLU is the activation function, α T is a trainable attention vector, h i and h j are the feature vectors of trajectory point i and trajectory point j respectively, and ∥ represents the vector concatenation operation.

5. The method for predicting open pit truck trajectories by using image prompts to optimize spatiotemporal pre-training according to claim 4, characterized in that: Step 4 includes: Taking the expertise of open-pit mine truck trajectory selection as input, a memory pool of open-pit mine truck trajectory route selection knowledge with a key-value pair structure is constructed M = {M1, M2, …, M i ,…,M n }, where M i ={F,V}, F is the vehicle state and road state conditions that affect the driving trajectory of open pit trucks, and V is the selected path information; The current vehicle state and road state conditions of the trajectory point are taken as input x, and the Euclidean distance formula is used to calculate the similarity between the vehicle state and road state conditions x of the current input open-pit mine truck driving trajectory and the key F in the memory pool to obtain the most relevant key-value pair output M i , the calculation expression is: In the formula, Dist(x,F) represents the similarity, x i represents the conditions of the current vehicle state and road state including the trajectory point, F i is the condition key in the memory pool; Output the most relevant key-value pair M i As the prompt vector P, the prompt vector P is combined with the attention coefficient e of the trajectory node ij Deep fusion, dynamically adjusts the attention parameters of the graph attention neural network model, expressed as: e ij ′=λ·e ij +(1-λ)·P Where λ is the adaptive coefficient; The attention coefficients of all adjacent nodes are normalized and the calculation formula is: In the formula, α ij is the attention coefficient of trajectory point i to trajectory point j, indicating the influence weight of trajectory point j on trajectory point i, ∑ k∈N(i) exp(e ik ′) is the normalization factor of the attention scores corresponding to all neighboring trajectory points k of trajectory point i; Using all the attention coefficients α ij The attention weight matrix B is combined into an attention weight matrix, which is used to adjust the attention scores between trajectory points and road network nodes to obtain the final spatiotemporal prediction model of open-pit mine truck trajectories.

6. The method for predicting open pit truck trajectories by using image prompts to optimize spatiotemporal pre-training according to claim 5, characterized in that: Step 5 includes: The real-time GPS trajectory dataset and the road network dataset are input into the final spatiotemporal prediction model of open-pit mine truck trajectory to obtain the predicted driving trajectory of the open-pit mine truck.

Images