Training method and device of pre-training model based on satellite positioning trajectory data

Abstract

The invention provides a training method and device for a pre-training model based on satellite positioning trajectory data, and the method comprises the steps: obtaining the satellite positioning trajectory data of a plurality of vehicles in a target region; processing the satellite positioning trajectory data to obtain dense vectors and trajectory labels of the satellite positioning trajectory data; obtaining an input vector group according to the dense vector and the trajectory label; and inputting the input vector group into a pre-training model for training. According to the embodiment, large-scale and various types of vehicle satellite positioning trajectory data are utilized and processed to obtain dense vectors and trajectory labels, so that an input vector group of a pre-training model is obtained, the input vector group is input into the constructed pre-training model for training, and a final pre-training model is obtained. The problem that the algorithm application scene is limited or the algorithm accuracy still needs to be improved due to the fact that the processes of training, verification, testing and the like of an algorithm model are only limited to satellite positioning data of a small amount or a single type of vehicles is solved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation, and in particular to a training method and device for a pre-training model based on satellite positioning trajectory data. Background Art

[0002] In recent years, satellite positioning systems, especially vehicle-mounted satellite positioning equipment, have been widely used and developed rapidly, and the amount of vehicle-mounted satellite positioning data has shown explosive growth.

[0003] In the field of intelligent transportation, algorithm models and applications based on vehicle-mounted satellite positioning data are emerging in an endless stream, such as map matching algorithms, vehicle trajectory prediction algorithms, traffic status estimation algorithms based on satellite positioning data, etc.; however, the training, verification, and testing processes of most of these algorithm models are limited to the satellite positioning data of a small number or a single type of vehicle, resulting in limitations in the application scenarios of the algorithms or the accuracy of the algorithms still needs to be improved. Summary of the invention

[0004] The purpose of the present invention is to provide a training method and device for a pre-trained model based on satellite positioning trajectory data, which is used to solve the problem that the training, verification, testing and other processes of the algorithm model in the prior art are limited to the satellite positioning data of a small number or a single type of vehicles, resulting in the limitation of the application scenarios of the algorithm or the accuracy of the algorithm still needs to be improved.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a training method of a pre-trained model based on satellite positioning trajectory data, wherein the method comprises:

[0006] Acquire satellite positioning trajectory data of multiple vehicles in a target area; wherein the satellite positioning trajectory data includes status data of the vehicle at multiple consecutive position points, and the status data includes at least latitude and longitude coordinates;

[0007] Processing the satellite positioning trajectory data to obtain a dense vector and a trajectory label of the satellite positioning trajectory data;

[0008] Obtain an input vector group according to the dense vector and the trajectory label;

[0009] The input vector group is input into a pre-trained model for training.

[0010] Optionally, the training method, wherein the processing of the satellite positioning trajectory data to obtain a dense vector and a trajectory label of the satellite positioning trajectory data, includes:

[0011] Dividing the target area into a plurality of non-overlapping grids;

[0012] For each position point in the satellite positioning trajectory data, calculate the distance between the position point and each grid center point according to the longitude and latitude coordinates of the position point, and generate a vectorized representation of the longitude and latitude coordinates according to the distance between the position point and each grid center point; replace the longitude and latitude coordinates in the status data of each position point in the satellite positioning trajectory data with the vectorized representation of the longitude and latitude coordinates to obtain the dense vector of the satellite positioning trajectory data;

[0013] Based on a map matching algorithm, the road segment code in the road network matched by each position point in the satellite positioning trajectory data is determined, and the road segment code is converted into a one-hot encoded vector as the label value corresponding to the position point to obtain the trajectory label.

[0014] Optionally, in the training method, the step of obtaining an input vector group according to the dense vector and the trajectory label comprises:

[0015] Constructing an initial vector group, the initial vector group comprising a trajectory vector group composed of the dense vectors of the satellite positioning trajectory data, and a starting vector and a paragraph vector having the same length as the dense vector, wherein the starting vector is set before the trajectory vector group, and the paragraph vector is set at the middle position and the end position of the trajectory vector;

[0016] Dividing the trajectory vector group into a first vector group and a second vector group according to the paragraph vector, embedding the first paragraph vector in the first vector group, and embedding the last paragraph vector in the second vector group;

[0017] Based on the position of each vector in the trajectory vector group, a position embedding vector is constructed to obtain an input vector.

[0018] Optionally, in the training method, after obtaining the input vector group, the method further comprises:

[0019] A label value of the satellite positioning trajectory data is randomly selected from the input vector group for prediction.

[0020] Optionally, the training method, wherein the step of inputting the input vector group into a pre-trained model for training, comprises:

[0021] The input vector groups are concatenated into matrices, and the trainable parameter matrices are multiplied respectively to obtain a query matrix, a key matrix, and a value matrix; an output matrix of the attention layer is calculated according to the query matrix, the key matrix, and the value matrix; a plurality of the output matrices are concatenated, and converted by matrix multiplication to obtain a multi-head attention layer;

[0022] Calculating residual links according to the input vector group and the previous layer function of the residual link, and normalizing each vector in the input vector group to obtain a normalized result, thereby obtaining a residual link and a normalized layer;

[0023] Calculating an activation function based on the input vector group and the trainable model parameters for linking a feed-forward layer consisting of two fully connected neural network layers;

[0024] The input vector group is input into the pre-training model composed of a plurality of the multi-head attention layers, the residual links and the normalization layer and the feed-forward layer connected in series for training.

[0025] In order to achieve the above object, the present invention also provides a training device for a pre-trained model based on satellite positioning trajectory data, wherein the device comprises:

[0026] A first acquisition module is used to acquire satellite positioning trajectory data of multiple vehicles in a target area; wherein the satellite positioning trajectory data includes status data of the vehicle at multiple consecutive position points, and the status data at least includes latitude and longitude coordinates;

[0027] A first processing module is used to process the satellite positioning trajectory data to obtain a dense vector and a trajectory label of the satellite positioning trajectory data;

[0028] A second acquisition module, used for acquiring an input vector group according to the dense vector and the trajectory label;

[0029] The second processing module is used to input the input vector group into a pre-training model for training.

[0030] The beneficial effects of the above technical solution of the present invention are as follows:

[0031] In the above scheme, the satellite positioning trajectory data of large-scale and multi-type vehicles (such as taxis, trucks, private cars and online ride-hailing cars, etc.) in the target area are used to obtain the corresponding dense vectors and trajectory labels after processing, thereby obtaining the input vector group of the pre-trained model, and the input vector group is input into the constructed pre-trained model for training to obtain the final pre-trained model. This solves the problem that the training, verification, and testing processes of the algorithm model are limited to the satellite positioning data of a small number or a single type of vehicle, resulting in the limitation of the application scenarios of the algorithm or the accuracy of the algorithm still needs to be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A schematic diagram of a training method for a pre-trained model based on satellite positioning trajectory data according to an embodiment of the present invention;

[0033] Figure 2A flowchart of a training method for a pre-trained model based on satellite positioning trajectory data according to an embodiment of the present invention;

[0034] Figure 3 A schematic diagram of an input vector group of a training method for a pre-training model based on satellite positioning trajectory data according to an embodiment of the present invention;

[0035] Figure 4 A schematic diagram of a pre-training model based on satellite positioning trajectory data according to an embodiment of the present invention;

[0036] Figure 5 A schematic diagram of a training device for a pre-trained model based on satellite positioning trajectory data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] In view of the problem that the training, verification, and testing processes of the algorithm model in the prior art are limited to the satellite positioning data of a small number or a single type of vehicles, resulting in limitations in the application scenarios of the algorithm or the accuracy of the algorithm still needs to be improved, the present invention provides a training method and device for a pre-trained model based on satellite positioning trajectory data.

[0039] like Figure 1 As shown, an embodiment of the present invention provides a training method for a pre-trained model based on satellite positioning trajectory data, wherein the method comprises:

[0040] Step S10, obtaining satellite positioning trajectory data of multiple vehicles in the target area; wherein the satellite positioning trajectory data includes status data of the vehicle at multiple consecutive position points, and the status data includes at least latitude and longitude coordinates;

[0041] Step S20, processing the satellite positioning trajectory data to obtain a dense vector and a trajectory label of the satellite positioning trajectory data;

[0042] Step S30, obtaining an input vector group according to the dense vector and the trajectory label;

[0043] Step S40: input the input vector group into a pre-trained model for training.

[0044] In this embodiment, if Figure 2As shown, the embodiment of the present invention utilizes the satellite positioning trajectory data of large-scale and multi-type vehicles (such as taxis, trucks, private cars and online ride-hailing vehicles, etc.) in the target area, and obtains the corresponding dense vector and the trajectory label after processing, thereby obtaining the input vector group of the pre-trained model, and inputs the input vector group into the constructed pre-trained model for training to obtain the final pre-trained model. The problem that the training, verification, and testing processes of the algorithm model are limited to the satellite positioning data of a small number or a single type of vehicle is solved, resulting in the limitation of the application scenarios of the algorithm or the accuracy of the algorithm still needs to be improved.

[0045] Optionally, in the training method, step S20 comprises:

[0046] A10, dividing the target area into a plurality of non-overlapping grids;

[0047] It should be noted that the satellite positioning data set needs to be preprocessed first to facilitate the input of the pre-trained model. Analogous to the Word2Vec method in the field of natural language processing (based on a large amount of text corpus, learning word features in an unsupervised manner, generating word vectors, quantifying the relationship between words, and facilitating subsequent language model training), the satellite positioning data needs to be vectorized, and the longitude and latitude fields contained in the satellite positioning data itself need to be expanded and converted into a machine-recognizable dense vector containing multiple spatial information. The specific method is as follows:

[0048] The target area Z is divided into m×m grid areas (i.e., multiple non-overlapping grids) based on the spatial distance, i.e., Z={g i,j |i,j=1,2,…,m},g i,j Represents a single grid with coordinates i, j; and calculates the coordinates c of the center point of each grid i,j =(x i,j ,y i,j ), where x i,j ,y i,j Represents the longitude and latitude values ​​respectively.

[0049] A20, for each position point in the satellite positioning trajectory data, calculate the distance between the position point and each grid center point according to the longitude and latitude coordinates of the position point, and generate a vectorized representation of the longitude and latitude coordinates according to the distance between the position point and each grid center point; replace the longitude and latitude coordinates in the status data of each position point in the satellite positioning trajectory data with the vectorized representation of the longitude and latitude coordinates to obtain the dense vector of the satellite positioning trajectory data;

[0050] It should be noted that the initial vector of the position point l of any satellite positioning trajectory data in the target area Z is It is composed of longitude, latitude, time, speed, direction, and elevation. The longitude and latitude coordinates (x, y) of the location point are calculated as follows:

[0051]

[0052] in, Indicates that the satellite positioning trajectory position point l reaches the grid center point c i,j The Euclidean distance, σ represents a constant coefficient, which is used to reduce the distance value for subsequent calculations. Replace the longitude and latitude coordinates (x, y) of the location point in the satellite positioning trajectory data with the dense vector E l , obtain the dense vector of the satellite positioning trajectory data

[0053] A30, based on a map matching algorithm, determining a road segment code in a road network matched by each position point in the satellite positioning trajectory data, converting the road segment code into a one-hot encoded vector as a label value corresponding to the position point, and obtaining the trajectory label;

[0054] It should be noted that, based on the existing map matching algorithm, the satellite positioning trajectory data is matched to the road network of the target area Z, that is, the initial vector of any satellite positioning trajectory point l is obtained as Considering that the link code of the road segment itself has no practical meaning, a one-hot encoding is used to represent the label value. The corresponding label value is

[0055]

[0056] Optionally, in the training method, step S30 includes:

[0057] A40, constructing an initial vector group, wherein the initial vector group includes a trajectory vector group composed of the dense vectors of the satellite positioning trajectory data, and a start vector and a paragraph vector having the same length as the dense vector, wherein the start vector is set before the trajectory vector group, and the paragraph vector is set at the middle position and the end position of the trajectory vector;

[0058] It should be noted that the embodiment of the present invention needs to construct the vector group required for the pre-trained model input to facilitate model training. The input mainly considers two tasks: trajectory prediction and map matching. First, the model input construction method based on the trajectory prediction task is as follows:

[0059] like Figure 3 As shown, the input of the pre-trained model each time is a set of vectors, including the trajectory vector set The starting vector TS and the segment vector SEP; TS and SEP are the same length as the dense trajectory vector, i.e., the length is d model TS is placed before the trajectory vector group, and SEP is inserted into the middle and end positions of the trajectory vector group.

[0060] A50, dividing the trajectory vector group into a first vector group and a second vector group according to the paragraph vector, embedding the first paragraph vector in the first vector group, and embedding the last paragraph vector in the second vector group;

[0061] It should be noted that the trajectory vector group is divided into two parts using the paragraph vector, and the first part is embedded in the first paragraph vector E A , the second part embeds the final segment vector E B .

[0062] A60, constructing a position embedding vector based on the position of each vector in the trajectory vector group, and obtaining an input vector;

[0063] It should be noted that, based on the position of each vector in the trajectory vector group, a position embedding vector E is constructed. POS , where pos represents the position of the vector in the vector group, E POS The specific formula is as follows:

[0064]

[0065]

[0066] Among them, i represents the dimension position of the specific value in each vector, and d model Indicates the length of a single vector. In the actual pre-training process, the input vector group after construction and embedding is not used for model training, but 50% of the input vector group is randomly selected and kept unchanged, the first half of the trajectory vectors of the remaining input vector group remain unchanged, and the second half of the trajectory vectors are randomly selected from the second half of the trajectory vectors under non-true identical trajectories; if the trajectory vector group is the true identical trajectory, then Label trace is 1; if the trajectory vector group is patchwork, then Label trace is 0. Thus, the input vector group of the trajectory prediction task is obtained.

[0067] Optionally, in the training method, after step S30, the method further comprises:

[0068] B10, randomly selecting a label value of the satellite positioning trajectory data from the input vector group for prediction;

[0069] It should be noted that, in the input vector group, the start vector TS and the paragraph vector SEP are removed, and the remaining trajectory vectors are processed by the map matching algorithm in the data preprocessing and have the label value Lable l In order to train the map matching ability of the model, a label value of a trajectory vector is randomly selected in each input vector group for prediction, so that the model has the ability to determine the road network location of a trajectory vector based on the trajectory.

[0070] Optionally, in the training method, step S40 includes:

[0071] A70, concatenate the input vector groups into matrices, and multiply them by trainable parameter matrices respectively to obtain a query matrix, a key matrix, and a value matrix; calculate an output matrix of an attention layer according to the query matrix, the key matrix, and the value matrix; concatenate multiple output matrices, and convert them using matrix multiplication to obtain a multi-head attention layer;

[0072] It should be noted that if Figure 4 As shown, the input of the attention layer is of dimension d k The query vector and key vector of v A vector of values ​​of .

[0073] In order to improve the calculation speed, the input vector group is concatenated into a matrix and multiplied by the trainable parameter matrix to obtain the query matrix Q, the key matrix K and the value matrix V.

[0074] The formula for calculating the output matrix of the attention layer is as follows:

[0075]

[0076] The attention layer can mine the spatial information between the location points of the satellite positioning data, but the above attention layer can only recognize one spatial pattern, which does not conform to the concept of the model, so the head attention layer is used. The multi-head attention layer is the result of multiple attention layers spliced ​​together, and then converted by matrix multiplication to ensure the output matrix format. The calculation formula of the multi-head attention layer is as follows:

[0077] MultiHead(Q,K,V)=Concat(head1,…,head h )W O

[0078]

[0079] in, h represents the number of heads, d k =d v =d model / h.

[0080] A80, calculating the residual link according to the input vector group and the previous layer function of the residual link, and normalizing each vector in the input vector group to obtain a normalized result, thereby obtaining a residual link and a normalized layer;

[0081] It should be noted that in order to ensure the transmission of information, a residual link is used. The residual link formula is as follows:

[0082] Output=x+F(x)

[0083] Where x is the input vector group, F is the previous layer function of the residual link

[0084] In order to solve the gradient problem of back propagation and accelerate the convergence of the model, it is necessary to normalize each vector in the input vector group.

[0085] The residual link and normalization operations are combined to form a residual link and normalization layer.

[0086] A90, calculating an activation function based on the input vector group and the trainable model parameters, for linking a feedforward layer consisting of two fully connected neural network layers;

[0087] It should be noted that the feedforward layer in the embodiment of the present invention has two fully connected neural network layers, and the activation function is Relu. The specific formula is as follows:

[0088] FFN(n)=max(0,xW1+b1)W2+b2

[0089] Where x is the input vector group, W1, W2, b1, b2 are trainable model parameters, and the dimensions of the parameters are all d model .

[0090] A100, inputting the input vector group into the pre-training model composed of a plurality of the multi-head attention layers, the residual links and the normalization layer and the feed-forward layer connected in series for training;

[0091] It should be noted that Figure 4 The structure in is a single pre-training layer. In fact, the pre-training model is composed of N single pre-training layers in series.

[0092] like Figure 5 As shown, in order to achieve the above-mentioned purpose, the present invention also provides a training device for a pre-trained model based on satellite positioning trajectory data, wherein the device comprises:

[0093] A first acquisition module 51 is used to acquire satellite positioning trajectory data of multiple vehicles in a target area; wherein the satellite positioning trajectory data includes status data of the vehicle at multiple consecutive position points, and the status data includes at least latitude and longitude coordinates;

[0094] A first processing module 52 is used to process the satellite positioning trajectory data to obtain a dense vector and a trajectory label of the satellite positioning trajectory data;

[0095] A second acquisition module 53, used to acquire an input vector group according to the dense vector and the trajectory label;

[0096] The second processing module 54 is used to input the input vector group into the pre-training model for training.

[0097] Optionally, in the training device, the first processing module 52 includes:

[0098] A first processing unit, configured to divide the target area into a plurality of non-overlapping grids;

[0099] A second processing unit is used to calculate, for each position point in the satellite positioning trajectory data, the distance between the position point and each grid center point according to the longitude and latitude coordinates of the position point, and generate a vectorized representation of the longitude and latitude coordinates according to the distance between the position point and each grid center point; replace the longitude and latitude coordinates in the status data of each position point in the satellite positioning trajectory data with the vectorized representation of the longitude and latitude coordinates to obtain the dense vector of the satellite positioning trajectory data;

[0100] The first acquisition unit is used to determine the section code in the road network matched by each position point in the satellite positioning trajectory data based on a map matching algorithm, convert the section code into a one-hot encoded vector as a label value corresponding to the position point, and obtain the trajectory label.

[0101] Optionally, in the training device, the second acquisition module 53 includes:

[0102] A first construction unit is used to construct an initial vector group, wherein the initial vector group includes a trajectory vector group composed of the dense vectors of the satellite positioning trajectory data, and a start vector and a paragraph vector having the same length as the dense vector, wherein the start vector is set before the trajectory vector group, and the paragraph vector is set at the middle position and the end position of the trajectory vector;

[0103] A first embedding unit is used to divide the trajectory vector group into a first vector group and a second vector group according to the paragraph vector, embed the first paragraph vector in the first vector group, and embed the last paragraph vector in the second vector group;

[0104] The second construction unit is used to construct a position embedding vector based on the position of each vector in the trajectory vector group to obtain an input vector.

[0105] Optionally, the training device further comprises:

[0106] The third processing module is used to randomly select a label value of the satellite positioning trajectory data from the input vector group for prediction.

[0107] Optionally, in the training device, the second processing module 54 includes:

[0108] A third processing unit is used to concatenate the input vector group into a matrix, and multiply them by a trainable parameter matrix respectively to obtain a query matrix, a key matrix, and a value matrix; calculate an output matrix of the attention layer according to the query matrix, the key matrix, and the value matrix; concatenate multiple output matrices, and convert them using matrix multiplication to obtain a multi-head attention layer;

[0109] A second acquisition unit is used to calculate the residual link according to the input vector group and the previous layer function of the residual link, and normalize each vector in the input vector group to obtain a normalized result, thereby obtaining a residual link and a normalized layer;

[0110] a fourth processing unit, for calculating an activation function for linking a feedforward layer consisting of two fully connected neural network layers according to the input vector group and the trainable model parameters;

[0111] The fifth processing unit is used to input the input vector group into the pre-training model composed of a plurality of the multi-head attention layers, the residual links and the normalization layer and the feedforward layer connected in series for training.

[0112] It should be noted here that the above-mentioned device provided in the embodiment of the present invention can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects that are the same as the method embodiment in this embodiment will not be described in detail here.

[0113] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A training method for a pre-trained model based on satellite positioning trajectory data, characterized in that: The method comprises: Acquire satellite positioning trajectory data of multiple vehicles in a target area; wherein the satellite positioning trajectory data includes status data of the vehicle at multiple consecutive position points, and the status data includes at least latitude and longitude coordinates; Processing the satellite positioning trajectory data to obtain a dense vector and a trajectory label of the satellite positioning trajectory data; Obtain an input vector group according to the dense vector and the trajectory label; The input vector group is input into a pre-trained model for training.

2. The training method according to claim 1, characterized in that: The processing of the satellite positioning trajectory data to obtain a dense vector and a trajectory label of the satellite positioning trajectory data includes: Dividing the target area into a plurality of non-overlapping grids; For each position point in the satellite positioning trajectory data, calculate the distance between the position point and each grid center point according to the longitude and latitude coordinates of the position point, and generate a vectorized representation of the longitude and latitude coordinates according to the distance between the position point and each grid center point; replace the longitude and latitude coordinates in the status data of each position point in the satellite positioning trajectory data with the vectorized representation of the longitude and latitude coordinates to obtain the dense vector of the satellite positioning trajectory data; Based on a map matching algorithm, the road segment code in the road network matched by each position point in the satellite positioning trajectory data is determined, and the road segment code is converted into a one-hot encoded vector as the label value corresponding to the position point to obtain the trajectory label.

3. The training method according to claim 1, characterized in that: The step of obtaining an input vector group according to the dense vector and the trajectory label includes: Constructing an initial vector group, the initial vector group comprising a trajectory vector group composed of the dense vectors of the satellite positioning trajectory data, and a starting vector and a paragraph vector having the same length as the dense vector, wherein the starting vector is set before the trajectory vector group, and the paragraph vector is set at the middle position and the end position of the trajectory vector; Dividing the trajectory vector group into a first vector group and a second vector group according to the paragraph vector, embedding the first paragraph vector in the first vector group, and embedding the last paragraph vector in the second vector group; Based on the position of each vector in the trajectory vector group, a position embedding vector is constructed to obtain an input vector.

4. The training method according to claim 1, characterized in that: After obtaining the input vector group, the method further includes: A label value of the satellite positioning trajectory data is randomly selected from the input vector group for prediction.

5. The training method according to claim 1, characterized in that: The step of inputting the input vector group into a pre-trained model for training includes: The input vector groups are concatenated into matrices, and the trainable parameter matrices are multiplied respectively to obtain a query matrix, a key matrix, and a value matrix; an output matrix of the attention layer is calculated according to the query matrix, the key matrix, and the value matrix; a plurality of the output matrices are concatenated, and converted by matrix multiplication to obtain a multi-head attention layer; Calculating residual links according to the input vector group and the previous layer function of the residual link, and normalizing each vector in the input vector group to obtain a normalized result, thereby obtaining a residual link and a normalized layer; Calculating an activation function based on the input vector group and the trainable model parameters for linking a feed-forward layer consisting of two fully connected neural network layers; The input vector group is input into the pre-training model composed of a plurality of the multi-head attention layers, the residual links and the normalization layer and the feed-forward layer connected in series for training.

6. A training device for a pre-trained model based on satellite positioning trajectory data, characterized in that: The device comprises: A first acquisition module is used to acquire satellite positioning trajectory data of multiple vehicles in a target area; wherein the satellite positioning trajectory data includes status data of the vehicle at multiple consecutive position points, and the status data at least includes latitude and longitude coordinates; A first processing module is used to process the satellite positioning trajectory data to obtain a dense vector and a trajectory label of the satellite positioning trajectory data; A second acquisition module, used for acquiring an input vector group according to the dense vector and the trajectory label; The second processing module is used to input the input vector group into a pre-training model for training.

7. The training device according to claim 6, characterized in that The first processing module comprises: A first processing unit, configured to divide the target area into a plurality of non-overlapping grids; A second processing unit is used to calculate, for each position point in the satellite positioning trajectory data, the distance between the position point and each grid center point according to the longitude and latitude coordinates of the position point, and generate a vectorized representation of the longitude and latitude coordinates according to the distance between the position point and each grid center point; replace the longitude and latitude coordinates in the status data of each position point in the satellite positioning trajectory data with the vectorized representation of the longitude and latitude coordinates to obtain the dense vector of the satellite positioning trajectory data; The first acquisition unit is used to determine the section code in the road network matched by each position point in the satellite positioning trajectory data based on a map matching algorithm, convert the section code into a one-hot encoded vector as a label value corresponding to the position point, and obtain the trajectory label.

8. The training device according to claim 6, characterized in that The second acquisition module includes: A first construction unit is used to construct an initial vector group, wherein the initial vector group includes a trajectory vector group composed of the dense vectors of the satellite positioning trajectory data, and a start vector and a paragraph vector having the same length as the dense vector, wherein the start vector is set before the trajectory vector group, and the paragraph vector is set at the middle position and the end position of the trajectory vector; A first embedding unit is used to divide the trajectory vector group into a first vector group and a second vector group according to the paragraph vector, embed the first paragraph vector in the first vector group, and embed the last paragraph vector in the second vector group; The second construction unit is used to construct a position embedding vector based on the position of each vector in the trajectory vector group to obtain an input vector.

9. The training device according to claim 6, characterized in that: The training device further comprises: The third processing module is used to randomly select a label value of the satellite positioning trajectory data from the input vector group for prediction.

10. The training device according to claim 6, characterized in that: The second processing module comprises: A third processing unit is used to concatenate the input vector group into a matrix, and multiply them by a trainable parameter matrix respectively to obtain a query matrix, a key matrix, and a value matrix; calculate an output matrix of the attention layer according to the query matrix, the key matrix, and the value matrix; concatenate multiple output matrices, and convert them using matrix multiplication to obtain a multi-head attention layer; A second acquisition unit is used to calculate the residual link according to the input vector group and the previous layer function of the residual link, and normalize each vector in the input vector group to obtain a normalized result, thereby obtaining a residual link and a normalized layer; a fourth processing unit, for calculating an activation function for linking a feedforward layer consisting of two fully connected neural network layers according to the input vector group and the trainable model parameters; The fifth processing unit is used to input the input vector group into the pre-training model composed of a plurality of the multi-head attention layers, the residual links and the normalization layer and the feedforward layer connected in series for training.

Images