A target vehicle trajectory prediction method for multi-lane scenarios

Abstract

This invention discloses a target vehicle trajectory prediction method for multi-lane scenarios, comprising: extracting historical trajectory features from the historical state information of the target vehicle and the historical state information of surrounding vehicles using a historical trajectory feature encoding module to obtain the historical trajectory features of the target vehicle and the historical trajectory features of surrounding vehicles; processing the historical trajectory features of the target vehicle and the historical trajectory features of surrounding vehicles using an interaction feature encoding module to obtain the interaction features of the target vehicle; fusing the historical trajectory features and interaction features of the target vehicle using a multi-model decomposition bilinear pooling module to obtain the trajectory encoding of the target vehicle; processing the trajectory encoding of the target vehicle using a multi-model identification module to obtain the lateral action classification result of the target vehicle; and predicting the trajectory distribution of the target vehicle and determining the reference driving trajectory of the target vehicle using a trajectory generation module based on the lateral action classification result and the trajectory encoding of the target vehicle.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a target vehicle estimation and prediction method, electronic device, and storage medium for multi-lane scenarios. Background Technology

[0002] For the foreseeable future, roads will be a mix of autonomous and human-driven vehicles, with autonomous vehicles inevitably interacting frequently with human-driven vehicles. The trajectories of human-driven vehicles are inherently random, necessitating accurate prediction of the future trajectories of vehicles encountering obstacles to ensure safety.

[0003] Existing trajectory prediction methods can be divided into two categories: model-based prediction methods and learning-based prediction methods. Model-based prediction methods are computationally efficient but have poor accuracy; while learning-based methods generally require long computation times, especially in complex traffic scenarios with many interactions, where their computational efficiency is extremely low. Therefore, there is still considerable room for improvement in the accuracy of existing trajectory prediction algorithms, and obtaining high-accuracy target vehicle trajectory prediction results with high computational efficiency remains a challenging problem in this technical field. Summary of the Invention

[0004] In view of the above problems, the present invention provides a target vehicle estimation and prediction method, electronic device and storage medium for multi-lane scenarios, in order to solve at least one of the above problems.

[0005] According to a first aspect of the present invention, a method for predicting the trajectory of a target vehicle in a multi-lane scenario is provided, characterized in that it includes:

[0006] The historical trajectory feature encoding module is used to extract historical trajectory features from the historical state information of the target vehicle and the historical state information of the surrounding vehicles, respectively, to obtain the historical trajectory features of the target vehicle and the historical trajectory features of the surrounding vehicles.

[0007] The interactive feature encoding module is used to process the historical trajectory features of the target vehicle and the historical trajectory features of surrounding vehicles to obtain the interactive features of the target vehicle.

[0008] The historical trajectory features and interaction features of the target vehicle are fused using the multimodal decomposition bilinear pooling module to obtain the trajectory code of the target vehicle;

[0009] The trajectory coding of the target vehicle is processed using a multi-model identification module to obtain the lateral motion classification results of the target vehicle;

[0010] Based on the lateral motion classification results and trajectory coding of the target vehicle, the trajectory generation module is used to predict the trajectory distribution of the target vehicle.

[0011] According to an embodiment of the present invention, the above-mentioned historical trajectory feature encoding module extracts historical trajectory features from the historical state information of the target vehicle and the historical state information of the surrounding vehicles, respectively, to obtain the historical trajectory features of the target vehicle and the historical trajectory features of the surrounding vehicles, including:

[0012] The historical trajectory feature encoding module uses a fully connected layer to upsample the historical state information of the target vehicle and the historical state information of surrounding vehicles, respectively.

[0013] The first long short-term memory network of the historical trajectory feature encoding module is used to extract features from the upsampling results of the target vehicle to obtain the historical trajectory features of the target vehicle;

[0014] The second long short-term memory network of the historical trajectory feature encoding module is used to extract features from the upsampling results of surrounding vehicles to obtain the historical trajectory features of surrounding vehicles. The first long short-term memory network and the second long short-term memory network have different parameters.

[0015] According to an embodiment of the present invention, the above-described method of using an interactive feature encoding module to process the historical trajectory features of the target vehicle and the historical trajectory features of surrounding vehicles to obtain the interactive features of the target vehicle includes:

[0016] By performing vector zero-filling and expansion operations on the historical trajectory features of surrounding vehicles, the social tensor of surrounding vehicles is obtained.

[0017] The social features of surrounding vehicles are obtained by using the long short-term memory network of the interactive feature encoding module to extract features from the social tensor of surrounding vehicles.

[0018] The social features of surrounding vehicles are processed using the convolutional social pooling layer of the interaction feature encoding module to obtain the spatial interaction feature vector of surrounding vehicles.

[0019] The spatial interaction feature vectors of surrounding vehicles are flattened and stacked using the interaction feature encoding module to obtain the historical interaction feature tensor of surrounding vehicles.

[0020] The multi-head attention mechanism of the interactive feature encoding module is used to process the historical trajectory features of the target vehicle and the historical interaction feature tensors of surrounding vehicles to obtain the attention head vector;

[0021] The attention head vector is fused using the linear gating unit of the interaction feature encoding module to obtain the interaction features representing the target vehicle. The interaction features of the target vehicle include the spatial interaction features and temporal interaction features between the target vehicle and surrounding vehicles.

[0022] According to an embodiment of the present invention, the above-mentioned feature fusion of the historical trajectory features and interaction features of the target vehicle using the multi-modal decomposition bilinear pooling module to obtain the trajectory encoding of the target vehicle includes:

[0023] The historical trajectory features and interaction features of the target vehicle are upsampled by the fully connected layer of the multimodal decomposition bilinear pooling module, respectively, to obtain the upsampled results of the historical trajectory features and the upsampled results of the interaction features.

[0024] The Hadamard product operation is performed on the upsampling results of historical trajectory features and the upsampling results of interaction features using the multimodal decomposition bilinear pooling module;

[0025] The result of the Hadamard product operation is processed by the summation pooling layer of the multimodal decomposition bilinear pooling module to obtain the pooling result. The window of the summation pooling layer has a preset value.

[0026] The trajectory code of the target vehicle is obtained by using the normalized layer fusion pooling result of the multimodal decomposition bilinear pooling module.

[0027] According to an embodiment of the present invention, the above-mentioned processing of the trajectory encoding of the target vehicle using a multi-model identification module to obtain the lateral motion classification result of the target vehicle includes:

[0028] The trajectory encoding of the target vehicle is encoded using a fully connected linear layer with preset parameters in the multi-model identification module to perform trajectory recognition, and the trajectory recognition result is obtained.

[0029] The trajectory recognition results are processed using the Softmax function in the multi-model identification module to obtain the lateral motion classification results of the target vehicle.

[0030] According to an embodiment of the present invention, the above-described prediction of the trajectory distribution of the target vehicle using the trajectory generation module based on the lateral motion classification result of the target vehicle and the trajectory code of the target vehicle includes:

[0031] The lateral motion classification results of the target vehicle are converted into a multi-dimensional motion trajectory vector using one-hot encoding.

[0032] The multidimensional motion trajectory vector is vector-connected with the trajectory code of the target vehicle to obtain the vector connection result;

[0033] The trajectory distribution of the target vehicle is obtained by processing the vector connection results using the long short-term memory network of the trajectory generation module.

[0034] According to an embodiment of the present invention, the above-mentioned lateral movement classification results include the target vehicle maintaining a straight line, the target vehicle moving to the left, and the target vehicle moving to the right.

[0035] According to a second aspect of the present invention, a method for training a target vehicle trajectory prediction model for a multi-lane scenario is provided, characterized in that it includes:

[0036] Based on a preset cluster center distance function, the K-means method is used to cluster the public dataset given by the user to obtain training samples with trajectory patterns, wherein the trajectory patterns represent the categories of lateral movements;

[0037] The historical trajectory feature encoding module of the target vehicle trajectory prediction model is used to extract the historical trajectory features of the target vehicle and surrounding vehicles of the training samples with trajectory patterns.

[0038] The interactive feature encoding module of the target vehicle trajectory prediction model is used to process the historical trajectory features of the target vehicle and the historical trajectory features of the surrounding vehicles to obtain the interactive features of the target vehicle.

[0039] The historical trajectory features and interaction features of the target vehicle are fused using the multi-modal decomposition bilinear pooling module of the target vehicle trajectory prediction model to obtain the trajectory code of the target vehicle.

[0040] The trajectory encoding of the target vehicle is processed using the multi-model identification module of the target vehicle trajectory prediction model to obtain the lateral motion classification result of the target vehicle;

[0041] Based on the lateral motion classification results of the target vehicle and the trajectory code of the target vehicle, the trajectory generation module of the target vehicle trajectory prediction model is used to predict the trajectory distribution of the target vehicle.

[0042] The loss function is used to calculate the loss value between the trajectory distribution of the target vehicle and the label value of the training samples with trajectory patterns, and the parameters of the target vehicle trajectory prediction model are optimized based on the loss value.

[0043] The target vehicle trajectory prediction model is iteratively trained and optimized until the preset conditions are met, resulting in a trained target vehicle trajectory prediction model. The trained target vehicle trajectory prediction model is then applied to the target vehicle trajectory prediction method in a multi-lane scenario.

[0044] According to a third aspect of the present invention, an electronic device is provided, characterized in that it comprises:

[0045] One or more processors;

[0046] Storage device for storing one or more programs.

[0047] When one or more programs are executed by one or more processors, the one or more processors execute a target vehicle trajectory prediction method for multi-lane scenarios and a training method for a target vehicle trajectory prediction model for multi-lane scenarios.

[0048] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, characterized in that executable instructions are stored thereon, which, when executed by a processor, cause the processor to perform a target vehicle trajectory prediction method for a multi-lane scenario and a training method for a target vehicle trajectory prediction model for a multi-lane scenario.

[0049] The target vehicle prediction method provided by this invention extracts interaction feature vectors with spatiotemporal information through a multi-head attention mechanism and extracts historical interaction information between vehicles with relatively low computational cost, thereby improving the accuracy of trajectory prediction. Simultaneously, this invention uses multi-modal decomposition and bilinear pooling to deeply fuse historical trajectory features and interaction features, further enhancing trajectory prediction accuracy. The predicted driving trajectory of the target vehicle obtained by the above-mentioned method provided by this invention is efficient, accurate, and reliable, and can improve the driving safety of the target vehicle in multi-lane scenarios. Attached Figure Description

[0050] Figure 1 This is a flowchart of a target vehicle trajectory prediction method for a multi-lane scenario according to an embodiment of the present invention;

[0051] Figure 2 This is a flowchart of obtaining historical trajectory features according to an embodiment of the present invention;

[0052] Figure 3 This is a flowchart illustrating the acquisition of target vehicle interaction features according to an embodiment of the present invention;

[0053] Figure 4 This is a flowchart of obtaining the trajectory code of a target vehicle according to an embodiment of the present invention;

[0054] Figure 5 This is a flowchart of obtaining the lateral motion classification results of the target vehicle according to an embodiment of the present invention;

[0055] Figure 6 This is a flowchart illustrating the prediction of the trajectory distribution of a target vehicle according to an embodiment of the present invention;

[0056] Appendix Figure 7 This is a schematic diagram of the structure of a target vehicle trajectory prediction model in a multi-lane scenario according to an embodiment of the present invention;

[0057] Figure 8 This is a flowchart of a training method for a target vehicle trajectory prediction model in a multi-lane scenario according to an embodiment of the present invention;

[0058] Appendix Figure 9 This describes the distribution of different model trajectories after the lateral model of the trajectory is re-divided using a clustering method according to an embodiment of the present invention.

[0059] Figure 10(a) shows the prediction of the target vehicle when it moves a large distance to the left according to an embodiment of the present invention;

[0060] Figure 10(b) shows the prediction of a target vehicle moving to the left within a lane according to an embodiment of the present invention;

[0061] Figure 10(c) shows the prediction of the target vehicle when it moves a large distance to the right according to an embodiment of the present invention;

[0062] Figure 10(d) shows the prediction when the target vehicle moves a small distance to the right according to an embodiment of the present invention;

[0063] Figure 10(e) shows the prediction of the target vehicle maintaining a straight-line driving situation according to an embodiment of the present invention;

[0064] Figure 10(f) is a prediction of the target vehicle maintaining a straight course in a congested traffic environment according to an embodiment of the present invention;

[0065] Appendix Figure 11 This is a schematic diagram of the prediction results and attention weight distribution in the case of congested road sections according to an embodiment of the present invention;

[0066] Appendix Figure 12 This is a schematic diagram of the prediction results and attention weight distribution under unobstructed road conditions according to an embodiment of the present invention;

[0067] Appendix Figure 13 This is a schematic diagram of the prediction results and attention weight distribution when the target vehicle changes lanes, according to an embodiment of the present invention.

[0068] Appendix Figure 14 This is a schematic diagram of the prediction results and attention weight distribution when a target vehicle avoids an obstacle in the lane, according to an embodiment of the present invention.

[0069] Figure 15 The diagram illustrates an electronic device suitable for implementing a target vehicle trajectory prediction method for multi-lane scenarios and a training method for a target vehicle trajectory prediction model for multi-lane scenarios, according to embodiments of the present invention. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0071] For trajectory prediction, the LSTM-based encoder-decoder structure is a widely accepted solution. However, how to better capture the interaction relationships between vehicles is a common problem encountered in existing research. Furthermore, how to achieve deep fusion of historical trajectory features and interaction features is also an urgent research question. Based on this, this invention provides a target vehicle trajectory prediction method for multi-lane scenarios to address the problems existing in this technical field, and the method also has significant engineering application value.

[0072] It should be particularly noted that, in the technical solution disclosed in this invention, the acquisition of relevant data of the target vehicle and its surrounding vehicles has been authorized by the parties involved, and the processing, application and storage of the above data are carried out with the permission of the parties involved. The relevant process complies with the provisions of laws and regulations, and necessary and reliable confidentiality measures have been taken, which is in line with the requirements of public order and good morals.

[0073] Figure 1 This is a flowchart of a target vehicle trajectory prediction method for a multi-lane scenario according to an embodiment of the present invention.

[0074] like Figure 1 As shown, the above-mentioned target vehicle trajectory prediction method for multi-lane scenarios includes operations S110 to S150.

[0075] In operation S110, the historical trajectory feature encoding module is used to extract historical trajectory features from the historical state information of the target vehicle and the historical state information of the surrounding vehicles, respectively, to obtain the historical trajectory features of the target vehicle and the historical trajectory features of the surrounding vehicles.

[0076] The surrounding vehicles mentioned above refer to vehicles adjacent to the target vehicle within a certain range. This range can be set according to the user's preferences.

[0077] The aforementioned historical status information refers to relevant information about the target vehicle over a period of time prior to the current moment, including vehicle location, speed, acceleration, and vehicle type information.

[0078] Figure 2 This is a flowchart for obtaining historical trajectory features according to an embodiment of the present invention.

[0079] like Figure 2 As shown, the above-mentioned historical trajectory feature encoding module extracts historical trajectory features from the historical state information of the target vehicle and the historical state information of the surrounding vehicles, respectively, to obtain the historical trajectory features of the target vehicle and the historical trajectory features of the surrounding vehicles, including operations S210 to S230.

[0080] In operation S210, the fully connected layer of the historical trajectory feature encoding module is used to upsample the historical state information of the target vehicle and the historical state information of the surrounding vehicles.

[0081] First, the collected historical status information This serves as the input to the target vehicle trajectory prediction network. Among them, Indicates the first The car from the timestamp Historical state information up to 0, among which, For the target vehicle, Indicates surrounding vehicles. The length of the historical trajectory. Historical state information vector. Includes vehicle location ,speed acceleration Vehicle type information .

[0082] The collected historical status information (including the historical status information of the target vehicle and surrounding vehicles) is upsampled using formula (1), as shown in formula (1):

[0083] (1),

[0084] in, , Indicates having parameters Multilayer perceptron.

[0085] In operation S220, the first long short-term memory network of the historical trajectory feature encoding module is used to extract features from the upsampling results of the target vehicle to obtain the historical trajectory features of the target vehicle.

[0086] In operation S230, the second long short-term memory network of the historical trajectory feature encoding module is used to extract features from the upsampling results of surrounding vehicles to obtain the historical trajectory features of surrounding vehicles. The first long short-term memory network and the second long short-term memory network have different parameters.

[0087] Operations S210 to S230 above utilize multiple Long Short-Term Memory (LSTM) networks (hereinafter the same) in the historical trajectory feature encoding module to extract the historical trajectory features of the target vehicle and surrounding vehicles respectively.

[0088] For the target vehicle, historical trajectory features will be extracted using formula (2). In this invention, the final hidden state will be used. The historical trajectory feature vector of the target vehicle is denoted as... As shown in formula (2):

[0089] (2),

[0090] in, These are the parameters of LSTM (Long Short-Term Memory Network).

[0091] The historical trajectories of surrounding vehicles will be extracted using formula (3). As shown in formula (3):

[0092] (3),

[0093] in, Indicates the first Adjacent vehicles, The parameters represent the LSTM parameters, which are shared by all surrounding vehicles. LSTM.

[0094] In operation S120, the interactive feature encoding module is used to process the historical trajectory features of the target vehicle and the historical trajectory features of surrounding vehicles to obtain the interactive features of the target vehicle.

[0095] Figure 3 This is a flowchart of obtaining the interaction features of a target vehicle according to an embodiment of the present invention.

[0096] like Figure 3 As shown, the above-mentioned interactive feature encoding module processes the historical trajectory features of the target vehicle and the historical trajectory features of surrounding vehicles to obtain the interactive features of the target vehicle, including operations S310 to S360.

[0097] In operation S310, vector zero-filling and expansion operations are performed on the historical trajectory features of surrounding vehicles to obtain the social tensor of surrounding vehicles.

[0098] When operating S320, the long short-term memory network of the interactive feature encoding module is used to extract features from the social tensor of surrounding vehicles to obtain the social features of surrounding vehicles.

[0099] In operation S330, the convolutional social pooling layer of the interaction feature encoding module is used to process the social features of surrounding vehicles, and the spatial interaction feature vector of surrounding vehicles is obtained.

[0100] In operation S340, the interaction feature encoding module is used to flatten and stack the spatial interaction feature vectors of surrounding vehicles to obtain the historical interaction feature tensor of surrounding vehicles.

[0101] When operating the S350, the multi-head attention mechanism of the interactive feature encoding module is used to process the historical trajectory features of the target vehicle and the historical interactive feature tensors of surrounding vehicles to obtain the attention head vector.

[0102] When operating S360, the linear gating unit of the interaction feature encoding module is used to fuse the attention head vector to obtain the interaction features representing the target vehicle. The interaction features of the target vehicle include the spatial interaction features and temporal interaction features between the target vehicle and surrounding vehicles.

[0103] A social network-based interaction feature encoding module is constructed, which puts the trajectory features of vehicles around different locations into the corresponding positions of the grid. Then, a convolutional social pooling layer is used to extract spatial interaction features, and a multi-head attention mechanism combined with a linear gating unit is used to extract temporal interaction features, thereby obtaining the interaction features of the target vehicle.

[0104] time The hidden state vectors of all neighboring vehicles at a given location Place it into a grid, fill the spaces without vehicles with zero vectors, and thus form a social tensor. The grid size is [size missing]. Then, the social tensor at each time step is processed by two convolutional layers and one max-pooling layer to output the historical interaction features between vehicles. This process can be represented by formula (4):

[0105] (4),

[0106] Among them, Indicates having Convolution kernel and parameters Convolutional layers, It has Convolution kernel and parameters The convolutional layer. For max pooling layer, This represents the flattening operation, and the historical interaction features obtained from this process. It is a one-dimensional column vector. Stacking all historical interaction feature vectors together forms a tensor. .

[0107] Then, a multi-head attention mechanism is employed to weight historical interaction features based on their influence on the target vehicle's behavior patterns. This is achieved first by utilizing three learnable fully connected layers with LeakyReLU non-linear activation functions. , and To process tensors Characteristics of historical trajectory As shown in formulas (5) to (7):

[0108] (5),

[0109] (6),

[0110] (7),

[0111] in, Indicates the first One point of attention, , , as well as These are the parameters of the fully connected layer. , as well as These represent the outputs of each fully connected layer, the first... The output of each attention head can be expressed by formula (8):

[0112] (8),

[0113] in, Represent two matrices and The inner product, express The length of the attention mechanism. This is a query statement; here are the processed historical trajectory features of the target vehicle. The query statement uses a "key". Obtain the weighted values ​​of historical interaction features and then merge the historical interaction features together.

[0114] Next, the results of all attention heads are integrated into a one-dimensional vector to obtain the attention head vector. A linear gated unit (GLU) is used to fuse the outputs of each attention head to obtain the interaction features of the target vehicle, as shown in equations (9) and (10):

[0115] (9),

[0116] (10)

[0117] in, and These represent the learnable weights and biases of the GLU, respectively. This represents the element-wise multiplication of the preceding and following vectors, i.e., the Hadamard product.

[0118] In operation S130, the multi-modal decomposition bilinear pooling module is used to fuse the historical trajectory features and interaction features of the target vehicle to obtain the trajectory code of the target vehicle.

[0119] Figure 4This is a flowchart for obtaining the trajectory code of a target vehicle according to an embodiment of the present invention.

[0120] like Figure 4 As shown, the above-mentioned use of the multi-modal decomposition bilinear pooling module to perform feature fusion on the historical trajectory features and interaction features of the target vehicle to obtain the trajectory encoding of the target vehicle includes operations S410 to S440.

[0121] In the operation of S410, the fully connected layer of the multimodal decomposition bilinear pooling module is used to upsample the historical trajectory features and interaction features of the target vehicle, respectively, to obtain the upsampled results of the historical trajectory features and the upsampled results of the interaction features.

[0122] In operating S420, the Hadamard product operation is performed on the upsampling results of historical trajectory features and the upsampling results of interaction features using the multimodal decomposition bilinear pooling module.

[0123] In operation S430, the result of the Hadamard product operation is processed by the summation pooling layer of the multimodal decomposition bilinear pooling module to obtain the pooling result. The window of the summation pooling layer has a preset value.

[0124] In operation S440, the trajectory code of the target vehicle is obtained by using the normalized layer fusion pooling result of the multimodal decomposition bilinear pooling module.

[0125] Operations S410 to S440 above utilize the multimodal decomposition bilinear pooling module to fuse the target vehicle's interaction features and historical trajectory features to complete trajectory encoding.

[0126] First, the historical trajectory features and interaction features of the target vehicle are upsampled through a fully connected layer, as shown in formulas (11) and (12):

[0127] (11),

[0128] (12),

[0129] in, and These are the parameters for the two fully connected layers. , This represents the output of the fully connected layer. Next, a Hadamard product operation is performed on the two outputs, and then passed through a window of size equal to... The feature fusion output is completed by a summation pooling layer and a normalization layer, as shown in formula (13):

[0130] (13)

[0131] in, The value depends on the upsampling factor in the first step. The final normalization layer can prevent the network from converging to a local minimum.

[0132] In operation S140, the trajectory encoding of the target vehicle is processed using the multi-model identification module to obtain the lateral motion classification result of the target vehicle.

[0133] Figure 5 This is a flowchart of obtaining the lateral motion classification results of a target vehicle according to an embodiment of the present invention.

[0134] like Figure 5 As shown, the above-mentioned use of the multi-model identification module to process the trajectory encoding of the target vehicle and obtain the lateral motion classification results of the target vehicle include operations S510 to S520.

[0135] When operating S510, the fully connected linear layer with preset parameters in the multi-model identification module is used to encode the trajectory of the target vehicle and perform trajectory recognition to obtain the trajectory recognition result.

[0136] When operating the S520, the Softmax function in the multi-model identification module is used to process the trajectory recognition results to obtain the lateral motion classification results of the target vehicle.

[0137] In operation S150, based on the lateral motion classification results of the target vehicle and the trajectory code of the target vehicle, the trajectory generation module is used to predict the trajectory distribution of the target vehicle.

[0138] Figure 6 This is a flowchart illustrating the prediction of the trajectory distribution of a target vehicle according to an embodiment of the present invention.

[0139] like Figure 6 As shown, the above-mentioned prediction of the trajectory distribution of the target vehicle using the trajectory generation module based on the lateral motion classification results and trajectory coding of the target vehicle includes operations S610 to S630.

[0140] When operating the S610, the lateral motion classification results of the target vehicle are converted into a multi-dimensional motion trajectory vector through one-hot encoding.

[0141] In operation S620, the multidimensional motion trajectory vector is vector-connected with the trajectory code of the target vehicle to obtain the vector connection result.

[0142] When operating the S630, the long short-term memory network of the trajectory generation module is used to process the vector connection results to obtain the trajectory distribution of the target vehicle.

[0143] Select a driving trajectory that meets the preset conditions from the trajectory distribution of the target vehicle as the reference driving trajectory for decision planning of the target vehicle.

[0144] According to an embodiment of the present invention, the above-mentioned lateral movement classification results include the target vehicle maintaining a straight line, the target vehicle moving to the left, and the target vehicle moving to the right.

[0145] The preset conditions include using the predicted trajectory with the highest probability as the reference trajectory for the target vehicle.

[0146] The target vehicle prediction method provided by this invention extracts interaction feature vectors with spatiotemporal information through a multi-head attention mechanism and extracts historical interaction information between vehicles with relatively low computational cost, thereby improving the accuracy of trajectory prediction. Simultaneously, this invention uses multi-modal decomposition and bilinear pooling to deeply fuse historical trajectory features and interaction features, further enhancing trajectory prediction accuracy. The predicted driving trajectory of the target vehicle obtained by the above-mentioned method provided by this invention is efficient, accurate, and reliable, and can improve the driving safety of the target vehicle in multi-lane scenarios.

[0147] The above operations S140 to S160 will be further explained in detail below with reference to specific embodiments.

[0148] The above method identifies the lateral model of the trajectory through a fully connected layer with a softmax activation function based on the encoder results, as shown in Equation (14):

[0149] (14)

[0150] in, For parameters Linear layers. ,in, , as well as These represent the probabilities of three behavioral patterns: moving straight, moving left, and moving right, respectively.

[0151] First, define the output format as shown in formula (15):

[0152] (15)

[0153] Formula (15) represents the target vehicle from timestamp 1 to... The distribution of locations, among which Represents the prediction range. Target vehicle at timestamp The location distribution at each point is described by a bivariate Gaussian distribution, as shown in formula (16):

[0154] (16)

[0155] in, It is a mean vector. Represents the covariance matrix. As shown in formulas (17) and (18):

[0156] (17)

[0157] (18)

[0158] in, , The mean of the predicted positions of the target vehicles. , They represent , The variance of the distribution This describes the position coordinates. and The covariance. To output the distribution of the target vehicle positions, as shown in formula (19):

[0159] (19).

[0160] Then, one-hot encoding is used to transform each pattern into a three-dimensional vector. ,in, The behavior patterns of moving straight, moving left, and moving right are represented respectively, as shown in formula (20):

[0161] (20).

[0162] Encoding each behavioral pattern with These are connected together as input to the trajectory generation module. The LSTM unit is used to generate the future trajectory of the target vehicle in different modes, as shown in formula (21):

[0163] (twenty one),

[0164] in, These are the parameters of the LSTM unit. The mode... Every moment in the future Output Combining these, we obtain the trajectory prediction outputs for each behavioral pattern: The final trajectory generation module uses vectors... Select the behavior pattern with the highest probability. The model's prediction output is the network's final prediction result, i.e. .

[0165] Appendix Figure 7 This is a schematic diagram of the structure of a target vehicle trajectory prediction model in a multi-lane scenario according to an embodiment of the present invention.

[0166] like Figure 7As shown, the target vehicle trajectory prediction model for the above multi-lane scenario is based on historical interaction feature fusion, including a historical trajectory feature encoding module, a multi-model decomposition bilinear pooling module, an interaction feature encoding module, a multi-model identification module, and a trajectory generation module.

[0167] First, based on the historical state of the target vehicle, a historical trajectory feature vector is extracted. This historical state includes vehicle position, speed, acceleration, and vehicle type information, and the time period is a period prior to the current moment. Then, the historical states of vehicles surrounding the target vehicle are collected, and an interactive feature encoder is used to extract interactive feature vectors with spatiotemporal information.

[0168] The multi-modal decomposition bilinear pooling module is used to deeply fuse the historical trajectory feature vector and interaction feature vector of the target vehicle, and the target vehicle is then encoded.

[0169] The decoder is used to decode the encoded result to generate the possible distribution of the target vehicle's future trajectory.

[0170] The above model will cluster the future lateral displacement of vehicles in the dataset to obtain three cluster centers, and then use the cluster centers to reclassify the vehicle behavior patterns into three categories: keeping a straight line, moving to the left, and moving to the right.

[0171] Interaction feature encoding uses convolutional social pooling layers to extract the interaction features between vehicles at each moment in the historical trajectory, and uses a multi-head attention mechanism to fuse the interaction features at all moments to obtain an interaction feature vector with spatiotemporal information.

[0172] The above model uses multiple LSTM-based trajectory generation modules to generate various possible future trajectories from the encoding results, calculates the possible trajectory patterns of the target vehicle based on the encoding results, and selects the optimal possible future trajectory distribution for output.

[0173] The model provided by this invention is characterized by high efficiency, accuracy and reliability, and can achieve good results in practical applications.

[0174] Figure 8 This is a flowchart of a training method for a target vehicle trajectory prediction model in a multi-lane scenario according to an embodiment of the present invention.

[0175] like Figure 8 As shown, the training method for the target vehicle trajectory prediction model in the multi-lane scenario includes operations S810 to S880.

[0176] In operation S810, the user-given public dataset is clustered using the K-means method according to the preset cluster center distance function to obtain training samples with trajectory patterns, wherein the trajectory patterns represent the categories of lateral movements.

[0177] In operation S820, the historical trajectory feature encoding module of the target vehicle trajectory prediction model is used to extract the historical trajectory features of the target vehicle and the surrounding vehicles of the target vehicle from the training samples with trajectory patterns.

[0178] In operation S830, the interactive feature encoding module of the target vehicle trajectory prediction model is used to process the historical trajectory features of the target vehicle and the historical trajectory features of the surrounding vehicles to obtain the interactive features of the target vehicle.

[0179] In operation S840, the multi-modal decomposition bilinear pooling module of the target vehicle trajectory prediction model is used to perform feature fusion on the historical trajectory features and interaction features of the target vehicle to obtain the trajectory code of the target vehicle.

[0180] In operation S850, the multi-model identification module of the target vehicle trajectory prediction model is used to process the trajectory encoding of the target vehicle to obtain the lateral motion classification result of the target vehicle.

[0181] In operation S860, based on the lateral motion classification result of the target vehicle and the trajectory code of the target vehicle, the trajectory generation module of the target vehicle trajectory prediction model is used to predict the trajectory distribution of the target vehicle.

[0182] When operating the S870, the loss function is used to calculate the loss value between the trajectory distribution of the target vehicle and the label value of the training samples with trajectory patterns, and the parameters of the target vehicle trajectory prediction model are optimized based on the loss value.

[0183] In the S880, the target vehicle trajectory prediction model is iteratively trained and optimized until the preset conditions are met, and the trained target vehicle trajectory prediction model is obtained. The trained target vehicle trajectory prediction model is applied to the target vehicle trajectory prediction method in multi-lane scenarios.

[0184] The following is in conjunction with the appendix Figure 9 The specific embodiments will further illustrate the above operations S810 to S880 in more detail.

[0185] Appendix Figure 9 This describes the distribution of different model trajectories after the lateral model of the trajectory is re-divided using a clustering method according to an embodiment of the present invention.

[0186] This invention targets the NGSIM public dataset, uses the lateral displacement of trajectories at different times to measure the similarity of different trajectories, then performs three-class clustering using the K-means clustering method, and finally uses the three cluster centers to divide the lateral model of the trajectory.

[0187] Specifically, first, the lateral displacement of the future trajectory in the dataset is... As clustering samples, among Represents the first in the dataset a car, This represents the value of the ordinate in the vehicle coordinate system. The predicted duration is then represented; then the distance function defined by formula (22) is used. As a criterion for measuring similarity:

[0188] (twenty two).

[0189] Next, the K-means clustering method is used to obtain three cluster centers, denoted as _____. , , For each trajectory data point in the dataset, the lateral displacement is classified, and the distance function with each cluster center is calculated. The cluster center with the smallest distance is selected as the lateral model for the current trajectory. The final distribution of lateral models for different trajectories is as follows: Figure 9 As shown, the three lateral models of the target vehicle are defined as: going straight, moving left, and moving right.

[0190] To better illustrate the beneficial effects of the prediction method and model training method provided by this invention, this invention is illustrated in conjunction with Figures 10-10. Figure 14 The method provided by the present invention will be further described in detail below.

[0191] Figure 10 shows the prediction results of the target vehicle under different conditions according to an embodiment of the present invention. Figure 10 includes 6 sub-figures, wherein Figure 10(a) is the prediction of the target vehicle when it moves a large distance to the left according to an embodiment of the present invention, Figure 10(b) is the prediction of the target vehicle when it moves to the left within the lane according to an embodiment of the present invention, Figure 10(c) is the prediction of the target vehicle when it moves a large distance to the right according to an embodiment of the present invention, Figure 10(d) is the prediction of the target vehicle when it moves a small distance to the right according to an embodiment of the present invention, Figure 10(e) is the prediction of the target vehicle when it keeps going straight according to an embodiment of the present invention, and Figure 10(f) is the prediction of the target vehicle when it keeps going straight in a congested traffic environment according to an embodiment of the present invention.

[0192] Appendix Figure 11 This is a schematic diagram of the prediction results and attention weight distribution in the case of congested road sections according to an embodiment of the present invention.

[0193] Appendix Figure 12 This is a schematic diagram of the prediction results and attention weight distribution under unobstructed road conditions according to an embodiment of the present invention.

[0194] Appendix Figure 13 This is a schematic diagram of the prediction results and attention weight distribution when the target vehicle changes lanes, according to an embodiment of the present invention.

[0195] Appendix Figure 14 This is a schematic diagram of the prediction results and attention weight distribution when a target vehicle avoids an obstacle within a lane, according to an embodiment of the present invention.

[0196] Figure 10 shows the multi-model prediction results under different traffic scenarios. Figure 10(a) shows the prediction when the target vehicle moves a large distance to the left; Figure 10(b) shows the prediction when the target vehicle moves to the left within the lane; Figure 10(c) shows the prediction when the target vehicle moves a large distance to the right; Figure 10(d) shows the prediction when the target vehicle moves a small distance to the right; Figure 10(e) shows the prediction when the target vehicle stays straight; and Figure 10(f) shows the prediction when the target vehicle stays straight in a congested traffic environment. In each sub-figure of Figure 10, the blue dotted horizontal lines in the trajectory graph on the left represent the historical trajectory around the vehicle, and the red dashed lines represent the historical trajectory of the target vehicle. The red circled dots represent the true values ​​of the future trajectory, and the green solid lines are the future trajectories generated by the three behavior modes. The histogram on the right shows the probability of each model: 1 for moving to the left, 2 for staying straight, and 3 for moving to the right. As can be seen from the various sub-figures of Figure 10, the prediction method provided by this invention can predict different lateral offset actions according to different surrounding environments. It can also be seen that the algorithm can accurately predict the longitudinal position at different speeds.

[0197] Appendix Figure 11 -Appendix Figure 14 This section shows the distribution of trajectory prediction results and attention weights of the algorithm in different scenarios. The upper part of each figure represents the predicted trajectory and historical trajectory, with the symbols having the same meaning as in Figure 10. Here, only the predicted trajectory of the most likely behavior pattern is drawn. The lower part shows the attention distribution of different attention heads along the historical trajectory. The darker the color, the higher the attention weight.

[0198] As can be seen, regardless of traffic conditions, attention head 1 consistently focuses on the last historical interaction features. Attention head 2, on the other hand, focuses on different timestamps under different conditions. In congested traffic, such as... Figure 11 As shown, Note2 considers previous features to be important, so Note2 focuses more on previous features. However, when there are no surrounding vehicles obstructing the target vehicle's movement, such as Figure 12As shown, Note Head 2 focuses more on recent historical interaction features, and like Note Head 1, it rarely considers previous interaction features.

[0199] Appendix Figure 13 and attached Figure 14 The results show that Attention Head 2 noticed different historical timestamps in a scenario where the target vehicle was obstructed. For the former, the target vehicle was prepared to change lanes much earlier, so Attention Head 2's attention was focused further ahead. However, for the latter... Figure 14 The target vehicle made the decision to move to the left to avoid the obstacle a little later, so pay more attention to the later historical interaction features in the first two cases.

[0200] In summary, after training, the proposed algorithm's attention mechanism autonomously learns how to assign weights to historical interaction features and pays attention to interaction features at different times by changing the weights.

[0201] Finally, trajectory prediction is performed on the NGSIM dataset, and the prediction accuracy of the prediction method provided by this invention is evaluated using the RSME index, calculated as shown in equation (22):

[0202] (twenty three),

[0203] in, Indicates time The RMSE value, This represents the total number of data points in the test set. The evaluation results are shown in Table 1.

[0204] Table 1 Evaluation Results

[0205]

[0206] Figure 15 The diagram illustrates an electronic device suitable for implementing a target vehicle trajectory prediction method for multi-lane scenarios and a training method for a target vehicle trajectory prediction model for multi-lane scenarios, according to embodiments of the present invention.

[0207] like Figure 15As shown, an electronic device 1500 according to an embodiment of the present invention includes a processor 1501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1502 or a program loaded from a storage portion 1508 into a random access memory (RAM) 1503. The processor 1501 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1501 may also include onboard memory for caching purposes. The processor 1501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0208] RAM 1503 stores various programs and data required for the operation of electronic device 1500. Processor 1501, ROM 1502, and RAM 1503 are interconnected via bus 1504. Processor 1501 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 1502 and / or RAM 1503. It should be noted that the programs may also be stored in one or more memories other than ROM 1502 and RAM 1503. Processor 1501 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in said one or more memories.

[0209] According to an embodiment of the present invention, the electronic device 1500 may further include an input / output (I / O) interface 1505, which is also connected to the bus 1504. The electronic device 1500 may also include one or more of the following components connected to the I / O interface 1505: an input section 1506 including a keyboard, mouse, etc.; an output section 1507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1508 including a hard disk, etc.; and a communication section 1509 including a network interface card such as a LAN card, modem, etc. The communication section 1509 performs communication processing via a network such as the Internet. A drive 1510 is also connected to the I / O interface 1505 as needed. A removable medium 1511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1510 as needed so that computer programs read from it can be installed into the storage section 1508 as needed.

[0210] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0211] According to embodiments of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of the present invention, a computer-readable storage medium may include ROM 1502 and / or RAM 1503 and / or one or more memories other than ROM 1502 and RAM 1503 described above.

[0212] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0213] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. It should be noted that implementations not illustrated or described in the drawings or the main text of the specification are forms known to those skilled in the art and are not described in detail. Furthermore, the definitions of the elements and methods described above are not limited to the various specific structures, shapes, or methods mentioned in the embodiments, and those skilled in the art can easily modify or substitute them.

[0214] Based on the above description, those skilled in the art should have a clear understanding of the trajectory prediction method for autonomous driving of the present invention.

[0215] It should also be noted that the directional terms mentioned in the embodiments, such as "up," "down," "front," "back," "left," and "right," are only for reference to the directions in the accompanying drawings and are not intended to limit the scope of protection of the present invention. Throughout the accompanying drawings, the same elements are represented by the same or similar reference numerals. Conventional structures or constructions will be omitted where they may cause confusion in understanding the present invention.

[0216] Furthermore, the shapes and dimensions of the components in the figures do not reflect actual size and proportion, but are only illustrative of embodiments of the present invention. Additionally, any reference signs placed between parentheses in the claims should not be construed as limiting the claims.

[0217] Unless otherwise stated, the numerical parameters in this specification and the appended claims are approximate values ​​and can be varied according to the desired characteristics obtained from the content of this invention. Specifically, all figures used in the specification and claims to indicate the content of components, reaction conditions, etc., should be understood to be modified by the term "about" in all cases. Generally, this means that there may be variations of ±10% in some embodiments, ±5% in some embodiments, ±1% in some embodiments, and ±0.5% in some embodiments.

[0218] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for predicting the trajectory of a target vehicle in a multi-lane scenario, characterized in that, include: The historical trajectory features of the target vehicle and the surrounding vehicles are extracted from the historical state information of the target vehicle and the historical state information of the surrounding vehicles respectively using the historical trajectory feature encoding module. The interaction features of the target vehicle are obtained by processing the historical trajectory features of the target vehicle and the historical trajectory features of the surrounding vehicles using the interaction feature encoding module. The historical trajectory features and interaction features of the target vehicle are fused using a multimodal decomposition bilinear pooling module to obtain the trajectory code of the target vehicle; The trajectory coding of the target vehicle is processed using a multi-model identification module to obtain the lateral motion classification result of the target vehicle; Based on the lateral motion classification results of the target vehicle and the trajectory code of the target vehicle, the trajectory distribution of the target vehicle is predicted using the trajectory generation module; Specifically, the interaction feature encoding module processes the historical trajectory features of the target vehicle and the historical trajectory features of the surrounding vehicles to obtain the interaction features of the target vehicle, including: The historical trajectory features of the surrounding vehicles are zero-filled and expanded to obtain the social tensor of the surrounding vehicles. The social features of the surrounding vehicles are obtained by using the long short-term memory network of the interaction feature encoding module to extract features from the social tensor of the surrounding vehicles. The social features of the surrounding vehicles are processed using the convolutional social pooling layer of the interaction feature encoding module to obtain the spatial interaction feature vector of the surrounding vehicles. The interaction feature encoding module is used to flatten and stack the spatial interaction feature vectors of the surrounding vehicles to obtain the historical interaction feature tensor of the surrounding vehicles. The historical trajectory features of the target vehicle and the historical interaction feature tensors of the surrounding vehicles are processed using the multi-head attention mechanism of the interaction feature encoding module to obtain the attention head vector. The attention head vector is fused using the linear gating unit of the interaction feature encoding module to obtain the interaction features characterizing the target vehicle. The interaction features of the target vehicle include the spatial interaction features and temporal interaction features between the target vehicle and the surrounding vehicles.

2. The method according to claim 1, characterized in that, Historical trajectory features are extracted from the historical state information of the target vehicle and the historical state information of the surrounding vehicles using a historical trajectory feature encoding module. The resulting historical trajectory features of the target vehicle and the surrounding vehicles include: The historical state information of the target vehicle and the historical state information of the surrounding vehicles are upsampled using the fully connected layer of the historical trajectory feature encoding module. The historical trajectory features of the target vehicle are obtained by extracting features from the upsampling results of the target vehicle using the first long short-term memory network of the historical trajectory feature encoding module. The historical trajectory features of the surrounding vehicles are obtained by using the second long short-term memory network of the historical trajectory feature encoding module to extract features from the upsampling results of the surrounding vehicles. The first long short-term memory network and the second long short-term memory network have different parameters.

3. The method according to claim 1, characterized in that, The historical trajectory features and interaction features of the target vehicle are fused using a multi-modal decomposition bilinear pooling module to obtain the trajectory encoding of the target vehicle, including: The historical trajectory features and interaction features of the target vehicle are upsampled using the fully connected layer of the multimodal decomposition bilinear pooling module to obtain the upsampled results of the historical trajectory features and the upsampled results of the interaction features. The Hadamard product operation is performed on the historical trajectory feature upsampling results and the interaction feature upsampling results using the multimodal decomposition bilinear pooling module. The result of the Hadamard product operation is processed by the summation pooling layer of the multimodal decomposition bilinear pooling module to obtain the pooling result, wherein the window of the summation pooling layer has a preset value; The trajectory code of the target vehicle is obtained by fusing the pooling results through the normalization layer of the multimodal decomposition bilinear pooling module.

4. The method according to claim 1, characterized in that, The trajectory encoding of the target vehicle is processed using a multi-model identification module to obtain the lateral motion classification results of the target vehicle, including: The trajectory encoding of the target vehicle is used to perform trajectory recognition by utilizing the fully connected linear layer with preset parameters in the multi-model identification module to obtain the trajectory recognition result; The trajectory recognition result is processed using the Softmax function in the multi-model identification module to obtain the lateral motion classification result of the target vehicle.

5. The method according to claim 1, characterized in that, Based on the lateral motion classification results of the target vehicle and the trajectory code of the target vehicle, the trajectory generation module predicts the trajectory distribution of the target vehicle, including: The lateral motion classification results of the target vehicle are converted into a multi-dimensional motion trajectory vector using one-hot encoding. The multidimensional motion trajectory vector is vector-connected with the trajectory code of the target vehicle to obtain the vector connection result; The long short-term memory network of the trajectory generation module is used to process the vector connection results to obtain the trajectory distribution of the target vehicle.

6. The method according to any one of claims 1-5, characterized in that, The lateral movement classification results include the target vehicle maintaining a straight line, the target vehicle moving to the left, and the target vehicle moving to the right.

7. A training method for a target vehicle trajectory prediction model in a multi-lane scenario, characterized in that, include: Based on a preset cluster center distance function, the K-means method is used to cluster the public dataset given by the user to obtain training samples with trajectory patterns, wherein the trajectory patterns represent the categories of lateral movements; The historical trajectory feature encoding module of the target vehicle trajectory prediction model is used to extract the historical trajectory features of the target vehicle and surrounding vehicles of the training samples with trajectory patterns. The interactive feature encoding module of the target vehicle trajectory prediction model is used to process the historical trajectory features of the target vehicle and the historical trajectory features of the surrounding vehicles to obtain the interactive features of the target vehicle. The target vehicle trajectory prediction model is used to fuse the historical trajectory features and interaction features of the target vehicle using the multi-modal decomposition bilinear pooling module to obtain the trajectory code of the target vehicle. The trajectory encoding of the target vehicle is processed using the multi-model identification module of the target vehicle trajectory prediction model to obtain the lateral motion classification result of the target vehicle; Based on the lateral motion classification results of the target vehicle and the trajectory code of the target vehicle, the trajectory generation module of the target vehicle trajectory prediction model is used to predict the trajectory distribution of the target vehicle. The loss function is used to calculate the loss value between the trajectory distribution of the target vehicle and the label value of the training samples with trajectory patterns, and the parameters of the target vehicle trajectory prediction model are optimized based on the loss value. The target vehicle trajectory prediction model is iteratively trained and optimized until a preset condition is met, resulting in a trained target vehicle trajectory prediction model, wherein the trained target vehicle trajectory prediction model is applied to the method described in any one of claims 1-5.

8. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1-5 and 7.

9. A computer-readable storage medium, characterized in that, It stores executable instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1-5 and 7.

Images