A method and device for processing predicted trajectory

Abstract

Embodiments of the present invention relate to a method and apparatus for processing predicted trajectories. The method includes: obtaining a first historical trajectory, a first position, and a first driving sign of an obstacle vehicle; determining the trajectory prediction modality type and the corresponding number of trajectory prediction modalities at the current moment based on the first driving sign; when the trajectory prediction modality type is single-modality, invoking N single-trajectory prediction models to perform single-trajectory prediction, and performing trajectory fusion on the N first predicted trajectories to generate a first fused trajectory; when the trajectory prediction modality type is multi-modality, invoking M multi-trajectory prediction models to perform multi-trajectory prediction to generate a first predicted trajectory set, and performing trajectory fusion on the M first predicted trajectory sets to generate a first fused trajectory set; and integrating the predicted data of the obtained first fused trajectory or first fused trajectory set based on the trajectory prediction modality type to generate predicted trajectory data output. This invention can improve prediction accuracy.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and device for processing predicted trajectories. Background Art

[0002] The trajectory prediction module of an autonomous driving system typically uses a preset trajectory prediction model to predict the obstructing vehicle's future single- or multi-modal trajectory based on its current historical trajectory, real-time location, and possible future driving states (straight ahead, left turn, right turn, U-turn, etc.). Single-modality refers to a single possible driving state (such as straight ahead, left turn, right turn, U-turn, etc.), while multi-modality refers to multiple possible driving states (straight ahead + left turn, straight ahead + right turn, left turn + U-turn, straight ahead + left turn + U-turn, etc.). In actual applications, we have found that due to the differences in the prediction capabilities of different trajectory prediction models under different road conditions, using only a single trajectory prediction model for trajectory prediction does not produce very ideal prediction results, and the predicted trajectory errors under different modes fluctuate significantly. Summary of the Invention

[0003] The purpose of the present invention is to address the defects of the prior art and provide a method, device, electronic device and computer-readable storage medium for processing predicted trajectories, which pre-integrate multiple single trajectory prediction models for single-modal trajectory prediction and multiple multi-trajectory prediction models for multi-modal trajectory prediction; when performing trajectory prediction, the trajectory prediction modal type (single-modal type and multi-modal type) and the number of trajectory prediction modes of the current obstacle vehicle are first confirmed; and when the trajectory prediction modal type is single-modal type, multiple preset single trajectory prediction models are called to perform single-modal trajectory prediction according to the historical trajectory and real-time position of the current obstacle vehicle at the current moment, and the multiple predicted trajectories are merged in a mean manner to obtain a trajectory prediction result. To the final fused trajectory, and the fused trajectory is output as the final predicted trajectory data; and when the trajectory prediction modality type is multimodal, call the preset multiple multi-trajectory prediction models to perform multimodal trajectory prediction based on the historical trajectory, real-time position and multiple possible driving states of the current obstacle vehicle at the current moment, and cluster all the predicted trajectories according to the number of trajectory prediction modes, and perform trajectory fusion and trajectory probability fusion on each clustered trajectory set in a mean manner to obtain the fused trajectory and mean probability of the corresponding mode, and normalize the mean probability of each mode to obtain the fused trajectory probability of the corresponding mode, and then combine the fused trajectory and fused trajectory probability of each mode to form the final predicted trajectory data output. Through the present invention, based on the subdivided mode, using the corresponding multiple single / multi-trajectory prediction models for prediction and trajectory fusion, it is possible to overcome the problem of predicted trajectory error fluctuation caused by using only a single trajectory prediction model for trajectory prediction in conventional technical solutions, and achieve the purpose of maintaining prediction stability and improving prediction accuracy.

[0004] To achieve the above objectives, a first aspect of an embodiment of the present invention provides a method for processing a predicted trajectory, the method comprising:

[0005] Obtain the historical trajectory, real-time position and real-time traffic sign of the lane where the obstacle vehicle is located at the current time t to generate the corresponding first historical trajectory H t-1 , first position p t and the first driving sign A t ; The first driving sign A t Includes unimodal and multimodal signs;

[0006] According to the first driving sign A t Determine the trajectory prediction modality type at the current moment and the corresponding trajectory prediction modality number X; X≥1; the trajectory prediction modality type includes a single modality type and a multi-modality type;

[0007] When the trajectory prediction modality type is a single modality type, a preset first number N of single trajectory prediction models B are called. i According to the first historical trajectory H t-1 and the first position p t Perform single trajectory prediction processing to generate the corresponding first predicted trajectory H i ; and the first predicted trajectory H of the first number N i Perform trajectory fusion processing to generate the corresponding first fusion trajectory 1≤N, 1≤i≤N;

[0008] When the trajectory prediction modality type is a multi-modal type, a preset second number M of multi-trajectory prediction models C are called. j According to the first historical trajectory H t-1 , the first position p t and the first driving sign A t Perform multi-trajectory prediction processing to generate the corresponding first predicted trajectory set G t,j ; and the first predicted trajectory set G of the second number M t,j Perform trajectory fusion processing to generate the corresponding first fusion trajectory set 1≤M, 1≤j≤M;

[0009] The first fusion trajectory obtained according to the trajectory prediction modality type Or the first fusion trajectory set The prediction data is integrated to generate the corresponding prediction trajectory data output.

[0010] Preferably, the single-mode sign includes a straight-ahead sign, a left-turn sign, a right-turn sign, and a U-turn sign; the multi-mode sign includes a straight-ahead + left-turn sign, a straight-ahead + right-turn sign, a left-turn + U-turn sign, and a straight-ahead + left-turn + U-turn sign;

[0011] The first predicted trajectory set G t,j The second predicted trajectory H including the trajectory prediction modality number X j,g And each of the second predicted trajectories H j,g Corresponding to a first predicted trajectory probability ρ j,g , 1≤g≤X;

[0012] The first fusion trajectory set The second fused trajectory including the number X of trajectory prediction modalities And each of the second fusion trajectories Corresponding to a first fusion trajectory probability

[0013] Preferably, the first driving sign A t Determine the trajectory prediction mode type and the corresponding trajectory prediction mode number X at the current moment, specifically including:

[0014] When the first driving sign A t When it belongs to the single mode flag, the trajectory prediction mode type is set to the single mode type, and the corresponding trajectory prediction mode number X is set to 1;

[0015] When the first driving sign A t When the multi-modal flag is set, the trajectory prediction mode type is set to multi-modal type, and the first driving flag A t When the sign is straight + left turn, straight + right turn or left turn + U-turn, the corresponding trajectory prediction mode number X is set to 2, and the first driving sign A t When the sign is straight ahead + left turn + U-turn, the corresponding trajectory prediction mode number X is set to 3.

[0016] Preferably, the first predicted trajectory H of the first number N i Perform trajectory fusion processing to generate the corresponding first fusion trajectory Specifically include:

[0017] The first predicted trajectory H of the first number N i Perform mean calculation to obtain the corresponding first fusion trajectory

[0018] Preferably, the first predicted trajectory set G for the second number Mt,j Perform trajectory fusion processing to generate the corresponding first fusion trajectory set Specifically include:

[0019] The first predicted trajectory set G of the second number M t,j All the second predicted trajectories H j,g The first trajectory set includes M*X second predicted trajectories H j,g ;

[0020] For the M*X second predicted trajectories H of the first trajectory set j,g Clustering is performed to obtain the first type trajectory set of the trajectory prediction modality number X; each of the first type trajectory sets includes M second prediction trajectories H j,g , and corresponding to the M first predicted trajectory probabilities ρ j,g ;

[0021] For each of the M second predicted trajectories H of the first type trajectory set j,g Perform mean calculation to obtain the corresponding second fusion trajectory

[0022] The M first predicted trajectory probabilities ρ corresponding to each of the first type trajectory sets j,g Perform mean calculation to obtain the corresponding first mean probability And predict the first mean probability of the modal number X of the obtained trajectory Perform normalization calculation to obtain the corresponding first fusion trajectory probability The first fusion trajectory probability Fusion trajectory with the second One-to-one correspondence;

[0023] The second fusion trajectory of all The first fusion trajectory set corresponding to

[0024] Preferably, the first fusion trajectory obtained according to the trajectory prediction modality type Or the first fusion trajectory set Integrate the predicted data to generate the corresponding predicted trajectory data output, including:

[0025] Identify the trajectory prediction modality type; if the trajectory prediction modality type is a single modality type, the first fusion trajectory obtained As the corresponding predicted trajectory data output; if the trajectory prediction modality type is multimodal type, the first fusion trajectory set obtained Each of the second fusion trajectories and the corresponding probability of the first fusion trajectory A corresponding first fused trajectory data group is formed, and the first fused trajectory data group of the obtained trajectory prediction modality number X constitutes the corresponding predicted trajectory data output.

[0026] A second aspect of an embodiment of the present invention provides a device for implementing the method for processing predicted trajectories according to the first aspect, the device comprising: an acquisition module, a preprocessing module, a single-modal trajectory prediction module, a multi-modal trajectory prediction module, and an output module;

[0027] The acquisition module is used to obtain the historical trajectory, real-time position and real-time traffic sign of the lane where the obstacle vehicle is located at the current time t to generate the corresponding first historical trajectory H t-1 , first position p t and the first driving sign A t ; The first driving sign A t Includes unimodal and multimodal signs;

[0028] The pre-processing module is used to t Determine the trajectory prediction modality type at the current moment and the corresponding trajectory prediction modality number X; X≥1; the trajectory prediction modality type includes a single modality type and a multi-modality type;

[0029] The single-mode trajectory prediction module is used to call a preset first number N of single trajectory prediction models B when the trajectory prediction modality type is a single-mode type. i According to the first historical trajectory H t-1 and the first position p t Perform single trajectory prediction processing to generate the corresponding first predicted trajectory H i ; and the first predicted trajectory H of the first number N i Perform trajectory fusion processing to generate the corresponding first fusion trajectory 1≤N, 1≤i≤N;

[0030] The multimodal trajectory prediction module is configured to call a preset second number M of multi-trajectory prediction models C when the trajectory prediction modality type is a multimodal type. j According to the first historical trajectory H t-1 , the first position p t and the first driving sign A t Perform multi-trajectory prediction processing to generate the corresponding first predicted trajectory set Gt,j ; and the first predicted trajectory set G of the second number M t,j Perform trajectory fusion processing to generate the corresponding first fusion trajectory set 1≤M, 1≤j≤M;

[0031] The output module is used to obtain the first fusion trajectory according to the trajectory prediction modality type. Or the first fusion trajectory set The prediction data is integrated to generate the corresponding prediction trajectory data output.

[0032] A third aspect of an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;

[0033] The processor is configured to be coupled to the memory, read and execute instructions in the memory, so as to implement the method steps described in the first aspect above;

[0034] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0035] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed by a computer, the computer executes the instructions of the method described in the first aspect above.

[0036] The embodiment of the present invention provides a method, device, electronic device and computer-readable storage medium for processing predicted trajectories; pre-integrates multiple single trajectory prediction models for single-modal trajectory prediction and multiple multi-trajectory prediction models for multi-modal trajectory prediction; when performing trajectory prediction, first confirms the trajectory prediction modality type (single-modal type and multi-modal type) and the number of trajectory prediction modalities of the current obstacle vehicle; and when the trajectory prediction modality type is single-modal type, calls multiple preset single trajectory prediction models to perform single-modal trajectory prediction based on the historical trajectory and real-time position of the current obstacle vehicle at the current moment, and performs trajectory fusion on the obtained multiple predicted trajectories in a mean manner to obtain a final fusion Trajectory, and the fused trajectory is output as the final predicted trajectory data; and when the trajectory prediction modality type is multimodal, multiple preset multi-trajectory prediction models are called to perform multimodal trajectory prediction based on the historical trajectory, real-time position and multiple possible driving states of the current obstacle vehicle at the current moment, and all the predicted trajectories obtained are clustered according to the number of trajectory prediction modalities, and each clustered trajectory set is subjected to trajectory fusion and trajectory probability fusion in a mean manner to obtain the fused trajectory and mean probability of the corresponding modality, and the mean probability of each modality is normalized to obtain the fused trajectory probability of the corresponding modality, and the fused trajectory and fused trajectory probability of each modality are combined to form the final predicted trajectory data output. Through the present invention, based on the subdivided modality, the corresponding multiple single / multi-trajectory prediction models are used for prediction and trajectory fusion, which overcomes the problem of predicted trajectory error fluctuation caused by using only a single trajectory prediction model for trajectory prediction in conventional technical solutions, and improves prediction stability and prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic diagram of a method for processing predicted trajectories provided in the first embodiment of the present invention;

[0038] Figure 2 A module structure diagram of a trajectory prediction processing device provided in the second embodiment of the present invention;

[0039] Figure 3 This is a structural diagram of an electronic device provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0040] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described herein are merely some, rather than all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0041] The first embodiment of the present invention provides a method for processing a predicted trajectory, such as Figure 1 A method for processing a predicted trajectory provided in accordance with the first embodiment of the present invention is shown in the following diagram. The method mainly includes the following steps:

[0042] Step 1: Obtain the historical trajectory, real-time position and real-time traffic sign of the lane where the obstacle vehicle is located at the current time t to generate the corresponding first historical trajectory H t-1 , first position p t and the first driving sign A t ;

[0043] Among them, the first driving sign A t It includes single-mode signs and multi-mode signs; single-mode signs include straight-ahead signs, left-turn signs, right-turn signs and U-turn signs; multi-mode signs include straight-ahead + left-turn signs, straight-ahead + right-turn signs, left-turn + U-turn signs and straight-ahead + left-turn + U-turn signs.

[0044] Here, there are multiple ways for the trajectory prediction module of the autonomous driving system to obtain the historical trajectory, real-time position, and real-time traffic sign of the lane of the obstacle vehicle at the current time t. One of the acquisition methods is to use the historical trajectory perception data, real-time position perception data, and perception data of the traffic sign of the lane corresponding to the current obstacle vehicle from the perception data set output by the upstream perception module as the corresponding first historical trajectory H. t-1 , first position p t and the first driving sign A t Another way to obtain the data is to use the historical trajectory perception data and real-time position perception data corresponding to the current obstacle vehicle from the perception data set output by the upstream perception module as the first historical trajectory H t-1 and the first position p t and obtain the first position p from the high-precision map provided by the upstream map module t The driving traffic sign data of the corresponding road segment is used as the corresponding first driving sign A t ;

[0045] First Historical Track H t-1 is the actual motion trajectory of the current obstacle vehicle for a fixed period of time before time t; the first position p t is the real-time position coordinate of the vehicle with the current obstacle at time t; the first driving mark A tis the lane traffic sign of the lane where the obstacle vehicle is currently located at time t. The lane traffic signs may include the following: straight sign, left turn sign, right turn sign, U-turn sign, straight + left turn sign, straight + right turn sign, left turn + U-turn sign, and straight + left turn + U-turn sign. In this embodiment of the present invention, the straight sign, left turn sign, right turn sign, and U-turn sign are classified as single-modal signs, and the straight + left turn sign, straight + right turn sign, left turn + U-turn sign, and straight + left turn + U-turn sign are classified as multi-modal signs.

[0046] Step 2: According to the first driving sign A t Determine the trajectory prediction mode type and the corresponding trajectory prediction mode number X at the current moment;

[0047] Among them, X≥1; trajectory prediction modality types include single modality type and multi-modality type;

[0048] Specifically including: when the first driving sign A t When it is a single mode sign, set the trajectory prediction mode type to single mode type and set the corresponding trajectory prediction mode number X to 1; when the first driving sign A t When it belongs to the multi-modal sign, set the trajectory prediction mode type to multi-modal type and set it to the first driving sign A t When the sign is straight + left turn, straight + right turn or left turn + U-turn, set the corresponding trajectory prediction mode number X to 2, and t When the sign is straight ahead + left turn + U-turn, the corresponding trajectory prediction mode number X is set to 3.

[0049] Here, the trajectory prediction mode number X is used to identify the total number of possible driving states of the current obstacle vehicle in the future period; if the first driving sign A t Belong to the single mode sign description first driving sign A t There is only one possible driving state, and the corresponding trajectory prediction mode number X = 1; if the first driving sign A t Belong to multimodal signs Description First driving sign A t There are multiple possible driving states. At this time, you need to follow the first driving sign A t The specific number of possible driving states included in sets the number of trajectory prediction modes X.

[0050] Step 3: When the trajectory prediction modality type is a single modality type, call a preset first number N of single trajectory prediction models B i According to the first historical trajectory H t-1 and the first position p t Perform single trajectory prediction processing to generate the corresponding first predicted trajectory H i ; and the first predicted trajectory H of the first number Ni Perform trajectory fusion processing to generate the corresponding first fusion trajectory

[0051] Wherein, 1≤N, 1≤i≤N; the preset first number N is a positive integer greater than or equal to 1; the model index i is a positive integer, and its value range is: 1≤i≤N;

[0052] Specifically, step 31, when the trajectory prediction modality type is a single modality type, calling a preset first number N of single trajectory prediction models B i According to the first historical trajectory H t-1 and the first position p t Perform single trajectory prediction processing to generate the corresponding first predicted trajectory H i ;

[0053] Here, the preset first number N of single trajectory prediction models B i is a set of pre-trained trajectory prediction models for single-modal trajectory prediction. These single trajectory prediction models B i The trajectory prediction can be performed based on the historical trajectory and real-time position of the obstacle. These single trajectory prediction models B i The prediction models can be different in network structure or the prediction models can be the same in network structure but with different network parameters. When the trajectory prediction mode type is single mode, it means that the current obstacle vehicle has only one possible driving state in the future time period. In this case, the embodiment of the present invention calls N single trajectory prediction models B i Based on the first historical trajectory H t-1 and the first position p t Perform trajectory prediction separately to obtain N predicted trajectories, namely N first predicted trajectories H i ;

[0054] Step 32: For the first number N of first predicted trajectories H i Perform trajectory fusion processing to generate the corresponding first fusion trajectory

[0055] Specifically including: a first number N of first predicted trajectories H i Perform mean calculation to obtain the corresponding first fusion trajectory

[0056] Here, the embodiment of the present invention performs trajectory fusion in a mean manner to obtain the fused trajectory of the current obstacle vehicle at the current time t when the trajectory prediction modality type is a single modality type, that is, the first fused trajectory

[0057] Step 4: When the trajectory prediction modality type is a multi-modal type, call a preset second number M of multi-trajectory prediction models C j According to the first historical trajectory H t-1 , first position p t and the first driving sign A t Perform multi-trajectory prediction processing to generate the corresponding first predicted trajectory set G t,j ; and for the first predicted trajectory set G of the second number M t,j Perform trajectory fusion processing to generate the corresponding first fusion trajectory set

[0058] Among them, 1≤M, 1≤j≤M; the first predicted trajectory set G t,j The second predicted trajectory H including the number of trajectory prediction modalities X j,g And each second predicted trajectory H j,g Corresponding to a first predicted trajectory probability ρ j,g , 1≤g≤X; the preset second number M is a positive integer greater than or equal to 1; the model index j is a positive integer, and its value range is: 1≤j≤M; the predicted trajectory index g is a positive integer, and its value range is: 1≤g≤X;

[0059] Specifically, step 41 includes: when the trajectory prediction modality type is a multi-modal type, calling a preset second number M of multi-trajectory prediction models C j According to the first historical trajectory H t-1 , first position p t and the first driving sign A t Perform multi-trajectory prediction processing to generate the corresponding first predicted trajectory set G t,j ;

[0060] Here, the preset second number M of multi-trajectory prediction models C j is a set of pre-trained trajectory prediction models for multimodal trajectory prediction. These multi-trajectory prediction models C j The trajectory prediction can be performed based on the historical trajectory, real-time position and driving signs used to identify multiple possible driving states of the obstacle and the corresponding predicted trajectory set can be output. The number of trajectories in the predicted trajectory set is the same as the number of possible driving states identified by the input driving signs and each predicted trajectory in the set corresponds to a predicted trajectory probability. These multi-trajectory prediction models C j The prediction models can be different in network structure or the prediction models can be the same in network structure but with different network parameters. When the trajectory prediction modality type is multimodal, it means that the current obstacle vehicle has multiple possible driving states in the future time period. In this case, the embodiment of the present invention calls M multi-trajectory prediction models C j Based on the first historical trajectory H t-1, first position p t and the first driving sign A t Perform trajectory prediction separately to obtain M predicted trajectory sets, namely M first predicted trajectory sets G t,j , each first predicted trajectory set G t,j It includes X prediction trajectories, namely the second prediction trajectory H j,g , and each second predicted trajectory H j,g Corresponding to a predicted trajectory probability, that is, the first predicted trajectory probability ρ j,g , the same first predicted trajectory set G t,j The corresponding X first predicted trajectory probabilities ρ j,g The sum of is 1;

[0061] Step 42: For the second number M of first predicted trajectory sets G t,j Perform trajectory fusion processing to generate the corresponding first fusion trajectory set

[0062] Among them, the first fusion trajectory set The second fusion trajectory including the number of trajectory prediction modalities X And each second fusion trajectory Corresponding to a first fusion trajectory probability

[0063] Specifically comprising: step 421, the first predicted trajectory set G of the second number M t,j All second predicted trajectories H j,g are gathered together to form a corresponding first trajectory set;

[0064] The first trajectory set includes M*X second predicted trajectories H j,g ;

[0065] For example, it is known that the trajectory prediction modality type is a multi-modal type, the number of trajectory prediction modalities X=3, and the second number M=4; through step 41, four first prediction trajectory sets G are obtained. t,j They are: G t,1 , G t,2 , G t ,3 and G t,4 ; The first predicted trajectory set G t,1 Including 3 second prediction trajectories H j,g For {H 1,1 , H 1,2 , H 1,3}, the corresponding predicted trajectory probability is {ρ 1,1 , ρ 1,2 , ρ 1,3}; The first predicted trajectory set G t,2 Including 3 second prediction trajectories Hj,g For {H 2,1 , H 2,2 , H 2,3}, the corresponding predicted trajectory probability is {ρ 2,1 , ρ 2,2 , ρ 2,3}; The first predicted trajectory set G t,3 Including 3 second prediction trajectories H j,g For {H 3,1 , H 3,2 , H 3,3}, the corresponding predicted trajectory probability is {ρ 3,1 , ρ 3,2 , ρ 3,3}; The first predicted trajectory set G t,4 Including 3 second prediction trajectories H j,g For {H 4,1 , H 4,2 , H 4,3}, the corresponding predicted trajectory probability is {ρ 4,1 , ρ 4,2 , ρ 4,3};

[0066] Then, the first predicted trajectory set G t,1 , G t,2 , G t,3 and G t,4 All second predicted trajectories H j,g The first trajectory set is formed by combining them together to form {{H 1,1 , H 1,2 , H 1,3}, {H 2,1 , H 2,2 , H 2,3}, {H 3,1 , H 3,2 , H 3,3}, {H 4,1 , H 4,2 , H 4,3}};

[0067] Step 422: For the M*X second predicted trajectories H of the first trajectory set j,g Clustering is performed to obtain the first type of trajectory set with the number of trajectory prediction modes X;

[0068] Each first-category trajectory set includes M second-predicted trajectories H j,g , and corresponding to the M first predicted trajectory probabilities ρ j,g ;

[0069] Here, the embodiment of the present invention can implement clustering based on the K-means algorithm (K-means clustering); in simple terms, first, the M*X second predicted trajectories H of the first trajectory set are j,g Sampling of trajectory points is performed separately to obtain M*X predicted trajectory sampling point sets, and then projection points are set for all sampling points of all predicted trajectory sampling point sets based on the same coordinate system, and then the total classification parameter K=X of the K-means algorithm is set, and then all projection points are clustered based on the K-means algorithm to obtain X first-class projection point sets, and then the first M second predicted trajectories H with the largest number of projection points in each first-class projection point set are selected. j,g Form a corresponding first-class trajectory set, thereby obtaining X first-class trajectory sets; the K-means algorithm mentioned here is a public clustering algorithm, and the detailed implementation steps of the algorithm can be obtained from public technical literature, so it will not be further described here;

[0070] For example, it is known that the trajectory prediction modality type is a multi-modal type, the number of trajectory prediction modalities X=3, the second number M=4, and the first trajectory set is {{H 1,1 , H 1,2 , H 1,3}, {H 2,1 , H 2,2 , H 2,3}, {H 3,1 , H 3,2 , H 3,3}, {H 4,1 , H 4,2 , H 4,3}};

[0071] Then, after clustering in the current step, three first-class trajectory sets will be obtained. Here, the first-class trajectory set 1 is {H 1,1 ,H 2,1 ,H 3,1 ,H 4,1}, the first type of trajectory set 2 is {H 1,2 ,H 2,2 ,H 3,2 ,H 4,2}, the first type of trajectory set 3 is {H 1,3 ,H 2,3 ,H 3,3 ,H 4,3};

[0072] Step 423: For each of the M second predicted trajectories H of the first type trajectory set j,g Perform mean calculation to obtain the corresponding second fusion trajectory

[0073] Here, the embodiment of the present invention performs trajectory fusion on each cluster trajectory set, i.e., each first type trajectory set, in a mean manner to obtain a fused trajectory under the corresponding mode, i.e., a second fused trajectory.

[0074] For example, it is known that the trajectory prediction modality type is a multimodal type, the number of trajectory prediction modalities X=3, the second number M=4, and the three first-category trajectory sets obtained in step 422 are respectively:

[0075] The first type of trajectory set 1{H 1,1 ,H 2,1 ,H 3,1 ,H 4,1},

[0076] The first type of trajectory set 2{H 1,2 ,H 2,2 ,H 3,2 ,H 4,2},

[0077] The first type of trajectory set 3{H 1,3 ,H 2,3 ,H 3,3 ,H 4,3};

[0078] Then, after the calculation in step 423, the corresponding three second fusion trajectories can be obtained.

[0079]

[0080]

[0081]

[0082] Step 424: calculate the M first predicted trajectory probabilities ρ corresponding to each first type trajectory set. j,g Perform mean calculation to obtain the corresponding first mean probability And predict the first mean probability of the modal number X for the obtained trajectory Perform normalization calculation to obtain the corresponding first fusion trajectory probability

[0083] Among them, the probability of the first fusion trajectory With the second fusion trajectory One-to-one correspondence;

[0084] Here, the embodiment of the present invention performs trajectory probability fusion on all predicted trajectory probabilities corresponding to each cluster trajectory set, i.e., each first-class trajectory set, in a mean manner to obtain the mean probability under the corresponding mode, i.e., the first mean probability. And the first mean probability of each mode Normalize to get the fusion trajectory probability of the corresponding mode, that is, the first fusion trajectory probability

[0085] For example, it is known that the trajectory prediction modality type is a multi-modal type, the number of trajectory prediction modalities X=3, the second number M=4, and the three first-category trajectory sets obtained in the aforementioned step 422 are respectively:

[0086] The first type of trajectory set 1{H 1,1 ,H 2,1 ,H 3,1 ,H 4,1},

[0087] The first type of trajectory set 2{H 1,2 ,H 2,2 ,H 3,2 ,H 4,2},

[0088] The first type of trajectory set 3{H 1,3 ,H 2,3 ,H 3,3 ,H 4,3};

[0089] From the output of step 421, we can obtain the trajectory probability sets corresponding to the three first-category trajectory sets:

[0090] The trajectory probability set {ρ 1,1 ,ρ 2,1 ,ρ 3,1 ,ρ 4,1},

[0091] The trajectory probability set {ρ 1,2 ,ρ 2,2 ,ρ 3,2 ,ρ 4,2},

[0092] The trajectory probability set {ρ 1,3 ,ρ 2,3 ,ρ 3,3 ,ρ 4,3};

[0093] Then, after the calculation in the current step 424, the corresponding three first mean probabilities can be obtained.

[0094]

[0095]

[0096]

[0097] For the above three first mean probabilities The three first fusion trajectory probabilities obtained by normalization calculation for:

[0098]

[0099]

[0100]

[0101] Step 425, all second fusion trajectories Composed of the corresponding first fusion trajectory set

[0102] For example, if the trajectory prediction modality type is multimodal, the number of trajectory prediction modalities X=3, and the second number M=4, three second fusion trajectories are obtained from the above step 423. They are and Three second fusion trajectories are obtained from the above step 423 The corresponding three first fusion trajectory probabilities They are and Then, the first fusion trajectory set obtained in the current step is Should be The corresponding fusion trajectory probability set should be and and The sum of is 1.

[0103] Step 5: The first fusion trajectory is obtained based on the trajectory prediction modality type Or the first fusion trajectory set Integrate the predicted data to generate the corresponding predicted trajectory data output;

[0104] Specifically include: identifying the trajectory prediction mode type; if the trajectory prediction mode type is a single mode type, the first fusion trajectory As the corresponding predicted trajectory data output; if the trajectory prediction modality type is multimodal type, the first fused trajectory set obtained Each second fusion trajectory and the corresponding first fusion trajectory probability A corresponding first fused trajectory data set is formed, and the first fused trajectory data set of the obtained trajectory prediction modality number X constitutes the corresponding predicted trajectory data output.

[0105] Here, in the embodiment of the present invention, when the trajectory prediction modality type obtained this time is a single modality type, the corresponding first fusion trajectory is obtained through step 3. At this time, the embodiment of the present invention will As the final predicted trajectory data output; when the trajectory prediction modality type obtained this time is multimodal, the corresponding first fusion trajectory set will be obtained through step 4 At this time, the embodiment of the present invention sets the first fusion trajectory Each second fusion trajectory The probability of the first fusion trajectory corresponding to it is extracted A data group, namely a first fused trajectory data group, is formed, and then the corresponding predicted trajectory data output is formed by the obtained X first fused trajectory data groups.

[0106] Figure 2 This is a module structure diagram of a trajectory prediction processing device provided in the second embodiment of the present invention. The device is a terminal device or server that implements the aforementioned method embodiment, and can also be a device that enables the aforementioned terminal device or server to implement the aforementioned method embodiment. For example, the device can be a device or chip system of the aforementioned terminal device or server. Figure 2 As shown, the device includes: an acquisition module 201 , a preprocessing module 202 , a single-modal trajectory prediction module 203 , a multi-modal trajectory prediction module 204 and an output module 205 .

[0107] The acquisition module 201 is used to obtain the historical trajectory, real-time position and real-time driving traffic sign of the lane where the obstacle vehicle is located at the current time t to generate the corresponding first historical trajectory H t-1 , first position p t and the first driving sign A t ; First driving sign A t Includes unimodal and multimodal flags.

[0108] The pre-processing module 202 is used to t Determine the trajectory prediction modality type at the current moment and the corresponding trajectory prediction modality number X; X ≥ 1; the trajectory prediction modality type includes a single modality type and a multi-modality type.

[0109] The single-mode trajectory prediction module 203 is used to call a preset first number N of single trajectory prediction models B when the trajectory prediction modality type is a single-mode type. i According to the first historical trajectory H t-1 and the first position p t Perform single trajectory prediction processing to generate the corresponding first predicted trajectory H i ; and the first predicted trajectory H of the first number N iPerform trajectory fusion processing to generate the corresponding first fusion trajectory 1≤N, 1≤i≤N.

[0110] The multimodal trajectory prediction module 204 is configured to call a preset second number M of multi-trajectory prediction models C when the trajectory prediction modality type is a multimodal type. j According to the first historical trajectory H t-1 , first position p t and the first driving sign A t Perform multi-trajectory prediction processing to generate the corresponding first predicted trajectory set G t,j ; and for the first predicted trajectory set G of the second number M t,j Perform trajectory fusion processing to generate the corresponding first fusion trajectory set 1≤M, 1≤j≤M.

[0111] The output module 205 is used to obtain the first fusion trajectory according to the trajectory prediction modality type. Or the first fusion trajectory set The prediction data is integrated to generate the corresponding prediction trajectory data output.

[0112] An embodiment of the present invention provides a processing device for predicting trajectories, which can execute the method steps in the above method embodiment. Its implementation principles and technical effects are similar and will not be described in detail here.

[0113] It should be noted that it should be understood that the division of the various modules of the above device is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. Moreover, these modules can all be implemented in the form of software called by a processing element; or they can all be implemented in the form of hardware; or some modules can be implemented in the form of software called by a processing element, and some modules can be implemented in the form of hardware. For example, the acquisition module can be a separately established processing element, or it can be integrated into a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called by a processing element of the above device to perform the functions of the above determination module. The implementation of other modules is similar. In addition, these modules can all or partly be integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed by the hardware integrated logic circuit in the processor element or by instructions in the form of software.

[0114] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented by scheduling program code through a processing element, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0115] In the above embodiments, all or part of the embodiments may be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the above method embodiments are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The above-mentioned computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the above-mentioned computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, Bluetooth, microwave, etc.) means. The above-mentioned computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The above-mentioned available medium can be a magnetic medium (such as a floppy disk, hard disk, tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0116] Figure 3 This is a schematic diagram of the structure of an electronic device provided in the third embodiment of the present invention. The electronic device may be the aforementioned terminal device or server, or may be a terminal device or server connected to the aforementioned terminal device or server to implement the method of the embodiment of the present invention. Figure 3As shown, the electronic device may include: a processor 301 (such as a CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transceiver 303's transceiver actions. Various instructions may be stored in the memory 302 for completing various processing functions and implementing the processing steps described in the aforementioned method embodiment. Preferably, the electronic device involved in the embodiment of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to realize communication connections between components. The above-mentioned communication port 306 is used for connection and communication between the electronic device and other peripherals.

[0117] exist Figure 3 The system bus 305 mentioned in the figure can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used to realize communication between the database access device and other devices (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory (Non-Volatile Memory), such as at least one disk storage.

[0118] The above-mentioned processors can be general-purpose processors, including central processing units (CPUs), network processors (NPs), graphics processing units (GPUs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0119] It should be noted that an embodiment of the present invention further provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, it enables the computer to execute the methods and processing procedures provided in the above embodiments.

[0120] An embodiment of the present invention further provides a chip for executing instructions, which is used to execute the processing steps described in the above method embodiment.

[0121] The embodiment of the present invention provides a method, device, electronic device and computer-readable storage medium for processing predicted trajectories; pre-integrates multiple single trajectory prediction models for single-modal trajectory prediction and multiple multi-trajectory prediction models for multi-modal trajectory prediction; when performing trajectory prediction, first confirms the trajectory prediction modality type (single-modal type and multi-modal type) and the number of trajectory prediction modalities of the current obstacle vehicle; and when the trajectory prediction modality type is single-modal type, calls multiple preset single trajectory prediction models to perform single-modal trajectory prediction based on the historical trajectory and real-time position of the current obstacle vehicle at the current moment, and performs trajectory fusion on the obtained multiple predicted trajectories in a mean manner to obtain a final fusion Trajectory, and the fused trajectory is output as the final predicted trajectory data; and when the trajectory prediction modality type is multimodal, multiple preset multi-trajectory prediction models are called to perform multimodal trajectory prediction based on the historical trajectory, real-time position and multiple possible driving states of the current obstacle vehicle at the current moment, and all the predicted trajectories obtained are clustered according to the number of trajectory prediction modalities, and each clustered trajectory set is subjected to trajectory fusion and trajectory probability fusion in a mean manner to obtain the fused trajectory and mean probability of the corresponding modality, and the mean probability of each modality is normalized to obtain the fused trajectory probability of the corresponding modality, and the fused trajectory and fused trajectory probability of each modality are combined to form the final predicted trajectory data output. Through the present invention, based on the subdivided modality, the corresponding multiple single / multi-trajectory prediction models are used for prediction and trajectory fusion, which overcomes the problem of predicted trajectory error fluctuation caused by using only a single trajectory prediction model for trajectory prediction in conventional technical solutions, and improves prediction stability and prediction accuracy.

[0122] Professionals should also be further aware that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0123] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0124] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for processing predicted trajectories, characterized in that: The method comprises: Obtain the historical trajectory, real-time position and real-time traffic sign of the lane where the obstacle vehicle is located at the current time t to generate the corresponding first historical trajectory H t-1 , first position p t and the first driving sign A t ; The first driving sign A t Includes unimodal and multimodal signs; According to the first driving sign A t Determine the trajectory prediction modality type at the current moment and the corresponding trajectory prediction modality number X; X≥1; the trajectory prediction modality type includes a single modality type and a multi-modality type; When the trajectory prediction modality type is a single modality type, a preset first number N of single trajectory prediction models B are called. i According to the first historical trajectory H t-1 and the first position p t Perform single trajectory prediction processing to generate the corresponding first predicted trajectory H i ; and the first predicted trajectory H of the first number N i Perform trajectory fusion processing to generate the corresponding first fusion trajectory 1≤N, 1≤i≤N; When the trajectory prediction modality type is a multi-modal type, a preset second number M of multi-trajectory prediction models C are called. j According to the first historical trajectory H t-1 , the first position p t and the first driving sign A t Perform multi-trajectory prediction processing to generate the corresponding first predicted trajectory set G t,j ; and the first predicted trajectory set G of the second number M t,j Perform trajectory fusion processing to generate the corresponding first fusion trajectory set 1≤M, 1≤j≤M; The first fusion trajectory obtained according to the trajectory prediction modality type Or the first fusion trajectory set The prediction data is integrated to generate the corresponding prediction trajectory data output.

2. The method for processing predicted trajectories according to claim 1, characterized in that: The single-mode signs include a straight ahead sign, a left turn sign, a right turn sign, and a U-turn sign; The multimodal sign includes a straight-ahead + left-turn sign, a straight-ahead + right-turn sign, a left-turn + U-turn sign, and a straight-ahead + left-turn + U-turn sign; The first predicted trajectory set G t,j The second predicted trajectory H including the trajectory prediction modality number X j,g And each of the second predicted trajectories H j,g Corresponding to a first predicted trajectory probability ρ j,g , 1≤g≤X; The first fusion trajectory set The second fused trajectory including the number X of trajectory prediction modalities And each of the second fusion trajectories Corresponding to the probability of a first fusion trajectory 3. The method for processing predicted trajectories according to claim 2, characterized in that: According to the first driving sign A t Determine the trajectory prediction mode type and the corresponding trajectory prediction mode number X at the current moment, specifically including: When the first driving sign A t When it belongs to the single mode flag, the trajectory prediction mode type is set to the single mode type, and the corresponding trajectory prediction mode number X is set to 1; When the first driving sign A t When the multi-modal flag is set, the trajectory prediction mode type is set to multi-modal type, and the first driving flag A t When the sign is straight + left turn, straight + right turn or left turn + U-turn, the corresponding trajectory prediction mode number X is set to 2, and the first driving sign A t When the sign is straight ahead + left turn + U-turn, the corresponding trajectory prediction mode number X is set to 3.

4. The method for processing predicted trajectories according to claim 2, wherein: The first predicted trajectory H for the first number N i Perform trajectory fusion processing to generate the corresponding first fusion trajectory Specifically include: The first predicted trajectory H of the first number N i Perform mean calculation to obtain the corresponding first fusion trajectory 5. The method for processing predicted trajectories according to claim 2, characterized in that: The first predicted trajectory set G for the second number M t,j Perform trajectory fusion processing to generate the corresponding first fusion trajectory set Specifically include: The first predicted trajectory set G of the second number M t,j All the second predicted trajectories H j,g The first trajectory set includes M*X second predicted trajectories H j,g ; For the M*X second predicted trajectories H of the first trajectory set j,g Clustering is performed to obtain the first type trajectory set of the trajectory prediction modality number X; each of the first type trajectory sets includes M second prediction trajectories H j,g , and corresponding to the M first predicted trajectory probabilities ρ j,g ; For each of the M second predicted trajectories H of the first type trajectory set j,g Perform mean calculation to obtain the corresponding second fusion trajectory The M first predicted trajectory probabilities ρ corresponding to each of the first type trajectory sets j,g Perform mean calculation to obtain the corresponding first mean probability And predict the first mean probability of the modal number X of the obtained trajectory Perform normalization calculation to obtain the corresponding first fusion trajectory probability The first fusion trajectory probability Fusion trajectory with the second One-to-one correspondence; The second fusion trajectory of all The first fusion trajectory set corresponding to 6. The method for processing predicted trajectories according to claim 5, characterized in that: The first fusion trajectory obtained according to the trajectory prediction modality type Or the first fusion trajectory set Integrate the predicted data to generate the corresponding predicted trajectory data output, including: Identify the trajectory prediction modality type; if the trajectory prediction modality type is a single modality type, the first fusion trajectory obtained As the corresponding predicted trajectory data output; if the trajectory prediction modality type is multimodal type, the first fusion trajectory set obtained Each of the second fusion trajectories and the corresponding probability of the first fusion trajectory A corresponding first fused trajectory data group is formed, and the first fused trajectory data group of the obtained trajectory prediction modality number X constitutes the corresponding predicted trajectory data output.

7. A device for executing the method for processing predicted trajectories according to any one of claims 1 to 6, characterized in that: The device includes: an acquisition module, a preprocessing module, a single-modal trajectory prediction module, a multi-modal trajectory prediction module and an output module; The acquisition module is used to obtain the historical trajectory, real-time position and real-time traffic sign of the lane where the obstacle vehicle is located at the current time t to generate the corresponding first historical trajectory H t-1 , first position p t and the first driving sign A t ; The first driving sign A t Includes unimodal and multimodal signs; The pre-processing module is used to t Determine the trajectory prediction modality type at the current moment and the corresponding trajectory prediction modality number X; X≥1; the trajectory prediction modality type includes a single modality type and a multi-modality type; The single-mode trajectory prediction module is used to call a preset first number N of single trajectory prediction models B when the trajectory prediction modality type is a single-mode type. i According to the first historical trajectory H t-1 and the first position p t Perform single trajectory prediction processing to generate the corresponding first predicted trajectory H i ; and the first predicted trajectory H of the first number N i Perform trajectory fusion processing to generate the corresponding first fusion trajectory 1≤N, 1≤i≤N; The multimodal trajectory prediction module is configured to call a preset second number M of multi-trajectory prediction models C when the trajectory prediction modality type is a multimodal type. j According to the first historical trajectory H t-1 , the first position p t and the first driving sign A t Perform multi-trajectory prediction processing to generate the corresponding first predicted trajectory set G t,j ; and the first predicted trajectory set G of the second number M t,j Perform trajectory fusion processing to generate the corresponding first fusion trajectory set 1≤M, 1≤j≤M; The output module is used to obtain the first fusion trajectory according to the trajectory prediction modality type. Or the first fusion trajectory set The prediction data is integrated to generate the corresponding prediction trajectory data output.

8. An electronic device, characterized in that: include: memory, processors, and transceivers; The processor is configured to be coupled to the memory, read and execute instructions in the memory, so as to implement the method steps described in any one of claims 1 to 6; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, which, when executed by a computer, enable the computer to execute the method according to any one of claims 1 to 6.

Images