Method, device, equipment and storage medium for predicting vehicle lane change

Abstract

The present application discloses a method, device, equipment and storage medium for predicting vehicle lane changes, wherein the method includes: collecting information on multiple target vehicles and roads around the vehicle to obtain an original image sequence; processing the original image sequence to obtain the trajectory of each target vehicle and the local map corresponding to each frame of the original image; processing the trajectory of each target vehicle and the local map corresponding to each frame of the original image based on the first processing network based on the spatial dimension and the time dimension respectively to obtain a spatiotemporal graph structure; processing the spatiotemporal graph structure based on the second processing network to obtain a feature embedding vector; based on the feature embedding vector, predicting the lane change intention of each target vehicle and obtaining a prediction result. The technical solution of the present application constructs a spatiotemporal graph structure to improve the accuracy of predicting vehicle lane changes.

Description

Technical Field

[0001] The present application belongs to the field of autonomous driving technology, and in particular, relates to a method, device, equipment and storage medium for predicting vehicle lane changes. Background Art

[0002] At present, in the field of autonomous driving technology, inaccurate predictions of the vehicle's lane-changing intentions of surrounding vehicles often bring about safety risks in autonomous driving and reduce user satisfaction.

[0003] In order to solve the above problems, some current technical solutions predict the lane-changing intention of surrounding vehicles based on the speed of the surrounding vehicles. However, this prediction method has poor prediction accuracy and low user satisfaction. Summary of the invention

[0004] The present application aims to solve one of the technical problems in the related art at least to a certain extent. To this end, one purpose of the present application is to provide a method, device, equipment and storage medium for predicting vehicle lane change.

[0005] In order to solve the above technical problems, the embodiments of the present application provide the following technical solutions:

[0006] A method for predicting a vehicle lane change, comprising:

[0007] Collect information about multiple target vehicles and roads around the vehicle to obtain an original image sequence; process the original image sequence to obtain the trajectory of each target vehicle and a local map corresponding to each frame of the original image;

[0008] Based on the first processing network, the trajectory of each target vehicle and the local map corresponding to each frame of the original image are processed based on the spatial dimension and the time dimension respectively to obtain a spatiotemporal graph structure; based on the second processing network, the spatiotemporal graph structure is processed to obtain a feature embedding vector;

[0009] Based on the feature embedding vector, the lane changing intention of each target vehicle is predicted, and a prediction result is obtained.

[0010] Optionally, the processing of the original image sequence to obtain the trajectory of each target vehicle and a local map corresponding to each frame of the original image includes:

[0011] Performing feature extraction on the original image sequence to obtain a feature vector of each target vehicle and a local map vector corresponding to each frame of the original image;

[0012] Processing the feature vector of each target vehicle and each local map vector to obtain the initial state of each target vehicle and the initial state of each local map; wherein the initial state of each target vehicle includes the initial category, initial coordinates, initial size and initial angle of each target vehicle; the initial state of each local map includes the coordinates of the starting point and the coordinates of the ending point of each local map;

[0013] Based on the initial state of each of the target vehicles, the actual predicted state of each of the target vehicles is obtained; based on the actual predicted state of each of the target vehicles at J moments, the trajectory of each of the target vehicles is obtained; based on the initial state of each of the local maps, the corresponding local map is obtained; wherein J is a positive integer.

[0014] Optionally, obtaining the actual predicted state of each target vehicle based on the initial state of each target vehicle includes:

[0015] Based on the initial state of each target vehicle at the jth moment, obtaining the initial predicted state of each target vehicle at the j+1th moment;

[0016] Based on the initial state of each target vehicle at the jth moment, obtain the initial rectangular frame of each target vehicle at the jth moment; based on the initial predicted state of each target vehicle at the j+1th moment, obtain the predicted rectangular frame of each target vehicle at the j+1th moment; wherein 1≤j≤J, and j is an integer;

[0017] Based on the initial rectangular frame of each target vehicle at the jth moment and the predicted rectangular frame at the j+1th moment, a cost matrix of each target vehicle at the j+1th moment is calculated;

[0018] Based on the cost matrix of each target vehicle at the j+1th moment, the actual predicted state of each target vehicle at the j+1th moment is obtained; the actual predicted state includes: actual predicted category, actual predicted coordinates, actual predicted size and actual predicted angle.

[0019] Optionally, the first processing network processes the trajectory of each target vehicle and the local map corresponding to each frame of the original image based on the spatial dimension and the temporal dimension respectively to obtain a spatiotemporal graph structure; and the second processing network processes the spatiotemporal graph structure to obtain a feature embedding vector, including:

[0020] The first processing network obtains the trajectory of each target vehicle and the local map corresponding to each frame of the original image;

[0021] In the spatial dimension, the first processing network constructs a spatial graph structure based on the plurality of local maps and the trajectories of the plurality of target vehicles;

[0022] In the time dimension, the first processing network constructs a time graph structure based on one of the target vehicles or one of the local maps;

[0023] The first processing network fuses each of the spatial graph structures and each of the temporal graph structures to obtain a spatiotemporal graph structure;

[0024] Based on the second processing network, a first convolution processing is performed on the spatiotemporal graph structure to obtain the feature embedding vector.

[0025] Optionally, in the spatial dimension, the first processing network constructs a spatial graph structure based on the multiple local maps and the trajectories of the multiple target vehicles, including:

[0026] The first processing network constructs a first spatial graph structure, a second spatial graph structure, and a third spatial graph structure based on the plurality of local maps and the trajectories of the plurality of target vehicles;

[0027] Wherein, the first processing network uses the Mth target vehicle and the Nth target vehicle as two first spatial vertices of a first spatial edge to obtain a first spatial graph structure; each of the first spatial edges includes a first spatial weight; M≠N, and M and N are both positive integers;

[0028] The first processing network uses the Oth target vehicle and the Pth local map as two second spatial vertices of a second spatial edge to obtain a second spatial graph structure; each of the second spatial edges includes a second spatial weight; O and P are both positive integers;

[0029] The first processing network uses the Qth local map and the Rth local map as two third space vertices of a third space edge to obtain a third space graph structure; each third space edge includes a third space weight; Q≠R, and Q and R are both positive integers.

[0030] Optionally, in the time dimension, the first processing network constructs a time graph structure based on one of the target vehicles or one of the local maps, including:

[0031] The first processing network uses the Sth target vehicle as a first time vertex, and connects every two first time vertices through a first time edge to obtain a first time graph structure; wherein each first time edge includes a first time weight; S is a positive integer;

[0032] The first processing network uses the Wth local map as a second time vertex, and connects every two second time vertices through a second time edge to obtain a second time graph structure; wherein each second time edge includes a second time weight; and W is a positive integer.

[0033] Optionally, predicting the lane change intention of each target vehicle based on the feature embedding vector and obtaining a prediction result includes:

[0034] Performing a second convolution process on the feature embedding vector to obtain the target vector;

[0035] Processing the target vector to obtain multiple intention probabilities;

[0036] Comparing the plurality of intention probabilities to obtain a target intention probability;

[0037] Based on the target intention probability, the prediction result is obtained.

[0038] The embodiment of the present application also provides a device for predicting vehicle lane change, comprising:

[0039] The acquisition module is used to collect information about multiple target vehicles and roads around the vehicle and obtain original image sequences;

[0040] A first processing module is used to process the original image sequence to obtain the trajectory of each target vehicle and a local map corresponding to each frame of the original image;

[0041] A second processing module is used to process the trajectory of each target vehicle and the local map corresponding to each frame of the original image based on the first processing network, based on the spatial dimension and the time dimension, respectively, to obtain a spatiotemporal graph structure; and to process the spatiotemporal graph structure based on the second processing network to obtain a feature embedding vector;

[0042] The prediction module is used to predict the lane changing intention of each target vehicle based on the feature embedding vector and obtain a prediction result.

[0043] An embodiment of the present application further provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the method as described above when executing the computer program.

[0044] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method described above.

[0045] The embodiments of the present application have the following technical effects:

[0046] The above-mentioned technical scheme of the present application, 1) collects information of multiple target vehicles around the own vehicle and obtains a sequence of original images, processes the original images to obtain the trajectory of each target vehicle and the local map of each frame of the original image, processes the trajectory of each target vehicle and the local map of each frame of the original image in the spatial dimension based on the first processing network to obtain a spatial map result; processes the trajectory of each target vehicle and the local map of each frame of the original image in the time dimension based on the first processing network to obtain a time map structure; fuses the time map structure and the space map structure to obtain a space-time map structure; performs convolution processing on the space-time map structure based on the second processing network to obtain a prediction result of the lane changing intention of each target vehicle around the own vehicle. The method is simple and has high prediction accuracy.

[0047] 2) Based on the trajectory of each target vehicle and the local map corresponding to each frame of the original image, a spatial graph structure is constructed in the spatial dimension, and a temporal graph structure is constructed in the temporal dimension. This can effectively construct the spatial relationship between target vehicles, target vehicles and local maps, and local maps and local maps. The contextual relationship of the target vehicles is also constructed in the temporal dimension. At the same time, the attention mechanism information is introduced to construct the spatiotemporal graph structure more accurately and effectively, thereby improving the accuracy of predicting vehicle lane changes.

[0048] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a structural schematic diagram of a vehicle lane change prediction system provided in an embodiment of the present application;

[0050] Figure 2 It is a flowchart of a method for predicting vehicle lane change provided by an embodiment of the present application;

[0051] Figure 3 is a schematic diagram of a space-time graph structure provided in an embodiment of the present application;

[0052] Figure 4 It is a schematic diagram of the working principle of a graph convolutional neural network provided in an embodiment of the present application;

[0053] Figure 5is a schematic diagram of a prediction result provided in an embodiment of the present application;

[0054] Figure 6 It is a structural schematic diagram of a device for predicting vehicle lane change provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0056] In order to facilitate the understanding of the embodiments by those skilled in the art, some terms are explained:

[0057] (1) EfficientNet: A deep network structure based on convolutional neural networks.

[0058] (2) Transformer: Transformer, including encoder and decoder.

[0059] (3) BEV: Bird’s Eye View.

[0060] (4)softmax: A mathematical function used to normalize values ​​between 0 and 1.

[0061] like Figure 1 As shown, an embodiment of the present application provides a system for predicting vehicle lane change, comprising:

[0062] A processing unit, the processing unit is used to predict lane-changing intentions of multiple target vehicles around the vehicle and output prediction results;

[0063] Camera; the camera and the processing unit exchange data based on the network; wherein the camera includes six cameras, which are respectively installed at the front, left front, right front, rear, left rear and right rear directions of the vehicle, and are used to collect information of multiple target vehicles and roads in multiple directions around the vehicle, and obtain an original image sequence, specifically, the original image sequence includes multiple frames of original images; after obtaining the original image sequence, the camera transmits the original image sequence to the processing unit based on the network, so as to realize processing of the original image sequence based on the processing unit and obtain processing results, and then the processing unit predicts the lane change intentions of multiple target vehicles around the vehicle based on the processing results, and outputs the prediction results;

[0064] Human-computer interaction unit; the human-computer interaction unit and the processing unit perform data exchange based on the network; the human-computer interaction unit can be implemented based on a smart phone or a smart terminal installed in the vehicle; the human-computer interaction unit is used to receive the prediction results sent by the processing unit based on the network, and can display the prediction results to prompt the user about the lane change intentions of multiple target vehicles around the vehicle;

[0065] Furthermore, after obtaining the lane-changing intentions of multiple target vehicles around the user's vehicle, the user can promptly adjust his or her sitting posture or other responses that make the user safer in the current driving environment.

[0066] Furthermore, after obtaining the lane-changing intentions of multiple target vehicles around the vehicle, the processing unit can generate corresponding decision commands based on the lane-changing intentions of multiple target vehicles around the vehicle, and send the decision commands to the control system of the vehicle, so as to adjust the driving strategy of the vehicle according to the lane-changing intentions of multiple target vehicles around the vehicle, thereby further ensuring the safety of the user.

[0067] In an optional embodiment of the present application, the processing unit is connected to a power supply, and the power supply is used to provide power to the processing unit to ensure the normal operation of the processing unit.

[0068] In an optional embodiment of the present application, the memory and the processing unit perform two-way data exchange based on a network. Specifically, the memory is used to store the predicted vehicle lane change unit and the original image sequence; the processing unit can call the predicted vehicle lane change unit in the memory to process the original image sequence, so as to predict the lane change intentions of multiple target vehicles around the vehicle and output the prediction results.

[0069] In an embodiment of the present application, the vehicle obtains driving information of multiple target vehicles around the vehicle based on the installed six-way cameras, generates an original image sequence based on the driving information, and then sends the original image sequence to a processing unit. After obtaining the original image sequence, the processing unit calls a vehicle lane change prediction unit to process the original image sequence and predicts the lane change intentions of multiple target vehicles around the vehicle. Finally, the processing unit controls the vehicle based on the predicted lane change intentions of multiple target vehicles around the vehicle, thereby improving the safety of the vehicle driving and improving user satisfaction.

[0070] like Figure 2 As shown, the embodiment of the present application provides a method for predicting vehicle lane change, which is applied to Figure 1 The predictive vehicle lane change system shown includes:

[0071] Step S21: collecting information about multiple target vehicles and roads around the vehicle to obtain an original image sequence;

[0072] In the embodiment of the present application, six cameras arranged on the vehicle are used to collect information of multiple target vehicles, lane lines and road edges around the vehicle, and obtain an original image sequence including multiple target vehicles, lane lines and road edges;

[0073] The original image sequence may include multiple frames of original images.

[0074] The embodiments of the present application expand the range of the vehicle's perception of the surrounding environment and target vehicles based on multi-view detection technology (six-way cameras), achieve 360-degree all-round perception, provide more effective information for predicting vehicle lane changes, and help improve the prediction accuracy of vehicle lane changes.

[0075] Step S22: Processing the original image sequence to obtain the trajectory of each target vehicle and the local map corresponding to each frame of the original image;

[0076] In an optional embodiment of the present application, the processing of the original image sequence to obtain the trajectory of each target vehicle and the local map corresponding to each frame of the original image includes:

[0077] Performing feature extraction on the original image sequence to obtain a feature vector of each target vehicle and a local map vector corresponding to each frame of the original image;

[0078] Processing the feature vector of each target vehicle and each local map vector to obtain the initial state of each target vehicle and the initial state of each local map; wherein the initial state of each target vehicle includes the initial category, initial coordinates, initial size and initial angle of each target vehicle; the initial state of each local map includes the coordinates of the starting point and the coordinates of the ending point of each local map;

[0079] The initial state of each target vehicle is updated to obtain the actual predicted state of each target vehicle; based on the actual predicted state of each target vehicle at J moments, the trajectory of each target vehicle is obtained; based on the initial state of each local map, the corresponding local map is obtained; wherein J is a positive integer.

[0080] In the embodiment of the present application, after obtaining the original image sequence, feature extraction can be performed on the original image sequence based on EfficientNet to obtain a feature vector of each target vehicle involved in the original image sequence and a local map vector corresponding to the local map involved in each frame of the original image; wherein the feature vector and the local map vector are both two-dimensional vectors;

[0081] After obtaining the feature vector of each target vehicle and each local map vector, the view is converted into a network; specifically, each feature vector and each local map vector are processed using a Transformer in combination with the internal and external parameters of the camera to obtain the initial state of each target vehicle and the initial state of each local map;

[0082] Among them, the initial state of each target vehicle and the initial state of each local map are also represented based on a two-dimensional vector and correspond to the BEV perspective.

[0083] Specifically, the initial state of each target vehicle includes: initial category (class), center point coordinates (x, y), size (w, h) and angle (angle);

[0084] The initial state of each of the local maps includes: the coordinates of the starting point, the coordinates of the ending point and the curvature information of the lane line; and the coordinates of the starting point, the coordinates of the ending point and the curvature information of the road edge.

[0085] In an optional embodiment of the present application, obtaining the actual predicted state of each target vehicle based on the initial state of each target vehicle includes:

[0086] Based on the initial state of each target vehicle at the jth moment, obtaining the initial predicted state of each target vehicle at the j+1th moment;

[0087] Based on the initial state of each target vehicle at the jth moment, obtain the initial rectangular frame of each target vehicle at the jth moment; based on the initial predicted state of each target vehicle at the j+1th moment, obtain the predicted rectangular frame of each target vehicle at the j+1th moment; wherein 1≤j≤J, and j is an integer;

[0088] Based on the initial rectangular frame of each target vehicle at the jth moment and the predicted rectangular frame at the j+1th moment, a cost matrix of each target vehicle at the j+1th moment is calculated;

[0089] Based on the cost matrix of each target vehicle at the j+1th moment, the actual predicted state of each target vehicle at the j+1th moment is obtained; the actual predicted state includes: actual predicted category, actual predicted coordinates, actual predicted size and actual predicted angle.

[0090] In an embodiment of the present application, the actual predicted state of each target vehicle is predicted based on the initial state of each target vehicle through Kalman filtering;

[0091] The initial state of a target vehicle may be a position in a frame of original image corresponding to the jth moment. Specifically, an initial rectangular frame of the target vehicle may be obtained, and a position of the target vehicle in a frame of original image corresponding to the j+1th moment may be predicted, that is, the position corresponds to a predicted rectangular frame.

[0092] Specifically, based on the initial rectangular box corresponding to each target vehicle and the intersection-over-union (IOU) between the predicted rectangular boxes, the allocation cost is calculated, and the cost matrix corresponding to each target vehicle is obtained;

[0093] Preset the cost matrix threshold, calculate the cost matrix value corresponding to each target vehicle, compare the value of each cost matrix with the cost matrix threshold, and delete the cost matrix if the cost matrix value is less than the cost matrix threshold. This means that the matching cost of the initial rectangular box and the predicted rectangular box is too high or the matching degree is low.

[0094] It should be noted that when calculating the intersection and union ratio, when matching a certain initial rectangular box with a predicted rectangular box, the matching is performed randomly, and the matching cost of each group of randomly matched initial rectangular boxes and predicted rectangular boxes is calculated based on the value of the cost matrix according to the above Hungarian algorithm;

[0095] When the value of the cost matrix is ​​greater than or equal to the cost matrix threshold, it indicates that the matching cost of the group of randomly matched initial rectangular boxes and the predicted rectangular boxes is small, that is, the matching degree is high, and the initial rectangular box is associated with the predicted rectangular box, that is, the target vehicle in a frame of the original image corresponding to the jth moment is associated with the target vehicle in a frame of the original image corresponding to the j+1th moment.

[0096] Furthermore, the above process can combine the initial state of each target vehicle with the actual predicted state based on the Kalman filter to make an optimal estimate of the system state, thereby achieving:

[0097] Specifically, it can be implemented based on the following formula:

[0098] Prediction equation:

[0099] Vector of initial predicted states:

[0100] P j+1 =AP j A T ;

[0101] Update equation: G j+1 =P j+1 C T (CP j+1 C T +R)-1 ;

[0102]

[0103] P′ j+1 =(1-G j+1 C)P j+1

[0104] Among them, A is the state transfer matrix; B is the control input matrix; is the vector of the initial state; u j is the driving input vector; P j+1 is the covariance matrix; G j+1 is the Kalman gain; R is the variance of the measurement noise; z j+1 is the initial prediction result; C is the measurement vector; is the vector of the actual predicted state; P′ j is the actual covariance matrix;

[0105] It should be noted that the state transfer matrix A; the control input matrix B; the drive input vector u j+1 ; Covariance matrix P j+1 ; Kalman gain G j+1 ; Variance of measurement noise R; Initial prediction result z j+1 ; The measurement vector C can be obtained based on system presets or manual input.

[0106] The above process finally outputs the actual prediction status, including: the actual prediction category (clss 实际 ), actual predicted coordinates (x 实际 ,y 实际 ), actual predicted size (w 实际 ,h 实际 ) and the actual predicted angle (angle 实际 ).

[0107] Among them, the initial category and the actual predicted category are used to represent the rectangular boxes corresponding to each target vehicle at each time, so as to ensure that the front and rear are the same vehicle.

[0108] The actual predicted coordinates (x 实际 ,y 实际 ) is determined based on the coordinates of the center point of the rectangular box corresponding to the actual predicted state; the actual predicted angle (angle 实际 ) is determined based on the actual predicted coordinates.

[0109] Actual predicted size (w 实际 ,h 实际 ) and the actual predicted category (clss 实际), which are determined based on the initial size and initial category respectively. Because the initial size and initial category generally do not change or change significantly with the driving of the target vehicle, they are rigid attributes of the target vehicle and can be basically determined based on the information collected by the six-way cameras.

[0110] In the embodiment of the present application, the vector of the actual predicted state has a higher matching degree with the vector of the initial state and a lower matching cost than the vector of the predicted state. Therefore, the above algorithm and calculation process are equivalent to correcting the predicted state, and the actual predicted state obtained has a more accurate prediction result.

[0111] Step S23: Based on the first processing network, the trajectory of each target vehicle and the local map corresponding to each frame of the original image are processed based on the spatial dimension and the time dimension respectively to obtain a spatiotemporal graph structure; based on the second processing network, the spatiotemporal graph structure is processed to obtain a feature embedding vector;

[0112] In an optional embodiment of the present application, the first processing network processes the trajectory of each target vehicle and the local map corresponding to each frame of the original image based on the spatial dimension and the time dimension respectively to obtain a spatiotemporal graph structure; and the second processing network processes the spatiotemporal graph structure to obtain a feature embedding vector, including:

[0113] The first processing network obtains the trajectory of each target vehicle and the local map corresponding to each frame of the original image;

[0114] In the spatial dimension, the first processing network constructs a spatial graph structure based on the plurality of local maps and the trajectories of the plurality of target vehicles;

[0115] In the time dimension, the first processing network constructs a time graph structure based on one of the target vehicles or one of the local maps;

[0116] The first processing network fuses each of the spatial graph structures and each of the temporal graph structures to obtain a spatiotemporal graph structure;

[0117] Based on the second processing network, a first convolution processing is performed on the spatiotemporal graph structure to obtain the feature embedding vector.

[0118] In an optional embodiment of the present application, based on the actual predicted coordinates (x 实际 ,y 实际 ) and arrange them in time sequence to obtain the trajectory; among them, each actual predicted coordinate (x 实际 ,y 实际 ), is the coordinate of the center point of the actual prediction rectangle corresponding to the corresponding actual prediction state.

[0119] Furthermore, the cubic curve equation is preset: y = ax 3 +bx 2 +cx+e, where a, b, c, e are coefficients and can be preset according to actual needs;

[0120] Assume a local map: the starting point of the lane line (x 始 ,y 始 ), the end point of the lane line ((x 止 ,y 止 ));

[0121] Then y 始 =ax 始 3 +bx 始 2 +cx 始 +e;

[0122] y 止 =ax 止 3 +bx 止 2 +cx 止 +e;

[0123] Based on the above two formulas, the complete lane lines corresponding to each local map can be depicted;

[0124] Similarly, assuming a local map: the starting point of the road edge line (x 始 ,y 始 ), the end point of the road edge line ((x 止 ,y 止 ));

[0125] Then y 始 =ax 始 3 +bx 始 2 +cx 始 +e;

[0126] y 止 =ax 止 3 +bx 止 2 +cx 止 +e;

[0127] Based on the above two formulas, the complete road edge line corresponding to each local map can be depicted. Based on the corresponding complete lane line and the complete road edge line, a complete local map can be obtained.

[0128] By analogy, multiple complete local maps can be obtained; that is, a complete local map can be obtained at each moment, and finally J complete local maps can be obtained;

[0129] It should be noted that, in the embodiment of the present application, the lane-changing intentions of multiple target vehicles around the vehicle are continuously predicted during the operation of the vehicle. Therefore, the actual value of J can be determined according to the system preset, or it can be adjusted based on actual needs. The embodiment of the present application does not make specific limitations on this.

[0130] The obtained multiple local maps and multiple trajectories are input into the first processing network and the second processing network respectively.

[0131] In the embodiment of the present application, the predicted state obtained based on the initial state is corrected based on the Kalman filter and the Hungarian algorithm to obtain the actual predicted state of each target vehicle at the next moment of the current moment, thereby improving the accuracy of predicting vehicle lane changes.

[0132] In an optional embodiment of the present application, in the spatial dimension, the first processing network constructs a spatial graph structure based on the multiple local maps and the trajectories of the multiple target vehicles, including:

[0133] The first processing network constructs a first spatial graph structure, a second spatial graph structure, and a third spatial graph structure based on the plurality of local maps and the trajectories of the plurality of target vehicles;

[0134] Wherein, the first processing network uses the Mth target vehicle and the Nth target vehicle as two first spatial vertices of a first spatial edge to obtain a first spatial graph structure; each of the first spatial edges includes a first spatial weight; M≠N, and M and N are both positive integers;

[0135] The first processing network uses the Oth target vehicle and the Pth local map as two second spatial vertices of a second spatial edge to obtain a second spatial graph structure; each of the second spatial edges includes a second spatial weight; O and P are both positive integers;

[0136] The first processing network uses the Qth local map and the Rth local map as two third space vertices of a third space edge to obtain a third space graph structure; each third space edge includes a third space weight; Q≠R, and Q and R are both positive integers.

[0137] An optional embodiment of the present application is as follows: Figure 3As shown, the first processing network can be implemented based on a spatiotemporal graph attention neural network; specifically, the first processing network is based on a first spatiotemporal attention mechanism, and self-learns to obtain a plurality of first spatial weights, a plurality of second spatial weights, and a plurality of third spatial weights;

[0138] Among them, the first space weight F1 = softmax(d1), where d1 is the distance between two first space vertices corresponding to a first space edge;

[0139] The second space weight F2 = softmax(d2), where d2 is the distance between two second space vertices corresponding to a second space edge;

[0140] The third space weight F3=softmax(d3), where d3 is the distance between two third space vertices corresponding to a third space edge.

[0141] Specifically, the first spatial weight / the second spatial weight / the third spatial weight / the fourth spatial weight may be 0.2, 0.5, 0.6 or 0.8, etc.;

[0142] Taking the first space weight as an example, the larger the value of the first space weight is, the closer the distance between the two first space vertices is, the greater the influence of each other on each other, and the closer the relationship is; conversely, the smaller the value of the first space weight is, the farther the distance between the two first space vertices is, the smaller the influence of each other on each other, and the more distant the relationship is.

[0143] In an optional embodiment of the present application, in the time dimension, the first processing network constructs a time graph structure based on information of one of the target vehicles or one of the local maps, and obtains a plurality of the time graph structures, including:

[0144] The first processing network uses the Sth target vehicle as a first time vertex, and connects every two first time vertices through a first time edge to obtain a first time graph structure; wherein each first time edge includes a first time weight; S is a positive integer;

[0145] The first processing network uses the Wth local map as a second time vertex, and connects every two second time vertices through a second time edge to obtain a second time graph structure; wherein each second time edge includes a second time weight; and W is a positive integer.

[0146] The embodiments of the present application are as follows: Figure 3 As shown, the first processing network obtains a plurality of first time weights and a plurality of second time weights through self-learning based on a first spatiotemporal attention mechanism;

[0147] Among them, the first time weight H1=softmax(d4), where d4 is the distance between two first space vertices corresponding to a first space edge;

[0148] The second time weight H2=softmax(d5), where d5 is the distance between two second space vertices corresponding to a second space edge.

[0149] Specifically, the first time weight / the second time weight may be 0.2, 0.5 or 0.8, etc.;

[0150] Taking the first time weight as an example, the larger the value of the first time weight, the shorter the time distance between the two first time fixed points, the greater the influence of each other on each other, and the closer the relationship; conversely, the smaller the value of the first time weight, the greater the time distance between the two first time vertices, the smaller the influence of each other on each other, and the more distant the relationship.

[0151] In an optional embodiment of the present application, a first spatial graph structure, a second spatial graph structure, and a third spatial graph structure are obtained based on the spatial dimension; and a first time graph structure and a second time graph structure are obtained based on the time dimension;

[0152] Since the same vertex part is involved, for example, in the first spatial graph structure and the first temporal graph structure, a certain first spatial vertex and a first temporal vertex correspond to the same vector of the trajectory of the target vehicle. Therefore, the first spatial graph structure and the first temporal graph structure are fused based on the first spatial vertex and the first temporal vertex. By analogy, based on the vectors of the trajectories of multiple same target vehicles, the first spatial graph structure and the first temporal graph structure can be completely fused.

[0153] And so on, such as Figure 4 As shown, the first processing network can fuse the first spatial graph structure, the second spatial graph structure, the third spatial graph structure, the first time graph structure and the second time graph structure to obtain a three-dimensional space-time graph structure; wherein the space-time graph structure includes multiple space-time vertices, each two space-time vertices are connected by a space-time edge, and each space-time edge is provided with a space-time weight, for example: the space-time weight is 0.3, 0.4, 0.5 or 0.6, etc.

[0154] The embodiments of the present application construct a spatial graph structure in the spatial dimension and a temporal graph structure in the temporal dimension based on the trajectory of each target vehicle and the local map corresponding to each frame of the original image. This can effectively construct the spatial relationship between target vehicles, target vehicles and local maps, and local maps and local maps. It also constructs the contextual relationship of the target vehicles in the temporal dimension. At the same time, it also introduces attention mechanism information to more accurately and effectively construct the spatiotemporal graph structure, thereby improving the accuracy of predicting vehicle lane changes.

[0155] Step S24: Based on the feature embedding vector, predict the lane changing intention of each target vehicle and obtain a prediction result.

[0156] In an optional embodiment of the present application, predicting the lane change intention of each target vehicle based on the feature embedding vector and obtaining a prediction result includes:

[0157] Performing a second convolution process on the feature embedding vector to obtain the target vector;

[0158] Processing the target vector to obtain multiple intention probabilities;

[0159] Comparing the plurality of intention probabilities to obtain a target intention probability;

[0160] Based on the target intention probability, the prediction result is obtained.

[0161] The embodiments of the present application can be implemented based on a graph convolutional neural network. Specifically, based on an embedding processing method (for example, global embedding), based on multiple convolution kernels, high-order features are extracted from the information contained in the spatiotemporal graph structure to obtain a feature embedding vector based on the time dimension.

[0162] Furthermore, after obtaining the feature embedding vector, feature extraction is performed on the feature embedding vector based on multiple convolution kernels in the time dimension, and a target vector is output;

[0163] The target vector is normalized based on softmax to obtain multiple intent probabilities, including:

[0164] left lane change intention probability, right lane change intention probability and straight going intention probability;

[0165] The sum of the probability of the left lane change intention, the probability of the right lane change intention, and the probability of the straight-ahead intention is 1, and the intention with the largest probability value is taken as the prediction result;

[0166] For example: 1) if the probability of the intention to change left lane is 0.3, the probability of the intention to change right lane is 0.3, and the probability of the intention to go straight is 0.4, then the target vehicle’s lane change intention is predicted to go straight, not to change lanes;

[0167] 2) The probability of the left lane change intention is 0.3, the probability of the right lane change intention is 0.5, and the probability of the straight-ahead intention is 0.2, then the target vehicle's lane change intention is predicted to be right lane change;

[0168] 3) The probability of the intention to change left is 0.6, the probability of the intention to change right is 0.2, and the probability of the intention to go straight is 0.2, then the lane change intention of the target vehicle is predicted to be left.

[0169] like Figure 5 As shown, based on the above prediction results, the lane changing direction of each target vehicle around the vehicle is determined.

[0170] In an embodiment of the present application, information of multiple target vehicles around the own vehicle is collected and a sequence of original images is obtained. The original images are processed to obtain the trajectory of each target vehicle and a local map of each frame of the original image. The trajectory of each target vehicle and the local map of each frame of the original image are processed in the spatial dimension based on a first processing network to obtain a spatial map result; the trajectory of each target vehicle and the local map of each frame of the original image are processed in the time dimension based on the first processing network to obtain a time map structure; the time map structure and the spatial map structure are fused to obtain a space-time map structure; the space-time map structure is convoluted based on a second processing network to obtain a prediction result of the lane changing intention of each target vehicle around the own vehicle. The method is simple and has high prediction accuracy.

[0171] like Figure 6 As shown, the embodiment of the present application further provides a device 60 for predicting vehicle lane change, comprising:

[0172] The acquisition module 61 is used to collect information of multiple target vehicles and roads around the vehicle to obtain an original image sequence;

[0173] A first processing module 62 is used to process the original image sequence to obtain the trajectory of each target vehicle and the local map corresponding to each frame of the original image;

[0174] The second processing module 63 is used to process the trajectory of each target vehicle and the local map corresponding to each frame of the original image based on the first processing network, based on the spatial dimension and the time dimension, respectively, to obtain a spatiotemporal graph structure; and process the spatiotemporal graph structure based on the second processing network to obtain a feature embedding vector;

[0175] The prediction module 64 is used to predict the lane-changing intention of each target vehicle based on the feature embedding vector and obtain a prediction result.

[0176] Optionally, the processing of the original image sequence to obtain the trajectory of each target vehicle and a local map corresponding to each frame of the original image includes:

[0177] Performing feature extraction on the original image sequence to obtain a feature vector of each target vehicle and a local map vector corresponding to each frame of the original image;

[0178] Processing the feature vector of each target vehicle and each local map vector to obtain the initial state of each target vehicle and the initial state of each local map; wherein the initial state of each target vehicle includes the initial category, initial coordinates, initial size and initial angle of each target vehicle; the initial state of each local map includes the coordinates of the starting point and the coordinates of the ending point of each local map;

[0179] Based on the initial state of each of the target vehicles, the actual predicted state of each of the target vehicles is obtained; based on the actual predicted state of each of the target vehicles at J moments, the trajectory of each of the target vehicles is obtained; based on the initial state of each of the local maps, the corresponding local map is obtained; wherein J is a positive integer.

[0180] Optionally, obtaining the actual predicted state of each target vehicle based on the initial state of each target vehicle includes:

[0181] Based on the initial state of each target vehicle at the jth moment, obtaining the initial predicted state of each target vehicle at the j+1th moment;

[0182] Based on the initial state of each target vehicle at the jth moment, obtain the initial rectangular frame of each target vehicle at the jth moment; based on the initial predicted state of each target vehicle at the j+1th moment, obtain the predicted rectangular frame of each target vehicle at the j+1th moment; wherein 1≤j≤J, and j is an integer;

[0183] Based on the initial rectangular frame of each target vehicle at the jth moment and the predicted rectangular frame at the j+1th moment, a cost matrix of each target vehicle at the j+1th moment is calculated;

[0184] Based on the cost matrix of each target vehicle at the j+1th moment, the actual predicted state of each target vehicle at the j+1th moment is obtained; the actual predicted state includes: actual predicted category, actual predicted coordinates, actual predicted size and actual predicted angle.

[0185] Optionally, the first processing network processes the trajectory of each target vehicle and the local map corresponding to each frame of the original image based on the spatial dimension and the temporal dimension respectively to obtain a spatiotemporal graph structure; and the second processing network processes the spatiotemporal graph structure to obtain a feature embedding vector, including:

[0186] The first processing network obtains the trajectory of each target vehicle and the local map corresponding to each frame of the original image;

[0187] In the spatial dimension, the first processing network constructs a spatial graph structure based on the plurality of local maps and the trajectories of the plurality of target vehicles;

[0188] In the time dimension, the first processing network constructs a time graph structure based on one of the target vehicles or one of the local maps;

[0189] The first processing network fuses each of the spatial graph structures and each of the temporal graph structures to obtain a spatiotemporal graph structure;

[0190] Based on the second processing network, a first convolution processing is performed on the spatiotemporal graph structure to obtain the feature embedding vector.

[0191] Optionally, in the spatial dimension, the first processing network constructs a spatial graph structure based on the multiple local maps and the trajectories of the multiple target vehicles, including:

[0192] The first processing network constructs a first spatial graph structure, a second spatial graph structure, and a third spatial graph structure based on the plurality of local maps and the trajectories of the plurality of target vehicles;

[0193] Wherein, the first processing network uses the Mth target vehicle and the Nth target vehicle as two first spatial vertices of a first spatial edge to obtain a first spatial graph structure; each of the first spatial edges includes a first spatial weight; M≠N, and M and N are both positive integers;

[0194] The first processing network uses the Oth target vehicle and the Pth local map as two second spatial vertices of a second spatial edge to obtain a second spatial graph structure; each of the second spatial edges includes a second spatial weight; O and P are both positive integers;

[0195] The first processing network uses the Qth local map and the Rth local map as two third space vertices of a third space edge to obtain a third space graph structure; each third space edge includes a third space weight; Q≠R, and Q and R are both positive integers.

[0196] Optionally, in the time dimension, the first processing network constructs a time graph structure based on one of the target vehicles or one of the local maps, including:

[0197] The first processing network uses the Sth target vehicle as a first time vertex, and connects every two first time vertices through a first time edge to obtain a first time graph structure; wherein each first time edge includes a first time weight; S is a positive integer;

[0198] The first processing network uses the Wth local map as a second time vertex, and connects every two second time vertices through a second time edge to obtain a second time graph structure; wherein each second time edge includes a second time weight; and W is a positive integer.

[0199] Optionally, predicting the lane change intention of each target vehicle based on the feature embedding vector and obtaining a prediction result includes:

[0200] Performing a second convolution process on the feature embedding vector to obtain the target vector;

[0201] Processing the target vector to obtain multiple intention probabilities;

[0202] Comparing the plurality of intention probabilities to obtain a target intention probability;

[0203] Based on the target intention probability, the prediction result is obtained.

[0204] An embodiment of the present application further provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the method as described above when executing the computer program.

[0205] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method described above.

[0206] In addition, other structures and functions of the device of the embodiment of the present application are known to those skilled in the art and will not be described in detail here to reduce redundancy.

[0207] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0208] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0209] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0210] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0211] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0212] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0213] In the present application, unless otherwise clearly specified and limited, a first feature being “above” or “below” a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being “above”, “above”, and “above” a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being “below”, “below”, and “below” a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0214] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for predicting vehicle lane change, characterized in that: include: Collect information about multiple target vehicles and roads around the vehicle to obtain original image sequences; Processing the original image sequence to obtain the trajectory of each target vehicle and a local map corresponding to each frame of the original image; Based on the first processing network, the trajectory of each target vehicle and the local map corresponding to each frame of the original image are processed based on the spatial dimension and the temporal dimension respectively to obtain a spatiotemporal graph structure; Processing the spatiotemporal graph structure based on a second processing network to obtain a feature embedding vector; Based on the feature embedding vector, predicting the lane change intention of each target vehicle and obtaining a prediction result; The first processing network processes the trajectory of each target vehicle and the local map corresponding to each frame of the original image based on the spatial dimension and the temporal dimension respectively to obtain a spatiotemporal graph structure; Processing the spatiotemporal graph structure based on a second processing network to obtain a feature embedding vector includes: The first processing network obtains the trajectory of each target vehicle and the local map corresponding to each frame of the original image; In the spatial dimension, the first processing network constructs a spatial graph structure based on the plurality of local maps and the trajectories of the plurality of target vehicles; In the time dimension, the first processing network constructs a time graph structure based on one of the target vehicles or one of the local maps; The first processing network fuses each of the spatial graph structures and each of the temporal graph structures to obtain a spatiotemporal graph structure; Performing a first convolution process on the spatiotemporal graph structure based on the second processing network to obtain the feature embedding vector; The predicting the lane change intention of each target vehicle based on the feature embedding vector and obtaining a prediction result includes: Performing a second convolution process on the feature embedding vector to obtain a target vector; The target vector is normalized based on softmax to obtain multiple intention probabilities, including left lane change intention probability, right lane change intention probability and straight driving intention probability; Comparing the plurality of intention probabilities to obtain a target intention probability; Based on the target intention probability, the prediction result is obtained.

2. The method according to claim 1, characterized in that The processing of the original image sequence to obtain the trajectory of each target vehicle and the local map corresponding to each frame of the original image includes: Performing feature extraction on the original image sequence to obtain a feature vector of each target vehicle and a local map vector corresponding to each frame of the original image; Processing the feature vector of each target vehicle and each local map vector to obtain the initial state of each target vehicle and the initial state of each local map; wherein the initial state of each target vehicle includes the initial category, initial coordinates, initial size and initial angle of each target vehicle; the initial state of each local map includes the coordinates of the starting point and the coordinates of the ending point of each local map; Based on the initial state of each of the target vehicles, the actual predicted state of each of the target vehicles is obtained; based on the actual predicted state of each of the target vehicles at J moments, the trajectory of each of the target vehicles is obtained; based on the initial state of each of the local maps, the corresponding local map is obtained; wherein J is a positive integer.

3. The method according to claim 2, characterized in that The obtaining the actual predicted state of each target vehicle based on the initial state of each target vehicle comprises: Based on the initial state of each target vehicle at the jth moment, obtaining the initial predicted state of each target vehicle at the j+1th moment; Based on the initial state of each target vehicle at the jth moment, obtain the initial rectangular frame of each target vehicle at the jth moment; based on the initial predicted state of each target vehicle at the j+1th moment, obtain the predicted rectangular frame of each target vehicle at the j+1th moment; wherein 1≤j≤J, and j is an integer; Based on the initial rectangular frame of each target vehicle at the jth moment and the predicted rectangular frame at the j+1th moment, a cost matrix of each target vehicle at the j+1th moment is calculated; Based on the cost matrix of each target vehicle at the j+1th moment, the actual predicted state of each target vehicle at the j+1th moment is obtained; the actual predicted state includes: actual predicted category, actual predicted coordinates, actual predicted size and actual predicted angle.

4. The method according to claim 1, characterized in that: In the spatial dimension, the first processing network constructs a spatial graph structure based on the multiple local maps and the trajectories of the multiple target vehicles, including: The first processing network constructs a first spatial graph structure, a second spatial graph structure, and a third spatial graph structure based on the plurality of local maps and the trajectories of the plurality of target vehicles; Wherein, the first processing network uses the Mth target vehicle and the Nth target vehicle as two first spatial vertices of a first spatial edge to obtain a first spatial graph structure; each of the first spatial edges includes a first spatial weight; M≠N, and M and N are both positive integers; The first processing network uses the Oth target vehicle and the Pth local map as two second spatial vertices of a second spatial edge to obtain a second spatial graph structure; each of the second spatial edges includes a second spatial weight; O and P are both positive integers; The first processing network uses the Qth local map and the Rth local map as two third space vertices of a third space edge to obtain a third space graph structure; each third space edge includes a third space weight; Q≠R, and Q and R are both positive integers.

5. The method according to claim 1, characterized in that In the time dimension, the first processing network constructs a time graph structure based on one of the target vehicles or one of the local maps, including: The first processing network uses the Sth target vehicle as a first time vertex, and connects every two first time vertices through a first time edge to obtain a first time graph structure; wherein each first time edge includes a first time weight; S is a positive integer; The first processing network uses the Wth local map as a second time vertex, and connects every two second time vertices through a second time edge to obtain a second time graph structure; wherein each second time edge includes a second time weight; and W is a positive integer.

6. A device for predicting vehicle lane change, characterized in that: include: The acquisition module is used to collect information about multiple target vehicles and roads around the vehicle and obtain original image sequences; A first processing module is used to process the original image sequence to obtain the trajectory of each target vehicle and a local map corresponding to each frame of the original image; A second processing module is used to process the trajectory of each target vehicle and the local map corresponding to each frame of the original image based on the first processing network, based on the spatial dimension and the time dimension respectively, to obtain a spatiotemporal graph structure; Processing the spatiotemporal graph structure based on a second processing network to obtain a feature embedding vector; A prediction module, used for predicting the lane-changing intention of each target vehicle based on the feature embedding vector, and obtaining a prediction result; The first processing network processes the trajectory of each target vehicle and the local map corresponding to each frame of the original image based on the spatial dimension and the temporal dimension respectively to obtain a spatiotemporal graph structure; Processing the spatiotemporal graph structure based on a second processing network to obtain a feature embedding vector includes: The first processing network obtains the trajectory of each target vehicle and the local map corresponding to each frame of the original image; In the spatial dimension, the first processing network constructs a spatial graph structure based on the plurality of local maps and the trajectories of the plurality of target vehicles; In the time dimension, the first processing network constructs a time graph structure based on one of the target vehicles or one of the local maps; The first processing network fuses each of the spatial graph structures and each of the temporal graph structures to obtain a spatiotemporal graph structure; Performing a first convolution process on the spatiotemporal graph structure based on the second processing network to obtain the feature embedding vector; The predicting the lane change intention of each target vehicle based on the feature embedding vector and obtaining a prediction result includes: Performing a second convolution process on the feature embedding vector to obtain a target vector; The target vector is normalized based on softmax to obtain multiple intention probabilities, including left lane change intention probability, right lane change intention probability and straight driving intention probability; Comparing the plurality of intention probabilities to obtain a target intention probability; Based on the target intention probability, the prediction result is obtained.

7. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 5.

Images