Vehicle motion prediction method and driving risk warning method

Abstract

The present invention belongs to the field of intelligent driving technology, and specifically relates to a vehicle motion prediction method and a driving risk warning method. The vehicle motion prediction method comprises: S1, collecting on-board visual data, serializing it to obtain sequence data, and using the sequence data to train a motion prediction model to obtain a motion prediction model; S2, constructing an encoder for the motion prediction model to obtain the aggregated hidden states between surrounding vehicles; S3, constructing a graph-based vehicle spatial interaction module to iteratively calculate the spatial interaction weights between surrounding vehicles using graph structure modeling; S4, constructing a decoder for the motion prediction model to obtain parameters describing a binary Gaussian distribution, and randomly sampling control points for each surrounding vehicle in the binary Gaussian distribution; S5, calculating a Bezier curve based on the control points, and obtaining high-order motion state quantities based on the Bezier curve. The present invention meets the current demand for predicting surrounding vehicle motion in actual complex and dynamic traffic scenarios for intelligent vehicles.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent driving technology, and specifically relates to a vehicle motion prediction method and a driving risk warning method. Background Art

[0002] Predicting the motion of surrounding traffic participants and assessing potential driving risks are crucial for smart cars. By predicting the position, speed, acceleration and other motion state parameters of surrounding traffic participants in the future and assessing and warning potential driving risks within the predicted time domain, smart cars can make safer and more reasonable decisions and plans.

[0003] Vehicles are the primary participants in traffic scenes, so predicting their motion is a typical motion prediction task. Vehicle motion prediction is typically performed from two perspectives: 1. Predicting the motion of vehicles within the field of view from a bird's-eye view (drone photography) or a near-bird's-eye view (roadside vision sensor photography); and 2. Predicting the motion of vehicles in front of the vehicle from a vision sensor positioned in front of the vehicle. This perspective is similar to that of a human driver, and is defined as the on-board perspective.

[0004] With the rapid mass production of intelligent vehicles equipped with autonomous driving technology, vehicle motion prediction methods based on a traffic perspective are unable to meet the actual needs of autonomous driving. Compared with vehicle motion prediction methods based on a traffic perspective, vehicle motion prediction methods based on an onboard perspective face more challenges, such as more obvious target occlusion and more frequent changes in target depth information. Therefore, vehicle motion prediction methods based on an onboard perspective have become a research focus. In addition, in terms of driving risk assessment, deterministic safety indicators such as Time to Collision (TTC) are often used to evaluate driving risk. These deterministic indicators generally ignore the uncertainty of the motion of the ego vehicle and surrounding vehicles and lack flexibility in handling two-dimensional motion with lateral and longitudinal motion. They cannot effectively assess the potential risk of the current driving situation. Therefore, there is a growing demand for driving risk assessment methods or indicators that can take into account the two-dimensional motion of the vehicle and its uncertainty.

[0005] Existing technologies typically rely on data-driven approaches, where data-driven motion prediction methods directly predict and output the vehicle's future motion state, directly mapping the input vehicle state information to future high-order state quantities (such as speed, acceleration, etc.). This approach directly predicts high-order quantities of the vehicle's motion state for a period of time in the future without complying with physical constraints such as vehicle kinematics and dynamics. This can lead to inconsistencies between the predicted results and the true values. For example, the predicted result shows that the vehicle's speed is zero, but the vehicle still experiences positional displacement, which can cause confusion or even errors in downstream decision-making and planning modules.

[0006] In the prior art, Chinese patent CN114820216A discloses a driving risk quantification, classification, and prediction method. This method uses deterministic risk assessment metrics such as TTC and THW (Time Headway) to assess and quantify driving risk in the current traffic environment. However, this method fails to consider the uncertainty of surrounding vehicle motion and the driver's subjective uncertainty in its driving cognitive risk approach. Furthermore, because the calculation formulas for TTC and THW are typically based on current driving data of the ego vehicle and target vehicle, they cannot accurately quantify driving risk over a period of time in the future. Summary of the Invention

[0007] The present invention addresses the technical problem in the prior art that, when predicting vehicle motion and assessing risk, physical constraints are not taken into account or uncertainty is not considered, resulting in difficulty in accurately predicting the movement of surrounding vehicles and inaccurate risk assessment. The purpose of the present invention is to provide a vehicle motion prediction method and a driving risk warning method.

[0008] In order to solve the aforementioned technical problem, a first aspect of the present invention provides a vehicle motion prediction method, the vehicle motion prediction method comprising:

[0009] S1, collecting vehicle-mounted visual data, serializing the vehicle-mounted visual data for each surrounding vehicle along the collection time sequence to obtain sequence data having a plurality of segments, and using the sequence data to train a preset motion prediction model to obtain a motion prediction model corresponding to each surrounding vehicle;

[0010] S2, constructing an encoder of the motion prediction model, assigning an encoder to each pericar respectively, and obtaining the aggregated hidden state between the pericars;

[0011] S3, with each vehicle as a node and the spatial interaction relationship between vehicles as an edge, builds a vehicle spatial interaction module based on a graph structure, and iteratively calculates the spatial interaction weights between surrounding vehicles using a graph structure modeling method;

[0012] S4, constructing a decoder of the motion prediction model, splicing the aggregated hidden states between the pericars and the spatial interaction weights between the pericars, regressing the bivariate Gaussian distribution of the control points under a preset Bezier curve control point output paradigm to obtain parameters describing the bivariate Gaussian distribution, and randomly sampling a preset number of control points from the bivariate Gaussian distribution for each pericar;

[0013] S5, calculating a Bezier curve based on the control points of each pericyte. The Bezier curve is the predicted trajectory of the pericyte. The high-order quantity of the motion state of the pericyte in the predicted time domain is obtained by derivation of the Bezier curve.

[0014] Optionally, in the vehicle motion prediction method as described above, in step S1, collecting vehicle-mounted visual data includes:

[0015] One or more visual sensors are arranged at preset positions on the vehicle, and a preset calibration algorithm is used to obtain the internal and external parameters of the visual sensors. After the calibration is successful, the vehicle-mounted visual data of the visual sensors are collected.

[0016] Optionally, in the vehicle motion prediction method as described above, after collecting the vehicle-mounted visual data in step S1, the method further includes:

[0017] Obtaining surrounding vehicle information, the surrounding vehicle information including the distance between each surrounding vehicle and the own vehicle and the vehicle number corresponding to each surrounding vehicle;

[0018] According to the different vehicle numbers, the vehicle-borne visual data is serialized for each vehicle along the collection time sequence to obtain sequence data.

[0019] Optionally, in the vehicle motion prediction method as described above, obtaining surrounding vehicle information includes:

[0020] Using a preset target detection module to perform target detection on the vehicle-mounted visual data, a target detection frame containing the position and category of each vehicle on each frame image is obtained;

[0021] Estimate the surrounding vehicles based on the target detection frame and the visual sensor parameters to obtain the distance between the surrounding vehicles and the self-vehicle;

[0022] A preset target tracking module is used to associate and track the surrounding vehicles detected at different times in the vehicle-borne visual data, and a corresponding vehicle number is assigned to each surrounding vehicle.

[0023] Optionally, in the vehicle motion prediction method as described above, when the on-board visual data is serialized for each vehicle along the acquisition time sequence, a sliding window method is used for serialization, and the sliding window is the sequence length.

[0024] Optionally, in the vehicle motion prediction method as described above, after the on-board visual data is serialized for each vehicle along the acquisition time sequence, outliers are removed to obtain sequence data.

[0025] Optionally, in the vehicle motion prediction method as described above, in step S1, the sequence data is used to train a preset motion prediction model to obtain a motion prediction model corresponding to each surrounding vehicle, including:

[0026] The sequence data is evenly divided into K parts using the preset K-fold cross-validation method, one of which is selected as the validation set and K-1 parts as the training set. The first h seconds of each sequence are used as the observation sequence and the last Hh seconds are used as the prediction sequence, where H is the sequence length and h is less than H;

[0027] The motion prediction model is trained using the training set, the trained motion prediction model is verified using the validation set, and an evaluation index is calculated. If the evaluation index reaches a preset expected index, the training of the motion prediction model is considered complete.

[0028] Optionally, in the vehicle motion prediction method as described above, the evaluation index is one of the final displacement error (FDE) and the average displacement error (ADE), or a combination thereof, and the calculation formulas are respectively formula (1) and formula (2):

[0029]

[0030] Among them, the middle T f Indicates the length of the prediction sequence, the subscript t represents each time step in the prediction time series, and x t and y t are the ordinate and abscissa of the true value of the trajectory, and are the ordinate and abscissa of the trajectory prediction value respectively.

[0031] Optionally, in the vehicle motion prediction method as described above, in step S2, an encoder is assigned to each surrounding vehicle to obtain the aggregated hidden state between the surrounding vehicles, including:

[0032] A LSTM (Long Short Term Memory network) encoder is assigned to each of the pericycles in the observation sequence. The LSTM encoder is used to extract features from the historical motion state information in the observation sequence. The extracted features are output in the hidden state. The calculation formula for the aggregated hidden state between the pericycles is formula (3):

[0033]

[0034] in, represents the aggregated hidden state of vehicle i at time step t, j represents another vehicle different from vehicle i, N represents the total number of vehicles in the week, M j represents the hidden state aggregation process from vehicle j to vehicle i, is the hidden state of vehicle j at time step t, is the hidden state of vehicle i at time step t.

[0035] Optionally, in the vehicle motion prediction method as described above, the hidden state aggregation process adopts one or a combination of splicing, addition and multiplication methods.

[0036] Optionally, in the vehicle motion prediction method as described above, in step S3, the vehicle space interaction module is an undirected graph, and the condition for the existence of an edge between two nodes is that the Euclidean distance between the two nodes is less than a preset distance threshold.

[0037] Optionally, in the vehicle motion prediction method as described above, in step S3, the spatial interaction weights between surrounding vehicles are iteratively calculated in a graph structure modeling manner, including:

[0038] Each vehicle is regarded as a node v in the graph structure, v={v1,v2,v3,...,v N}, N is the total number of weekly vehicles;

[0039] At time step t, establish spatial interaction based on graph structure and define any node v in the undirected graph G i and node v j The symmetric adjacency matrix of is A(i,j), as shown in formula (4):

[0040]

[0041] in, Represents node v i With node v j The spatial interaction relationship between thresh is the distance threshold;

[0042] The non-regularized Laplacian matrix L = DA of the undirected graph G is defined as a symmetric matrix as shown in formula (5):

[0043]

[0044] Where D(i,j) is the main diagonal of the degree matrix D of the undirected graph G, The remaining elements are all 0;

[0045] The Laplace matrix defined at time step t is represented as L t , then the Laplace matrix L of the next time step t+1 The update method is as follows:

[0046]

[0047] Among them, ff transrepresents the outer product of rank 2, f is a sparse matrix and satisfies ||f||0<<N. Each time a new non-zero entry appears in the j-th row of f, it means that a new weekly vehicle has been added to the current undirected graph G. The maximum number of nodes in the undirected graph G is the total number of weekly vehicles N.

[0048] Optionally, in the vehicle motion prediction method as described above, in step S4, the formula of the n-order Bezier curve is as follows:

[0049]

[0050] in, represents the Bessel basis function of the i-th order, η represents the Bessel scale factor, P i Indicates the control point of the Bezier curve, t f is any time step in the prediction domain, T f represents the predicted sequence length,

[0051] Optionally, in the vehicle motion prediction method as described above, in step S4, when regressing the binary Gaussian distribution of the control points, a preset multi-layer perceptron (MLP) is used to regress the binary Gaussian distribution of the Bezier curve control points to obtain the parameter μ describing the binary Gaussian distribution. x 、μ y , σ x , σ y and ρ represent the x-mean of the ordinate, the y-mean of the abscissa, the x-standard deviation of the ordinate, the y-standard deviation of the abscissa, and the correlation coefficient of the bivariate Gaussian distribution, respectively.

[0052] Optionally, in the vehicle motion prediction method as described above, in step S5, the Bezier curve coordinates are formula (12):

[0053]

[0054] Among them, Y pos Represents the position coordinates, B represents the constant Bessel basis matrix, and P represents the predicted control point coordinates.

[0055] Optionally, in the vehicle motion prediction method as described above, in step S5, the high-order quantity of the motion state includes at least one of velocity, acceleration and jerk, and the high-order quantity of the motion state is obtained by taking high-order derivatives of the Bezier curve.

[0056] In order to solve the aforementioned technical problem, a second aspect of the present invention provides a driving risk warning method, which includes:

[0057] The parameters of the surrounding vehicles in the bivariate Gaussian distribution obtained in step S4 are used to calculate the collision probability between the ego vehicle and the surrounding vehicles in the predicted time domain. The high-order motion state variables obtained in step S5 are used to calculate the expected collision severity of the ego vehicle. The product of the collision probability and the expected collision severity of the ego vehicle is used as the driving risk of the ego vehicle.

[0058] The driving risk is compared with a preset risk threshold, and if the driving risk is greater than the risk threshold, a risk warning is issued.

[0059] Optionally, in the aforementioned driving risk warning method, the parameters of the surrounding vehicles in the bivariate Gaussian distribution obtained in step S4 are used to calculate the collision probability between the vehicle and the surrounding vehicles in the prediction time domain, including:

[0060] The parameters of the bivariate Gaussian distribution of the pericycle obtained in step S4 are given by formula (13):

[0061]

[0062] Among them, the parameter μ of the bivariate Gaussian distribution is x 、μ y , σ x , σ y and ρ, respectively represent the x-mean, y-mean, x-standard deviation, y-standard deviation, and correlation coefficient of the bivariate Gaussian distribution. f is any time step in the prediction domain, It is the Gaussian distribution representation of the two-dimensional motion of the pericycle;

[0063] When the ego vehicle is driven by a human driver, there is also subjective driving uncertainty. Formula (13) is rewritten and the collision probability is calculated by formula (14):

[0064]

[0065] in, They are the parameter representations after introducing the binary Gaussian distribution of the subjective uncertainty of human drivers, and represents the standard deviation of the Gaussian distribution of subjective uncertainty of human drivers;

[0066] When the vehicle is in the automatic driving state, the collision probability is calculated by formula (15):

[0067]

[0068] Among them, x, and is the vertical and horizontal coordinates of the vehicle at any time step in the prediction domain, and Provided by the preset autonomous driving planning module, L sub and L sur is the length of the vehicle and the length of the surrounding vehicle, W sub and W sur is the width of the vehicle and the surrounding vehicles, L sub and W sub is the default value, L sur and W sur The vehicle-mounted visual data collected in step S1 is obtained through target detection.

[0069] Optionally, in the aforementioned driving risk warning method, the high-order motion state quantity obtained in step S5 is used to calculate the expected collision severity of the vehicle, including:

[0070] The expected collision severity of the ego vehicle is calculated by formula (16):

[0071] s(t+Δt)=0.5Mβ 2 (Δv(t+Δt)) 2 (16)

[0072] Where M is the weight of the vehicle, M is the preset value, β is the weight ratio of the surrounding vehicle, M sur is the weight of the car, M sur The empirical value is obtained according to a preset empirical formula after the type and length and width information of the surrounding vehicles are obtained through target detection using the on-board visual data collected in step S1. Δv(t+Δt) is the speed difference between the ego vehicle and the surrounding vehicles between time step t and time increment Δt. The ego vehicle speed is provided by the preset autonomous driving planning module, and the surrounding vehicle speed is provided by the high-order motion state quantity obtained in step S5.

[0073] Optionally, in the driving risk warning method as described above, the risk threshold is a dynamic risk threshold, and the dynamic risk threshold has corresponding different risk levels, and the risk levels are obtained through subjective driving risk cognition experiments in different driving scenarios.

[0074] The positive progress effect of the present invention is:

[0075] 1. The present invention better associates the upstream perception module with the motion prediction module from the vehicle-mounted perspective to realize the surrounding vehicle motion prediction and driving risk warning method, which helps smart cars to achieve safer and more reasonable decision-making and planning, and meets the current smart car's surrounding vehicle motion prediction needs in actual complex dynamic traffic scenarios.

[0076] 2. The present invention is based on a graph-structured spatial interaction module for surrounding vehicles, which reflects the spatial interaction relationship between surrounding vehicles in an actual traffic environment. The model output paradigm based on Bezier curve control points overcomes the problem that previous data-driven motion prediction methods may make errors in directly predicting high-order quantities of motion states without physical constraints.

[0077] 3. Based on the uncertainty of vehicle motion prediction, the present invention introduces the driver's subjective uncertainty for human-machine co-driving situations, more objectively describing the subjective cognitive risks of drivers and passengers in complex dynamic scenarios. While ensuring objective safety, it enables drivers and passengers to better recognize the driving risks of smart cars. The present invention can provide a reliable reference for the decision-making and planning modules downstream of the autonomous driving system.

[0078] 4. The present invention determines the dynamic risk threshold through the driver's subjective risk perception results, and warns the driver of the vehicle when the driving risk is greater than the risk threshold of the current scenario. This solves the problem that previous deterministic safety assessment indicators cannot objectively describe driving risks in complex dynamic traffic scenarios, and thus it is difficult to provide accurate driving risk warnings for the vehicle. It can provide safety guarantees for the implementation of reasonable decision-making and planning of smart cars.

[0079] 5. The present invention solves the problem that the vehicle motion prediction method based on a bird's-eye view or near-bird's-eye view is difficult to accurately predict the motion of surrounding vehicles due to reasons such as occlusion by surrounding vehicles and frequent changes in depth information in actual traffic scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] The disclosure of the present invention will become more apparent with reference to the accompanying drawings. It should be understood that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. In the drawings:

[0081] Figure 1 A flow chart of a method of the present invention;

[0082] FIG2( a ) is a diagram showing the acquisition viewing angle of the vehicle-mounted visual sensor after installation in Example 1 of the present invention;

[0083] FIG2( b ) is a schematic diagram of a road section collected on an electronic map in Example 1 of the present invention;

[0084] Figure 3 is the target detection result of a frame of image after target detection in Example 1 of the present invention;

[0085] Figure 4 This is a schematic diagram of the space interaction between the vehicle and the surrounding vehicle in Example 1 of the present invention. DETAILED DESCRIPTION

[0086] The following describes the embodiments of the present invention through specific examples. Those skilled in the art will readily understand the other advantages and benefits of the present invention from the disclosure herein. The present invention may also be implemented or applied through various other specific embodiments, and the details in this specification may be modified or altered based on different viewpoints and applications without departing from the spirit of the present invention.

[0087] It should be noted that, unless there is any conflict, the following embodiments and features therein may be combined with each other.

[0088] In the description of the present invention, it should be noted that, for directional words, such as the terms "outside", "middle", "inside", "outside", etc., the directions and positional relationships indicated are based on the directions or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and cannot be understood as limiting the specific scope of protection of the present invention.

[0089] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features. Therefore, the terms "first" and "second" may explicitly or implicitly refer to one or more of these features. Throughout the description of the present invention, "several" and "a number" mean two or more, unless otherwise specifically defined.

[0090] Reference Figure 1 An embodiment of the present invention provides a vehicle motion prediction method, which accurately predicts the motion of surrounding vehicles and evaluates the current driving risk from a vehicle-mounted perspective. The method specifically utilizes the surrounding environment information from a vehicle-mounted perspective and proposes a surrounding vehicle motion prediction method that considers the spatial interaction of surrounding vehicles. At the same time, an output paradigm based on Bezier curve control points is proposed in the prediction process. This method allows the motion state of surrounding vehicles and their high-order derivatives to be predicted at any time step in the prediction time domain.

[0091] The vehicle motion prediction method includes the following specific steps:

[0092] S1, data acquisition and preprocessing: collect vehicle-mounted visual data, serialize the vehicle-mounted visual data for each surrounding vehicle along the acquisition time sequence, obtain sequence data with several segments, use the sequence data to train the preset motion prediction model, and obtain the motion prediction model corresponding to each surrounding vehicle.

[0093] The data collected in this step is the vehicle-mounted visual data from the vehicle-mounted perspective. By processing the vehicle-mounted visual data using existing methods, the surrounding vehicle information under the vehicle-mounted vision can be obtained, including the position of each vehicle in a frame of image, vehicle type, vehicle length and width, and vehicle code.

[0094] In this step, when using sequence data for training, each peripheral vehicle is trained separately to obtain a motion prediction model corresponding to each peripheral vehicle.

[0095] In some embodiments, in step S1, collecting onboard visual data includes calibrating an onboard visual sensor and collecting onboard viewpoint data. Specifically, one or more visual sensors are arranged at predetermined positions on the vehicle, and a predetermined calibration algorithm is used to obtain internal and external parameters of the visual sensors. After successful calibration, onboard visual data from the visual sensors is collected.

[0096] In this embodiment, the calibration algorithm is an existing technology, such as Zhang's calibration method, calibration based on an optimization algorithm, calibration based on a genetic algorithm, and calibration based on a neural network. After successful calibration, the vehicle is driven to collect image data from the vehicle's perspective, i.e., vehicle-mounted visual data.

[0097] Preferably, the perception range of the visual sensor can be selected as 200 meters, the time window can be selected as 10 minutes, and the acquisition frequency can be 10 Hz.

[0098] In some embodiments, in step S1, after collecting the vehicle-mounted visual data, the method further includes:

[0099] Obtain surrounding vehicle information, which includes the distance between each surrounding vehicle and the own vehicle and the vehicle number corresponding to each surrounding vehicle.

[0100] Serialize vehicle-mounted visual data: According to the different vehicle numbers, the vehicle-mounted visual data is serialized for each vehicle along the collection time sequence to obtain serial data.

[0101] In this embodiment, when obtaining surrounding vehicle information, the target detection module and target tracking module in the intelligent automobile environment perception system in the existing technology can be used to detect and track vehicles within the perception range of the visual sensor to obtain information about the vehicle during the collection process.

[0102] In this embodiment, the vehicle-mounted visual data and the surrounding vehicle information obtained from the vehicle-mounted visual data can be saved in a CSV file for integration, and serialized according to the different vehicle numbers and along the collection time sequence.

[0103] In some embodiments, the process of obtaining surrounding vehicle information based on the target detection module and the target tracking module is as follows:

[0104] The object detection module performs object detection on the onboard visual data, generating object detection frames containing the location and category of each surrounding vehicle in each frame. Based on the object detection frames and visual sensor parameters, the surrounding vehicles are estimated to obtain their distance from the ego vehicle. The object tracking module associates and tracks the surrounding vehicles detected at different times in the onboard visual data, assigning each surrounding vehicle a corresponding vehicle number.

[0105] For target detection, a target detection frame of surrounding vehicles is obtained from a frame of image data in the vehicle-mounted visual data. Depending on the visual sensor used, the dimension of the target detection frame obtained may be two-dimensional or three-dimensional, but it should at least include the length, width, and center point information of the target detection frame. The target detection module can use existing technologies. For example, the method used by the target detection module for vehicle target detection can be a single-stage method (such as the YOLO series algorithm, the DETR series algorithm, etc.), a two-stage method (the RCNN series algorithm), or a Transformer-based method.

[0106] For target tracking, surrounding vehicles detected by the visual sensor at different times during data collection are correlated and tracked to obtain unique vehicle numbers. This means that each vehicle in the scene has its own unique vehicle number. The target tracking module can utilize existing technologies. For example, the target tracking module can employ one or a combination of rule-based and deep learning methods.

[0107] In some embodiments, when the vehicle-borne visual data is serialized for each vehicle along the acquisition time sequence, a sliding window method is used for serialization, and the sliding window is the sequence length.

[0108] For example, if the sliding window is set to 5 seconds, the sequence length is also 5 seconds.

[0109] In some embodiments, after the vehicle-borne visual data is serialized for each vehicle along the acquisition time sequence, outliers are removed to obtain sequence data.

[0110] Serialized data may contain outliers due to errors in target detection and tracking, among other reasons. These include, but are not limited to, sudden changes in vehicle position caused by inconsistent vehicle numbers due to tracking errors. Outliers can be identified and removed using manual inspection, statistical feature analysis, or methods based on domain knowledge and specific rules, or a combination thereof.

[0111] In some embodiments, in step S1, using the sequence data to train a preset motion prediction model to obtain the motion prediction model corresponding to each surrounding vehicle includes dividing the sequence data set using a K-fold cross-validation method.

[0112] Specifically, the K-fold cross-validation method is used to evenly divide the sequence data into K parts. One of the parts is selected as the validation set, and K-1 parts are selected as the training set. The first h seconds of each sequence are used as the observation sequence, and the last Hh seconds are used as the prediction sequence, where H is the sequence length and h is less than H. The motion prediction model is trained using the training set, and the trained motion prediction model is validated using the validation set. Evaluation indicators are calculated, and if the evaluation indicators meet the preset expected indicators, the motion prediction model training is considered complete.

[0113] Serialized sequence data consists of several segments. K-fold cross-validation is used to divide the dataset into K equal subsets, and multiple iterations of training and validation are performed. This overcomes the limitations of a simple training-test set partition. Especially when maximizing the value of limited data, K-fold cross-validation can effectively utilize data, reduce overfitting, and provide a more reliable assessment of model generalization ability. The preferred value of K is between [2, 10].

[0114] This embodiment also divides each sequence into an observation sequence and a prediction sequence. For example, when the sequence length is 5 seconds, the first 3 seconds are used as the observation sequence and the last 2 seconds are used as the prediction sequence.

[0115] The evaluation index and its calculation method in this embodiment are based on the existing technology. The evaluation index is preferably one of the final displacement error (FDE) and the average displacement error (ADE), or a combination thereof. The calculation formula of FDE is formula (1), and the calculation formula of ADE is formula (2):

[0116]

[0117] Among them, the middle T f Indicates the length of the prediction sequence, the subscript t represents each time step in the prediction time series, and x t and y t are the ordinate and abscissa of the true value of the trajectory, and are the ordinate and abscissa of the trajectory prediction value respectively.

[0118] FDE and ADE can be used to evaluate the accuracy and precision of the motion prediction model. Small FDE and ADE values, such as those below the corresponding expected indicators, indicate that the predicted results are close to the actual trajectory position, indicating that the motion prediction model performs well and training has been completed. Of course, if the FDE and ADE values are not less than the expected indicators, the model can be adjusted by modifying model parameters, and training verification and evaluation can be repeated to complete the training.

[0119] S2, build encoder to extract features: build the encoder of the motion prediction model, assign an encoder to each pericar respectively, and obtain the aggregated hidden state between the pericars.

[0120] The encoder in this step is used to encode the feature information in the motion prediction model into an intermediate state for use in generating trajectories in the subsequent decoding stage.

[0121] In some embodiments, in step S2, an encoder is assigned to each surrounding vehicle to obtain the aggregated hidden state between the surrounding vehicles, including:

[0122] A LSTM (Long Short Term Memory network) encoder is assigned to each of the surrounding vehicles in the observation sequence. The LSTM encoder is used to extract features from the historical motion state information in the observation sequence. The extracted features are output in the hidden state and then used as input to the motion prediction model decoder.

[0123] The calculation formula of the aggregated hidden state between the surrounding vehicles is formula (3):

[0124]

[0125] in, represents the aggregated hidden state of vehicle i at time step t, j represents another vehicle different from vehicle i, N represents the total number of vehicles in the week, M j represents the hidden state aggregation process from vehicle j to vehicle i, is the hidden state of vehicle j at time step t, is the hidden state of vehicle i at time step t.

[0126] The hidden state aggregation process uses one or a combination of concatenation, addition and multiplication methods.

[0127] S3, constructing a vehicle spatial interaction module based on a graph structure: with each vehicle as a node and the spatial interaction relationship between vehicles as an edge, a vehicle spatial interaction module based on a graph structure is constructed, and the spatial interaction weights between surrounding vehicles are iteratively calculated using a graph structure modeling method.

[0128] This step uses a graph structure to express the interactions between surrounding vehicles in a traffic scenario. A graph is a nonlinear data structure consisting of nodes (also called vertices) and edges. Graphs can be used to represent various relationship networks. A node is the basic unit in a graph and represents an entity. An edge is a line connecting two nodes, indicating a certain relationship between the two nodes. Undirected graphs have no directionality; if there is an edge from A to B, there is also an edge from B to A. Degree represents the number of edges connected to a node.

[0129] The spatial interaction weights between surrounding vehicles are iteratively calculated by graph structure modeling, so that at time step t, the smaller the relative distance between two vehicles, the stronger the interaction.

[0130] In some embodiments, in step S3, the vehicle space interaction module is an undirected graph, and the condition for the existence of an edge between two nodes is that the Euclidean distance between the two nodes is less than a preset distance threshold.

[0131] In the traffic scene, each vehicle is regarded as a node v in the graph structure, v = {v1, v2, v3, ..., v N}, subscript N is the total number of vehicles in the scene, and the condition for defining the existence of an edge between two nodes is that the Euclidean distance between the two nodes is less than a pre-set distance threshold d thresh , then the spatial relative position relationship between vehicles is regarded as an edge.

[0132] In some embodiments, in step S3, the spatial interaction weights between surrounding vehicles are iteratively calculated in a graph structure modeling manner, including:

[0133] Each vehicle is regarded as a node v in the graph structure, v={v1,v2,v3,...,v N}, N is the total number of weekly vehicles;

[0134] a. Establish spatial interaction based on graph structure in the scenario of time step t and total number of vehicles N. Define any node v in the undirected graph G i and node v j The symmetric adjacency matrix of is A(i,j), as shown in formula (4):

[0135]

[0136] in, Represents node v i With node v j The spatial interaction relationship between thresh is the distance threshold; of course, v iRepresents vehicle i as a node in the undirected graph G, v j Represent another vehicle different from vehicle i as another node in the undirected graph G;

[0137] b. Define the non-regularized Laplacian matrix L=DA of the undirected graph G as a symmetric matrix as shown in formula (5):

[0138]

[0139] Where D(i,j) is the main diagonal of the degree matrix D of the undirected graph G, The rest of the elements are all 0;

[0140] c. The Laplace matrix of each time step is related to the Laplace matrix of the past time step. The Laplace matrix defined at time step t is expressed as L t , then the Laplace matrix L of the next time step t+1 The update method is as follows:

[0141]

[0142] Among them, ff trans represents the outer product of rank 2, f is a sparse matrix and satisfies ||f||0<<N. Each time a new non-zero entry appears in the j-th row of f, it means that a new weekly vehicle has been added to the current undirected graph G. The maximum number of nodes in the undirected graph G is the total number of weekly vehicles N.

[0143] S4, builds a decoder of the motion prediction model and outputs Bezier curve control points: concatenates the aggregated hidden states between the pericars and the spatial interaction weights between the pericars, regresses the bivariate Gaussian distribution of the control points under the preset Bezier curve control point output paradigm, obtains the parameters describing the bivariate Gaussian distribution, and randomly samples a preset number of control points from the bivariate Gaussian distribution for each pericar.

[0144] In this step, in order to reliably predict the motion state of the pericar and its high-order quantities for reference by the downstream decision-making and planning modules, a model output paradigm based on Bezier curve control points is proposed. By splicing the aggregated hidden state between the pericars obtained in step S2 and the spatial interaction weights between the pericars obtained in step S3, and under the Bezier curve control point output paradigm of this step, the bivariate Gaussian distribution of the control points is finally regressed to obtain the parameter μ describing the bivariate Gaussian distribution. x 、μ y , σ x , σ yand ρ, representing the x-mean, y-mean, x-standard deviation, y-standard deviation, and correlation coefficient of the distribution, respectively. A preset number of control points are randomly sampled from this distribution. The number of control points is determined by the Bézier curve. For an n-th order Bézier curve, n+1 control points are sampled. For example, if a 5th order Bézier curve is predetermined, six control points are randomly sampled from a bivariate Gaussian distribution.

[0145] In some embodiments, in step S4, the formula of the n-th order Bezier curve is as shown in formula (7):

[0146]

[0147] in, represents the Bessel basis function of order i, represents the binomial coefficient, η represents the Bessel proportional coefficient, P i Represents the control points of the Bezier curve. For a Bezier curve of order n, there are n+1 control points.

[0148] It should be noted that since η∈[0,1], the time step values in the prediction domain are also normalized, as shown in formula (8):

[0149]

[0150] Among them, t f is any time step in the prediction domain, T f Indicates the predicted sequence length;

[0151] Then formula (7) can be rewritten as formula (9):

[0152]

[0153] At this point, the formula for the n-th order Bezier curve is obtained as shown in formula (9).

[0154] In some embodiments, in step S4, when regressing the binary Gaussian distribution of the control points, a preset multi-layer perceptron (MLP) is used to regress the binary Gaussian distribution of the Bezier curve control points to obtain the parameter μ describing the binary Gaussian distribution. x 、μ y , σ x , σ y and ρ, respectively, represent the x-mean of the ordinate, the y-mean of the abscissa, the x-standard deviation of the ordinate, the y-standard deviation of the abscissa, and the correlation coefficient of the bivariate Gaussian distribution.

[0155] S5, calculate the Bezier curve and obtain the motion state and its high-order quantities: According to the control points of each pericyte, calculate the Bezier curve. The Bezier curve is the predicted trajectory of the pericyte. According to the derivation of the Bezier curve, the high-order quantities of the motion state of the pericyte in the predicted time domain are obtained.

[0156] The Bezier curve of each circumferential vehicle can be obtained by using the control points of each circumferential vehicle obtained in step S4.

[0157] High-order quantities of the motion state include at least one of velocity, acceleration, and jerk. Since the derivative of a Bezier curve is still a Bezier curve, the high-order quantities of the motion state can be obtained by taking the first-order, second-order, or higher-order derivatives of the Bezier curve representing the trajectory. For example, taking the first-order derivative of the Bezier curve yields a Bezier curve representing velocity, taking the second-order derivative of the Bezier curve yields a Bezier curve representing acceleration, and taking the third-order derivative of the Bezier curve yields a Bezier curve representing jerk. Similarly, high-order quantities of the motion state of the perimeter vehicle within the prediction time domain can be obtained by taking derivatives of the Bezier curve.

[0158] In some embodiments, when calculating the Bezier curve through the control points of each perimeter obtained in step S4, the mathematical property of the curve, that is, the higher-order derivative of the Bezier curve is still the Bezier curve, is used as the control point expression as shown in formula (10), where the subscript i indicates the i-th order and the superscript indicates the first-order derivative of the control point of that order; the higher-order derivative expression of the Bezier curve is shown in formula (11):

[0159]

[0160] In some embodiments, after the control points of each periscope are obtained in step S4, the position coordinates in the motion state can be obtained by formula (12):

[0161]

[0162] Among them, Y pos Represents the position coordinates, B represents the constant Bessel basis matrix, and P represents the predicted control point coordinates.

[0163] In formula (12), the Bessel basis matrix dimension is T f ×(n+1), the dimension of the control point coordinates is (n+1)×2. After multiplying according to formula (12), the trajectory coordinate dimension is T f ×2, which means that in the prediction time domain T f The 2D position coordinates of the pericycle motion state within.

[0164] Based on this, we can further obtain the high-order motion state quantities required by the downstream decision-making and planning modules. The high-order motion state quantities can be: speed, acceleration, jerk, etc. For example, after taking the first-order derivative of the Bezier curve and rewriting formula (12), we can obtain the speed information of the surrounding vehicles at any time step in the prediction time domain. After taking the second-order derivative, we can obtain the acceleration information of the surrounding vehicles at any time step in the prediction time domain, which can be used in the steps of the driving risk warning method.

[0165] Steps S2 to S5 constitute the pericar motion prediction process. Through steps S1 to S5, the predicted trajectory (i.e., motion state) and high-order motion state quantities of each pericar are finally obtained.

[0166] An embodiment of the present invention further provides a driving risk warning method, which includes the following steps:

[0167] S6, determine the driving risk of the ego vehicle: calculate the collision probability between the ego vehicle and the surrounding vehicles in the predicted time domain based on the parameters of the binary Gaussian distribution of the surrounding vehicles obtained in step S4, and calculate the expected collision severity of the ego vehicle based on the high-order motion state variables obtained in step S5. The product of the collision probability and the expected collision severity of the ego vehicle is the driving risk of the ego vehicle.

[0168] The driving risk of the ego vehicle is calculated by multiplying two factors: the collision probability between the ego vehicle and surrounding vehicles within the prediction time domain and the expected collision severity of the ego vehicle. The collision probability between the ego vehicle and surrounding vehicles and the expected collision severity of the ego vehicle within the prediction time domain can be calculated simultaneously or one before the other.

[0169] In some embodiments, the parameters of the surrounding vehicle in the bivariate Gaussian distribution obtained in step S4 are used to calculate the collision probability between the ego vehicle and the surrounding vehicle in the predicted time domain, including:

[0170] The parameters of the bivariate Gaussian distribution of the pericycle obtained in step S4 are given by formula (13):

[0171]

[0172] Among them, the parameter μ of the bivariate Gaussian distribution is x 、μ y , σ x , σ y and ρ, respectively represent the x-mean, y-mean, x-standard deviation, y-standard deviation, and correlation coefficient of the bivariate Gaussian distribution. f is any time step in the prediction domain, It is the Gaussian distribution representation of the two-dimensional motion of the pericycle.

[0173] The collision probability between the ego vehicle and surrounding vehicles in the predicted time domain is also divided into two scenarios: the ego vehicle is driven by a human driver and the ego vehicle is in an autonomous driving state.

[0174] When the ego vehicle is driven by a human driver, there is also subjective driving uncertainty. For this reason, an additional bivariate Gaussian distribution is set to describe the subjective uncertainty brought by the human driver. The parameters of this bivariate Gaussian distribution are consistent with the two-dimensional Gaussian distribution describing the motion of surrounding vehicles. Rewriting formula (13), the collision probability when the ego vehicle is driven by a human driver is calculated by formula (14):

[0175]

[0176] in, They are the parameter representations after introducing the binary Gaussian distribution of the subjective uncertainty of human drivers, and represents the standard deviation of the Gaussian distribution of subjective uncertainty of human drivers, which is preset;

[0177] When the ego vehicle is in the autonomous driving state, the motion state (position, speed, etc.) of the ego vehicle in the prediction time domain can be provided by the autonomous driving planning module. A rectangle is used to describe the occupancy of the ego vehicle and surrounding vehicles. The length and width information of the ego vehicle are known, and the length and width information of the surrounding vehicles are provided by the target detection module and the target tracking module in step S1. The collision probability when the ego vehicle is in the autonomous driving state is calculated by formula (15):

[0178]

[0179] in, x , and is the vertical and horizontal coordinates of the vehicle at any time step in the prediction domain, and Provided by the preset autonomous driving planning module, L sub and L sur is the length of the vehicle and the length of the surrounding vehicle, W sub and W sur is the width of the vehicle and the surrounding vehicles, L sub and W sub is the default value, L sur and W sur The vehicle-mounted visual data collected in step S1 is obtained through target detection.

[0180] In some embodiments, the high-order motion state variables obtained in step S5 are used to calculate the expected collision severity of the ego-vehicle, including:

[0181] The expected collision severity of the ego vehicle is calculated by formula (16):

[0182] s(t+Δt)=0.5Mβ 2 (Δv(t+Δt)) 2 (16)

[0183] Among them, M is the weight of the vehicle, M is the preset value, β is the weight ratio of the surrounding vehicle, M sur is the weight of the car, M sur The empirical value is obtained according to a preset empirical formula after the vehicle type and length and width information of the surrounding vehicles are obtained through target detection using the on-board visual data collected in step S1. Δv(t+Δt) is the speed difference between the ego vehicle and the surrounding vehicles between time step t and time increment Δt. The ego vehicle speed is provided by the preset autonomous driving planning module, and the surrounding vehicle speed is obtained by taking the first-order derivative of the Bezier curve obtained in step S5.

[0184] S7, determining whether to issue a risk warning: comparing the driving risk with a preset risk threshold. If the driving risk is greater than the risk threshold, a risk warning is issued.

[0185] In some embodiments, the risk threshold is a dynamic risk threshold, and the dynamic risk threshold has corresponding different risk levels. The risk levels are obtained through subjective driving risk cognition experiments in different driving scenarios.

[0186] Factors influencing the driver's subjective driving risk perception include but are not limited to the driver's driving style, driving skills, and physical and mental state. For this purpose, a subjective driving risk perception experiment was conducted, and the risk level of the driving scenario was used as a quantitative indicator of the driver's subjective risk perception results, and a corresponding risk threshold was set for each risk level.

[0187] Risk levels can be divided into multiple levels, such as Level 2, Level 3, Level 4, or more, with each level corresponding to a risk threshold. For example, if the risk level is divided into four levels (absolutely safe, relatively safe, relatively dangerous, and absolutely dangerous), the corresponding risk thresholds are 50 joules, 40 joules, 30 joules, and 10 joules.

[0188] The risk level can be manually determined according to different driving scenarios. When the driving risk is greater than the risk threshold corresponding to the risk level, a risk warning will be issued.

[0189] In some embodiments, the subjective driving risk perception experiment is performed in the following manner.

[0190] Drivers of varying driving experience and gender were recruited to participate in a subjective risk perception data collection experiment. This experiment was conducted using a driving simulator. Driving scenarios included highways and urban roads, with starting and ending points set within each scenario. Participants were required to complete designated sections within the corresponding scenarios based on their daily driving habits. Finally, driving data from each participant was collected, including driving videos, the location, speed, and acceleration of surrounding vehicles, and more.

[0191] Collect participants' subjective risk perception results: Have each driver participating in the experiment watch a driving video other than their own (the driving video can be divided into several segments) and provide corresponding risk assessment quantitative indicators; to prevent participants from habituating their subjective evaluations, the order of the videos will be randomly disrupted to ensure that participants do not watch consecutive segments from the same driving video continuously; at the same time, to reduce the error in the subjective risk assessment results, participants will watch and evaluate two rounds of driving videos.

[0192] An embodiment of the present invention further provides a vehicle motion prediction and driving risk warning device, the vehicle motion prediction and driving risk warning device comprising:

[0193] The data acquisition and preprocessing module is used to collect vehicle-mounted visual data, serialize the vehicle-mounted visual data for each surrounding vehicle along the acquisition time sequence, obtain sequence data with several segments, and use the sequence data to train the preset motion prediction model to obtain the motion prediction model corresponding to each surrounding vehicle;

[0194] The peri-vehicle motion prediction module is used to construct an encoder for the motion prediction model. An encoder is assigned to each peri-vehicle to obtain the aggregated hidden state between the peri-vehicles. A vehicle spatial interaction module based on a graph structure is constructed with each vehicle as a node and the spatial interaction relationship between vehicles as an edge. The spatial interaction weights between the peri-vehicles are iteratively calculated using a graph structure modeling method. A decoder for the motion prediction model is constructed to concatenate the aggregated hidden state between the peri-vehicles and the spatial interaction weights between the peri-vehicles. Under the preset Bezier curve control point output paradigm, the binary Gaussian distribution of the control points is regressed to obtain the parameters describing the binary Gaussian distribution. A preset number of control points are randomly sampled from the binary Gaussian distribution for each peri-vehicle. Based on the control points of each peri-vehicle, a Bezier curve is calculated. The Bezier curve is the predicted trajectory of the peri-vehicle. The high-order quantity of the motion state of the peri-vehicle in the predicted time domain is obtained by derivation of the Bezier curve.

[0195] The driving risk warning module is used to calculate the collision probability between the vehicle and surrounding vehicles within the predicted time domain, calculate the expected collision severity of the vehicle, and take the product of the collision probability and the expected collision severity of the vehicle as the driving risk of the vehicle; compare the driving risk with the preset risk threshold, and issue a risk warning if the driving risk is greater than the risk threshold.

[0196] Example 1:

[0197] Taking the following intelligent car road test experiment as an example, the effectiveness of the vehicle motion prediction method and driving risk warning method proposed in the present invention from a vehicle-mounted perspective is illustrated.

[0198] The first step is to calibrate the on-board visual sensor and collect on-board visual data.

[0199] First, the visual sensor must be placed in the smart car. In this example, a binocular camera is used. The placement shown in Figure 2(a) is a suitable approach to acquire onboard visual data. Zhang's calibration method and the Matlab Camera Calibration Toolbox are used to calibrate the visual sensor's internal and external parameters. After calibration, the smart car is driven along the road section marked with blue lines in Figure 2(b) to collect data.

[0200] The second step is to obtain the surrounding vehicle information based on the target detection module and the target tracking module.

[0201] Based on the data collected in the first step, this embodiment uses the single-stage target detection algorithm YOLOv8 to detect surrounding vehicles. This algorithm is a commonly used target detection algorithm in the industry. The positioning and classification of surrounding vehicles on the image are obtained. The target detection effect of one frame is as follows: Figure 3 As shown, the orange rectangular frames are target detection frames, each containing information about the length, width, and center point of the rectangle. The depth information of the surrounding vehicles is estimated by combining the target detection frame information of surrounding vehicles and the principle of monocular ranging, thereby obtaining the distance of the surrounding vehicles relative to the vehicle. The target tracking module obtains the surrounding vehicle number information. In this embodiment, the Deep SORT (Deep Learning based Object Tracking and Data Association for Multi-Object Tracking) algorithm, a deep learning-based multi-target tracking algorithm, is used to track the surrounding vehicles and assign each vehicle a unique vehicle number.

[0202] In the third step, the output results obtained in the second step are saved in a CSV file for integration, and serialized according to the vehicle number and along the collection time sequence.

[0203] In this example, each segment of data processed through steps three and four is serialized using a sliding window method. The sliding window is set to 5 seconds, meaning the sequence length is also 5 seconds, with the first 2 seconds being the observed sequence and the last 3 seconds being the predicted sequence. Furthermore, this example uses a manual inspection method to handle outliers, discarding any sequence data that contains outliers.

[0204] The fourth step is to evenly divide the sequence data obtained in the third step based on the K-fold cross-validation method.

[0205] In this example, K=5 is set, meaning one set is selected as the validation set and the other four sets are selected as the training set. Models are trained on the datasets obtained from the five partitioning methods, and the performance of each model is evaluated using evaluation metrics. The FDE (FDE) and ADE (ADE) evaluation metrics are used for evaluation. Of course, other existing evaluation metrics can also be used to evaluate the model.

[0206] In the fifth step, the encoder of the motion prediction model extracts sequence data features.

[0207] Each vehicle in the scene is assigned an LSTM-based encoder, and the aggregated hidden state between the surrounding vehicles is obtained according to formula (3). In this embodiment, the hidden state aggregation method between the surrounding vehicles is selected as addition, and the dimension of the hidden state is set to [N, T h , d model ], where N is the total number of weekly vehicles in this scenario, T h is the observation sequence length. Since the sampling frequency in this embodiment is 10 Hz and the observation time is 3 seconds, T h =30,d model = 128 is the specified hidden state dimension.

[0208] The sixth step is to build a vehicle-space interaction module based on graph structure.

[0209] In order to reflect the spatial interaction relationship between surrounding vehicles in the actual traffic environment, an undirected graph is established with vehicles as nodes and the spatial interaction relationship between vehicles as edges. In this embodiment, the spatial interaction relationship between vehicles is the relative distance between vehicles, that is, Figure 4 The relative distance between car A1 and cars A2 to A5. The condition for the spatial interaction between the two nodes is that the relative distance between the two nodes is less than the artificially set distance threshold d thresh , where the distance threshold is set to 100 meters. The spatial interaction weights between the surrounding vehicles are iteratively calculated using a graph structure modeling approach. In this embodiment, the corresponding calculations are performed according to formulas (4), (5), and (6), and the spatial interaction weights between the surrounding vehicles are finally obtained. The physical meaning of this weight is that at time step t, the smaller the relative distance between the two vehicles, the stronger the interaction.

[0210] Step 7: Build the decoder part of the motion prediction model and output the Bezier curve control points.

[0211] Unlike most previous models that directly output the future motion state of surrounding vehicles, this embodiment outputs Bezier curve control points. Control points have specific spatial meanings. For an n-order Bezier curve, there are n+1 control points. This embodiment selects a 5th-order Bezier curve to describe the future trajectory of surrounding vehicles and predicts the time domain length T f = 20, meaning the curve requires 6 control points. In the decoder, the aggregated hidden states between the surrounding vehicles obtained in step 5 and the spatial interaction weights between the surrounding vehicles obtained in step 6 are concatenated, with the dimension being 128. A binary Gaussian distribution is used to describe the uncertainty of future motion states. A multi-layer perceptron receives the concatenated results and outputs the five parameters μ of a two-dimensional Gaussian distribution describing the uncertainty of the surrounding vehicle motion at each time step. x 、μ y , σ x , σ y and ρ, the output dimension is [N, 20, 5]. Finally, each pericar is randomly sampled 6 times in this two-dimensional Gaussian distribution to obtain the 6 control points of each pericar.

[0212] Step 8: Calculate the Bezier curve and obtain the motion state and its high-order quantities.

[0213] The seventh step obtains the control point information of the surrounding vehicle in the prediction time domain, and a Bezier curve can be obtained. This embodiment uses formula (12) to calculate the Bezier curve coordinates corresponding to the control point, that is, the predicted trajectory of the surrounding vehicles. At the same time, according to formulas (10) and (11), the first-order derivative of the Bezier curve can be obtained and then rewritten into formula (12) to obtain the speed information of the surrounding vehicles at any time step in the prediction time domain. After obtaining the second-order derivative, the acceleration information of the surrounding vehicles at any time step in the prediction time domain can be obtained for use in the steps of driving risk warning.

[0214] The ninth step is to calculate the components of driving risk, namely the collision probability between the vehicle and surrounding vehicles in the predicted time domain and the expected collision severity of the vehicle.

[0215] Regarding the collision probability between the ego vehicle and surrounding vehicles within the prediction time domain, in this embodiment, it is assumed that the ego vehicle is driven by a human driver. Therefore, formula (14) is used to take the driver's subjective uncertainty into account when calculating the collision probability. The length and width of the ego vehicle used in this embodiment are 4.88 meters and 1.82 meters, respectively. The length and width information of the surrounding vehicles are provided by the target detection module. All weights are based on the curb weight. The weight of the sedan is set to 1200 kg, and the weight of the truck is set to 6000 kg based on the medium-sized truck. Finally, the collision probability is calculated according to formula (15). Regarding the expected collision severity of the ego vehicle, according to formula (16), the velocity information of the surrounding vehicles at any time step within the prediction time domain can be obtained from the motion prediction results of the surrounding vehicles, and the expected collision severity of the ego vehicle is calculated.

[0216] The tenth step is to assess the driving risk of the vehicle and determine whether to issue a risk warning.

[0217] First, the driving risk of the ego vehicle is calculated. The driving risk of the ego vehicle is the product of the collision probability between the ego vehicle and surrounding vehicles in the prediction time domain and the expected collision severity of the ego vehicle.

[0218] Secondly, a driver's subjective risk perception experiment was conducted. In this example, 18 participants (3 male and 3 female drivers with different driving experience (1, 3, and 5 years)) were recruited to participate in the experiment. They drove for 30 minutes on a highway and urban road scenarios on a driving simulator, and ultimately collected a total of 36 driving videos. The videos were sliced into 5-minute time windows, and each participant was asked to watch a driving video other than their own and conduct a subjective risk assessment.

[0219] In this embodiment, four levels of scenario risk are set, namely, absolute safety, relative safety, relative danger, and absolute danger. A corresponding driving risk threshold is set for each risk level. The risk threshold corresponding to each risk level is determined based on the results of the subjective risk perception experiment. The driving risk threshold in the absolutely safe scenario is 50 joules, the driving risk threshold in the relatively safe scenario is 40 joules, the driving risk threshold in the relatively dangerous scenario is 30 joules, and the driving risk threshold in the absolutely dangerous scenario is 10 joules.

[0220] Finally, the driver selects a risk level based on their driving habits and driving scenarios. The risk threshold corresponding to that risk level becomes the dynamic risk threshold for the current driving scenario. When the driver's driving risk exceeds the dynamic risk threshold for the current driving scenario, a risk warning is issued.

[0221] The present invention has been described in detail above with reference to the embodiments of the accompanying drawings. A person skilled in the art can make various modifications to the present invention based on the above description. Therefore, certain details in the embodiments should not be construed as limiting the present invention. The scope of protection of the present invention shall be determined by the scope defined by the appended claims.

Claims

1. A vehicle motion prediction method, characterized in that: The vehicle motion prediction method comprises: S1, collecting vehicle-mounted visual data, serializing the vehicle-mounted visual data for each surrounding vehicle along the collection time sequence to obtain sequence data having a plurality of segments, and using the sequence data to train a preset motion prediction model to obtain a motion prediction model corresponding to each surrounding vehicle; S2, constructing an encoder of the motion prediction model, assigning an encoder to each pericar respectively, and obtaining the aggregated hidden state between the pericars; S3, with each vehicle as a node and the spatial interaction relationship between vehicles as an edge, builds a vehicle spatial interaction module based on a graph structure, and iteratively calculates the spatial interaction weights between surrounding vehicles using a graph structure modeling method; S4, constructing a decoder of the motion prediction model and outputting Bezier curve control points: concatenating the aggregated hidden states between the pericars obtained in step S2 and the spatial interaction weights between the pericars obtained in step S3, regressing the bivariate Gaussian distribution of the control points under a preset Bezier curve control point output paradigm to obtain parameters describing the bivariate Gaussian distribution, and randomly sampling a preset number of control points from the bivariate Gaussian distribution for each pericar; S5, calculating a Bezier curve based on the control points of each pericyte. The Bezier curve is the predicted trajectory of the pericyte. The high-order quantity of the motion state of the pericyte in the predicted time domain is obtained by derivation of the Bezier curve.

2. The vehicle motion prediction method according to claim 1, wherein: In step S1, vehicle-mounted visual data is collected, including: Arrange one or more visual sensors at preset positions on the vehicle, use a preset calibration algorithm to obtain internal and external parameters of the visual sensors, and after successful calibration, collect on-board visual data of the visual sensors; And / or, in step S1, after collecting the vehicle-mounted visual data, the method further includes: Obtaining surrounding vehicle information, the surrounding vehicle information including the distance between each surrounding vehicle and the own vehicle and the vehicle number corresponding to each surrounding vehicle; According to the different vehicle numbers, the vehicle-borne visual data is serialized for each vehicle along the collection time sequence to obtain sequence data; And / or, in step S1, using the sequence data to train a preset motion prediction model, obtaining a motion prediction model corresponding to each surrounding vehicle includes: The sequence data is evenly divided into K parts using the preset K-fold cross-validation method, one of which is selected as the validation set and K-1 parts as the training set. The first h seconds of each sequence are used as the observation sequence and the last Hh seconds are used as the prediction sequence, where H is the sequence length and h is less than H; The motion prediction model is trained using the training set, the trained motion prediction model is verified using the validation set, and an evaluation index is calculated. If the evaluation index reaches a preset expected index, the training of the motion prediction model is considered complete.

3. The vehicle motion prediction method according to claim 2, wherein: The obtaining of weekly vehicle information includes: Using a preset target detection module to perform target detection on the vehicle-mounted visual data, a target detection frame containing the position and category of each vehicle on each frame image is obtained; Estimate the surrounding vehicles based on the target detection frame and the visual sensor parameters to obtain the distance between the surrounding vehicles and the self-vehicle; Using a preset target tracking module to associate and track the surrounding vehicles detected at different times in the vehicle-mounted visual data, and assigning a corresponding vehicle number to each surrounding vehicle; and / or, when serializing the vehicle-borne visual data for each vehicle along the order of acquisition time, a sliding window method is used for serialization, where the sliding window is the sequence length; and / or, after serializing the vehicle-borne visual data for each vehicle along the order of acquisition time, outliers are removed to obtain sequence data; And / or, the evaluation index is one of the final displacement error and the average displacement error or a combination thereof, and the calculation formulas are respectively formula (1) and formula (2): Among them, the middle T f Indicates the length of the prediction sequence, the subscript t represents each time step in the prediction time series, and x t and y t are the ordinate and abscissa of the true value of the trajectory, and are the ordinate and abscissa of the trajectory prediction value respectively.

4. The vehicle motion prediction method according to claim 1, wherein: In step S2, an encoder is assigned to each surrounding vehicle to obtain the aggregated hidden state between the surrounding vehicles, including: An LSTM encoder is assigned to each of the pericycles in the observation sequence. The LSTM encoder is used to extract features from the historical motion state information in the observation sequence. The extracted features are output in the hidden state. The calculation formula for the aggregated hidden state between the pericycles is formula (3): in, represents the aggregated hidden state of vehicle i at time step t, j represents another vehicle different from vehicle i, N represents the total number of vehicles in the week, M j represents the hidden state aggregation process from vehicle j to vehicle i, is the hidden state of vehicle j at time step t, is the hidden state of vehicle i at time step t; The hidden state aggregation process adopts one or a combination of concatenation, addition and multiplication methods.

5. The vehicle motion prediction method according to claim 1, wherein: In step S3, the vehicle space interaction module is an undirected graph, and the condition for the existence of an edge between two nodes is that the Euclidean distance between the two nodes is less than a preset distance threshold; And / or, in step S3, iteratively calculating the spatial interaction weights between surrounding vehicles in a graph structure modeling manner, including: Each vehicle is regarded as a node v in the graph structure, v={v1,v2,v3,...,v N }, N is the total number of weekly vehicles; At time step t, establish spatial interaction based on graph structure and define any node v in the undirected graph G i and node v j The symmetric adjacency matrix of is A(i,j), as shown in formula (4): in, Represents node v i With node v j The spatial interaction relationship between thresh is the distance threshold; The non-regularized Laplacian matrix L = DA of the undirected graph G is defined as a symmetric matrix as shown in formula (5): Where D(i,j) is the main diagonal of the degree matrix D of the undirected graph G, The rest of the elements are all 0; The Laplace matrix defined at time step t is represented as L t , then the Laplace matrix L of the next time step t+1 The update method is as follows: where, ff trans represents an outer product of rank 2, f is a sparse matrix and satisfies ||f||0 < < N. Each time a new non-zero term appears in the j-th row of f, it represents that a new Zhou vehicle is added to the current undirected graph G. The maximum number of nodes in the undirected graph G is the total number of Zhou vehicles N.

6. The vehicle motion prediction method according to claim 1, wherein: In step S4, the formula of the n-th order Bezier curve is as shown in formula (9): in, represents the Bessel basis function of the i-th order, η represents the Bessel scale factor, P i Indicates the control point of the Bezier curve, t f is any time step in the prediction domain, T f represents the predicted sequence length, And / or, in step S4, when regressing the binary Gaussian distribution of the control points, a preset multi-layer perceptron is used to regress the binary Gaussian distribution of the Bezier curve control points to obtain the parameter μ describing the binary Gaussian distribution. x 、μ y , σ x , σ y and ρ, respectively, represent the x-mean of the ordinate, the y-mean of the abscissa, the x-standard deviation of the ordinate, the y-standard deviation of the abscissa, and the correlation coefficient of the bivariate Gaussian distribution.

7. The vehicle motion prediction method according to claim 6, wherein: In step S5, the Bezier curve coordinates are given by formula (12): Among them, Y pos represents the position coordinates, B represents the constant Bessel basis matrix, and P represents the predicted control point coordinates; And / or, in step S5, the high-order quantity of the motion state includes at least one of velocity, acceleration and jerk, and the high-order quantity of the motion state is obtained by calculating the high-order derivative of the Bezier curve.

8. A driving risk warning method, characterized in that: The driving risk warning method includes: Calculate the collision probability between the ego vehicle and the surrounding vehicles in the predicted time domain using the parameters of the bivariate Gaussian distribution of the surrounding vehicles obtained by the vehicle motion prediction method according to any one of claims 1 to 7; calculate the expected collision severity of the ego vehicle using the high-order quantity of the motion state obtained by the vehicle motion prediction method according to any one of claims 1 to 7; and use the product of the collision probability and the expected collision severity of the ego vehicle as the driving risk of the ego vehicle; The driving risk is compared with a preset risk threshold, and if the driving risk is greater than the risk threshold, a risk warning is issued.

9. The driving risk warning method according to claim 8, characterized in that: Calculate the collision probability between the vehicle and surrounding vehicles in the prediction time domain, including: The parameters of the bivariate Gaussian distribution of Zhouche are given by formula (13): Among them, the parameter μ of the bivariate Gaussian distribution is x 、μ y , σ x , σ y and ρ, respectively represent the x-mean, y-mean, x-standard deviation, y-standard deviation, and correlation coefficient of the bivariate Gaussian distribution. f is any time step in the prediction domain, It is the Gaussian distribution representation of the two-dimensional motion of the pericycle; When the ego vehicle is driven by a human driver, there is also subjective driving uncertainty. Formula (13) is rewritten and the collision probability is calculated by formula (14): in, They are the parameter representations after introducing the binary Gaussian distribution of the subjective uncertainty of human drivers, and represents the standard deviation of the Gaussian distribution of subjective uncertainty of human drivers; When the vehicle is in the automatic driving state, the collision probability is calculated by formula (15): in, and is the vertical and horizontal coordinates of the vehicle at any time step in the prediction domain, and Provided by the preset autonomous driving planning module, L sub and L sur is the length of the vehicle and the length of the surrounding vehicle, W sub and W sur is the width of the vehicle and the surrounding vehicles, L sub and W sub is the default value, L sur and W sur Obtained by target detection from the vehicle-mounted visual data collected in step S1; And / or, calculate the expected collision severity of the ego vehicle, including: The expected collision severity of the ego vehicle is calculated by formula (16): s(t+Δt)=0.5Mβ 2 (Δv(t+Δt)) 2 (16) Where M is the weight of the vehicle, M is the preset value, β is the weight ratio of the surrounding vehicle, M sur is the weight of the car, M sur The empirical value is obtained according to a preset empirical formula after the type and length and width information of the surrounding vehicles are obtained through target detection using the on-board visual data collected in step S1. Δv(t+Δt) is the speed difference between the ego vehicle and the surrounding vehicles between time step t and time increment Δt. The ego vehicle speed is provided by the preset autonomous driving planning module, and the surrounding vehicle speed is provided by the high-order motion state quantity obtained in step S5.

10. The driving risk warning method according to claim 8, characterized in that: The risk threshold is a dynamic risk threshold, and the dynamic risk threshold has corresponding different risk levels. The risk levels are obtained through subjective driving risk cognition experiments in different driving scenarios.

Images