Training of a pose angle prediction model, pose angle prediction method, device and equipment

Abstract

The application discloses a training method of a vehicle attitude angle prediction model, a vehicle attitude angle prediction method, device and equipment, and is used for realizing real-time and accurate vehicle attitude angle prediction. The training method comprises the following steps: collecting vehicle operation data, wherein the vehicle operation data comprises a vehicle speed, vehicle position information and attitude information, the attitude information comprises an angle value of at least one Euler angle; decomposing the vehicle operation data into vehicle operation data corresponding to a target Euler angle, the target Euler angle being each Euler angle; intercepting, from the vehicle operation data corresponding to the target Euler angle, vehicle operation data corresponding to the target Euler angle at a continuous first number of time points and angle values corresponding to the target Euler angle at a continuous second number of time points as training samples; the continuous first number of time points and the continuous second number of time points form continuous time points; training an initial LSTM time sequence network by using the training samples, and generating a vehicle attitude angle prediction model corresponding to the target Euler angle.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a training method for a vehicle attitude angle prediction model, a vehicle attitude angle prediction method, apparatus, and equipment. Background Technology

[0002] During vehicle operation, various factors can cause changes in vehicle posture. For example, driver control, uneven road surfaces, going uphill or downhill, and passing speed bumps can all lead to changes in vehicle posture.

[0003] In some scenarios, it is necessary to detect vehicle attitude in order to further control the vehicle. Vehicle attitude can generally be characterized by vehicle attitude angles. Sensors can typically detect vehicle status in real time, including vehicle attitude angles. However, due to sensor errors and delays, it is impossible to obtain highly accurate vehicle attitude angles in real time. Summary of the Invention

[0004] In view of this, embodiments of this application provide a training method for a vehicle attitude angle prediction model, a vehicle attitude angle prediction method, apparatus, and device to achieve real-time acquisition of relatively accurate vehicle attitude angles.

[0005] To address the above problems, the technical solutions provided in this application are as follows:

[0006] In a first aspect, embodiments of this application provide a method for training a vehicle attitude angle prediction model, the method comprising:

[0007] Collect vehicle operation data, which includes vehicle speed, vehicle position information, and attitude information, wherein the attitude information includes at least one Euler angle value.

[0008] The vehicle operation data is decomposed into vehicle operation data corresponding to target Euler angles, wherein the target Euler angles are each of the Euler angles.

[0009] The vehicle operation data corresponding to the target Euler angle at a consecutive first number of time intervals and the angle values ​​corresponding to the target Euler angle at a consecutive second number of time intervals are extracted from the vehicle operation data corresponding to the target Euler angle as training samples; the consecutive first number of time intervals and the consecutive second number of time intervals constitute consecutive time intervals.

[0010] The initial LSTM temporal network is trained using the training samples to generate a vehicle attitude angle prediction model corresponding to the target Euler angle.

[0011] In one possible implementation, the collection of vehicle operation data includes:

[0012] Acquire vehicle speed and attitude information collected by onboard sensors;

[0013] It acquires positioning data collected through the Global Positioning System, image data collected by cameras, and radar data collected by radar.

[0014] The positioning data is corrected based on the image data and / or the radar data to obtain vehicle position information; the vehicle speed, the vehicle position information, and the attitude information constitute vehicle operation data.

[0015] In one possible implementation, before decomposing the vehicle operation data into vehicle operation data corresponding to the target Euler angles, the method further includes:

[0016] The vehicle operation data is divided into vehicle operation data for multiple time periods;

[0017] The vehicle operation data for each time period are aligned with the coordinate system to generate aligned vehicle operation data for each time period, and the aligned vehicle operation data for each time period is redefined as vehicle motion data.

[0018] In one possible implementation, the step of training an initial LSTM temporal network using the training samples to generate a vehicle attitude angle prediction model corresponding to the target Euler angle includes:

[0019] The vehicle operation data corresponding to the target Euler angle at the first consecutive number of time steps is input into the initial LSTM time series network to obtain the predicted angle value corresponding to the target Euler angle at the second consecutive number of time steps.

[0020] The loss value is calculated based on the predicted angle value corresponding to the target Euler angle at the second consecutive number of time steps and the angle value corresponding to the target Euler angle at the second consecutive number of time steps.

[0021] The initial LSTM timing network is adjusted based on the loss value;

[0022] The process of repeatedly inputting the vehicle operation data corresponding to the target Euler angle at the first consecutive time step into the initial LSTM temporal network and subsequent steps continues until the preset conditions are met, thereby generating the vehicle attitude angle prediction model corresponding to the target Euler angle.

[0023] Secondly, embodiments of this application provide a method for predicting vehicle attitude angles, the method comprising:

[0024] Acquire real-time vehicle operation data for a first consecutive number of time intervals. The real-time vehicle operation data includes vehicle speed, vehicle position information, and attitude information. The attitude information includes at least one Euler angle value.

[0025] The real-time vehicle operation data of the first consecutive number of time steps are input into the vehicle attitude angle prediction model corresponding to the target Euler angle to obtain the predicted angle value of the target Euler angle of the second consecutive number of time steps; the target Euler angle is each Euler angle, the first consecutive number of time steps and the second consecutive number of time steps constitute a consecutive time step, and the vehicle attitude angle prediction model corresponding to the target Euler angle is generated by training the vehicle attitude angle prediction model of the above-mentioned training method.

[0026] In one possible implementation, the method further includes:

[0027] The predicted angle value of the target Euler angle at the current moment is obtained from the predicted angle values ​​of the target Euler angle at the second consecutive number of time moments;

[0028] The projection matrix of the vehicle projection system is adjusted based on the predicted angle value of the target Euler angle at the current moment.

[0029] Thirdly, embodiments of this application provide a training apparatus for a vehicle attitude angle prediction model, the apparatus comprising:

[0030] The acquisition unit is used to acquire vehicle operation data, which includes vehicle speed, vehicle position information and attitude information, and the attitude information includes at least one Euler angle value.

[0031] A decomposition unit is used to decompose the vehicle operation data into vehicle operation data corresponding to target Euler angles, wherein the target Euler angles are each of the Euler angles.

[0032] The interception unit is used to intercept the vehicle operation data corresponding to the target Euler angle for a first consecutive number of time moments and the angle value corresponding to the target Euler angle for a second consecutive number of time moments from the vehicle operation data corresponding to the target Euler angle as training samples; the first consecutive number of time moments and the second consecutive number of time moments constitute a continuous time moment;

[0033] The training unit is used to train an initial LSTM temporal network using the training samples to generate a vehicle attitude angle prediction model corresponding to the target Euler angle.

[0034] Fourthly, embodiments of this application provide a vehicle attitude angle prediction device, the device comprising:

[0035] The acquisition unit is used to acquire real-time vehicle operation data for a first number of consecutive time intervals. The real-time vehicle operation data includes vehicle speed, vehicle position information, and attitude information. The attitude information includes at least one Euler angle value.

[0036] The prediction unit is used to input the real-time vehicle operation data of the first consecutive number of time steps into the vehicle attitude angle prediction model corresponding to the target Euler angle, and obtain the predicted angle value of the target Euler angle for the second consecutive number of time steps; the target Euler angle is each Euler angle, the first consecutive number of time steps and the second consecutive number of time steps constitute a continuous time step, and the vehicle attitude angle prediction model corresponding to the target Euler angle is trained and generated according to the above-mentioned training method of the vehicle attitude angle prediction model.

[0037] Fifthly, embodiments of this application provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the training method for the vehicle attitude angle prediction model as described above, or the vehicle attitude angle prediction method as described above.

[0038] Sixthly, embodiments of this application provide a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform the training method for the vehicle attitude angle prediction model described above, or the vehicle attitude angle prediction method described above.

[0039] Therefore, the embodiments of this application have the following beneficial effects:

[0040] This application collects a large amount of vehicle operation data, including vehicle speed, vehicle position information, and attitude information, and then breaks down the vehicle operation data into vehicle operation data corresponding to each Euler angle. For the vehicle operation data corresponding to each Euler angle, vehicle operation data for T consecutive time moments and the angle values ​​of that Euler angle for the next N time moments are extracted as training samples to train an initial LSTM temporal network to generate a vehicle attitude angle prediction model corresponding to that Euler angle. Then, real-time vehicle operation data for T consecutive time moments is collected and input into the vehicle attitude angle prediction model corresponding to a certain Euler angle, which can then predict the predicted angle values ​​of that Euler angle for the next N time moments, achieving real-time prediction of vehicle attitude angles with high accuracy. Attached Figure Description

[0041] Figure 1 A schematic diagram illustrating an exemplary application scenario provided in this application embodiment;

[0042] Figure 2 A flowchart illustrating a training method for a vehicle attitude angle prediction model provided in this application embodiment;

[0043] Figure 3 A flowchart illustrating a vehicle attitude angle prediction method provided in this application embodiment;

[0044] Figure 4A schematic diagram of a training device for a vehicle attitude angle prediction model provided in an embodiment of this application;

[0045] Figure 5 This is a schematic diagram of a vehicle attitude angle prediction device provided in an embodiment of this application. Detailed Implementation

[0046] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the embodiments of this application will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0047] To facilitate understanding and explanation of the technical solutions provided in the embodiments of this application, the background technology of this application will be described first.

[0048] During vehicle operation, various factors can cause changes in vehicle posture. For example, driver control, uneven road surfaces, going uphill or downhill, and passing speed bumps can all lead to changes in vehicle posture.

[0049] In some scenarios, it is necessary to detect the vehicle's attitude in order to further control the vehicle. The vehicle's attitude can generally be characterized by its attitude angles.

[0050] For example, AR (Augmented Reality)-HUD (Head-Up Display) is an application that uses a combination of virtual and real technologies to project images onto the driver's line of sight, blending them with the real scene. Because AR-HUD projection needs to be blended with the real scene, changes in vehicle posture during operation can cause the projection to misalign with the real object. Therefore, it is necessary to predict various possible jitter scenarios, i.e., to detect the vehicle's attitude angle and provide image stabilization.

[0051] However, current methods of detecting vehicle attitude angles using sensors cannot obtain accurate vehicle attitude angles in real time due to sensor errors and delays. In specific application scenarios, such as AR-HUD scenarios, due to sensor errors and delays, the vehicle may have already entered a new attitude when image stabilization is performed, resulting in ineffective image stabilization or worse fit.

[0052] Based on this, embodiments of this application provide a training method for a vehicle attitude angle prediction model, a vehicle attitude angle prediction method, apparatus, and device. A large amount of vehicle operation data, including vehicle speed, vehicle position information, and attitude information, is collected, and then the vehicle operation data is broken down into vehicle operation data corresponding to each Euler angle. For each Euler angle, vehicle operation data at T consecutive time points and the angle values ​​of that Euler angle at the following N time points are extracted as training samples to train an initial LSTM temporal network to generate a vehicle attitude angle prediction model corresponding to that Euler angle. Then, real-time vehicle operation data at T consecutive time points is collected and input into the vehicle attitude angle prediction model corresponding to a certain Euler angle, which can then predict the predicted angle values ​​of that Euler angle at the following N time points, achieving real-time prediction of vehicle attitude angles with high accuracy. In specific application scenarios, such as in AR-HUD scenarios, using the real-time predicted vehicle attitude angles of this application embodiment for image stabilization can improve the display fit effect.

[0053] To facilitate understanding of the training method for the vehicle attitude angle prediction model and the vehicle attitude angle prediction method provided in the embodiments of this application, the following is combined with... Figure 1 The example scenario is shown below. See also... Figure 1 As shown in the figure, this figure is a schematic diagram of an exemplary application scenario provided in the embodiments of this application.

[0054] First, a large amount of vehicle operation data is collected. Each data point includes vehicle speed, position information, and attitude information, with the attitude information including at least one Euler angle. These Euler angles together constitute the vehicle attitude angles. In this example, we will use roll, pitch, and yaw angles as examples. Then, the vehicle operation data is decomposed into three datasets: data corresponding to roll angles, pitch angles, and yaw angles. For each dataset, T consecutive time intervals of vehicle operation data and N subsequent time intervals of angle values ​​are extracted as training samples to train three vehicle attitude angle prediction models: one for roll angles, one for pitch angles, and one for yaw angles.

[0055] In practical applications, by collecting real-time vehicle operation data for T consecutive time periods and inputting them into the three vehicle attitude angle prediction models mentioned above, the predicted roll angle, pitch angle, and yaw angle values ​​for the next N time periods can be obtained from the three vehicle attitude angle prediction models, thereby realizing the prediction of vehicle attitude angles.

[0056] Those skilled in the art will understand that Figure 1The schematic diagram shown is merely one example in which embodiments of this application can be implemented. The scope of application of the embodiments of this application is not limited by any aspect of this framework.

[0057] To facilitate understanding of this application, the following description, in conjunction with the accompanying drawings, illustrates a training method for a vehicle attitude angle prediction model provided in an embodiment of this application.

[0058] See Figure 2 As shown, this figure is a flowchart of a training method for a vehicle attitude angle prediction model provided in an embodiment of this application. Figure 2 As shown, the method may include S201-S204:

[0059] S201: Collect vehicle operation data, including vehicle speed, vehicle position information and attitude information, including at least one Euler angle value.

[0060] Vehicle-mounted equipment can collect a large amount of vehicle operation data. In practical applications, vehicle operation data can be collected continuously according to a sampling period. For example, if one piece of vehicle operation data is collected every 0.1 seconds, then 600 pieces of vehicle operation data can be collected in one minute. Each piece of vehicle operation data corresponds to a time value, which can be represented by a timestamp. Each piece of vehicle operation data includes vehicle speed, vehicle position information, and attitude information. Among them, attitude information includes at least one Euler angle value. Typically, Euler angles include roll angle, pitch angle, and yaw angle, and these Euler angles together constitute the vehicle attitude angles.

[0061] In subsequent embodiments, the specific implementation process of S201 collecting vehicle operation data will be described in detail, and will not be repeated here.

[0062] S202: Decompose the vehicle operation data into vehicle operation data corresponding to the target Euler angles, where the target Euler angles are each Euler angle.

[0063] Since each vehicle operation data point includes at least one Euler angle value, a set of vehicle operation data can be decomposed into at least one set of vehicle operation data, so that each set of vehicle operation data includes only one Euler angle value. In practical applications, if we define each Euler angle as a target Euler angle, then the vehicle operation data can be decomposed into the vehicle operation data corresponding to the target Euler angle.

[0064] For example, Euler angles include roll angle, pitch angle, and yaw angle. Vehicle operation data can be decomposed into vehicle operation data corresponding to roll angle, pitch angle, and yaw angle. Vehicle operation data corresponding to roll angle includes vehicle speed, vehicle position information, and the roll angle value; vehicle operation data corresponding to pitch angle includes vehicle speed, vehicle position information, and the pitch angle value; and vehicle operation data corresponding to yaw angle includes vehicle speed, vehicle position information, and the yaw angle value.

[0065] S203: Extract the vehicle operation data corresponding to the target Euler angle for a first consecutive number of time steps and the angle value corresponding to the target Euler angle for a second consecutive number of time steps from the vehicle operation data corresponding to the target Euler angle as training samples; the first consecutive number of time steps and the second consecutive number of time steps constitute a continuous time step.

[0066] For each target Euler angle, the vehicle operation data corresponding to the target Euler angle at T consecutive time points and the angle values ​​corresponding to the target Euler angle at N consecutive time points are extracted as training samples. Here, T and N are positive integers, generally T is greater than N, and the T consecutive time points are contiguous with the N consecutive time points. To achieve attitude angle prediction, in the training samples, the angle values ​​corresponding to the target Euler angle at N consecutive time points can be understood as labels for the vehicle operation data corresponding to the target Euler angle at T consecutive time points.

[0067] For example, if the target Euler angle is the roll angle, and the vehicle operation data corresponding to the roll angle includes 600 vehicle operation data points corresponding to the roll angle, with T set to 60 and N set to 10, the vehicle operation data from time 1 to time 60 and the roll angle values ​​from time 61 to time 70 can be selected as one training sample, and the vehicle operation data from time 2 to time 61 and the roll angle values ​​from time 62 to time 71 can be selected as another training sample, and so on, to obtain the training samples corresponding to the target Euler angle.

[0068] S204: Train an initial LSTM temporal network using training samples to generate a vehicle attitude angle prediction model corresponding to the target Euler angle.

[0069] Using training samples corresponding to the target Euler angles, the initial LSTM temporal network can be trained to obtain a vehicle attitude angle prediction model corresponding to the target Euler angles. For example, using training samples corresponding to the roll angle, a vehicle attitude angle prediction model corresponding to the roll angle can be obtained; using training samples corresponding to the pitch angle, a vehicle attitude angle prediction model corresponding to the pitch angle can be obtained; and using training samples corresponding to the yaw angle, a vehicle attitude angle prediction model corresponding to the yaw angle can be obtained.

[0070] LSTM (Long Short-Term Memory) temporal networks are essentially a specific form of RNN (Recurrent Neural Network). LSTM addresses the short-term memory problem of RNNs by adding gates, enabling RNNs to effectively utilize long-range temporal information. LSTM adds three logical control units—input gate, output gate, and forget gate—to the basic RNN structure, each connected to a multiplication element. By setting the weights at the edges connecting the memory cells to other parts of the network, it controls the input and output of information and the state of the memory cells, thus solving the gradient vanishing problem in RNNs. Through the control of these gates, LSTM achieves the ability to retain information for long periods.

[0071] By using the vehicle operation data corresponding to the target Euler angle at T consecutive time points in the training sample and the angle values ​​corresponding to the target Euler angle at N consecutive time points, the LSTM temporal network can learn the temporal information of the vehicle operation data. The trained vehicle attitude angle prediction model corresponding to the target Euler angle can predict the angle value of the target Euler angle.

[0072] In subsequent embodiments, the specific implementation process of S204 using training samples to train the initial LSTM temporal network and generate the vehicle attitude angle prediction model corresponding to the target Euler angle will be described in detail, and will not be repeated here.

[0073] Based on the descriptions in S201-S204 above, this embodiment of the application collects a large amount of vehicle operation data, including vehicle speed, vehicle position information, and attitude information. This vehicle operation data is then broken down into vehicle operation data corresponding to each Euler angle. For each Euler angle, vehicle operation data from T consecutive time periods and the angle values ​​of that Euler angle from the following N time periods are extracted as training samples to train an initial LSTM temporal network, generating a vehicle attitude angle prediction model corresponding to that Euler angle. Then, real-time vehicle operation data from T consecutive time periods is input into the vehicle attitude angle prediction model corresponding to a certain Euler angle, which can then predict the angle values ​​of that Euler angle from the following N time periods, achieving real-time prediction of vehicle attitude angles with high accuracy.

[0074] To achieve more accurate collection of vehicle operation data, in one possible implementation, the specific implementation of collecting vehicle operation data in step S201 may include steps A1-A3:

[0075] A1: Acquire vehicle speed and attitude information collected by onboard sensors.

[0076] In practical applications, vehicle speed can be collected using the vehicle speed sensor in the vehicle-mounted sensors, and attitude information can be collected using the gyroscope in the vehicle-mounted sensors.

[0077] A2: Acquire positioning data collected through the Global Positioning System, image data collected by cameras, and radar data collected by radar.

[0078] GPS (Global Positioning System) can be used to collect positioning data. Since GPS-collected positioning data generally contains errors, image data from vehicle-mounted cameras and radar data from radar can be used to correct the positioning data. In some implementations, positioning data can also be acquired using RTK (Real-time Kinematic) technology, which generally provides higher accuracy.

[0079] In practical applications, vehicle-mounted cameras can collect image data from the front of the vehicle. This image data can include the vehicle's lane, the lane colors and alignments of adjacent lanes, the distance from the vehicle to the lane boundary, and the distance from the vehicle to target landmarks. Radar equipment can also collect radar data from the front of the vehicle, such as millimeter-wave radar point cloud data. This radar data can determine the distance of radar to static objects, such as road signs.

[0080] A3: The positioning data is corrected based on image data and / or radar data to obtain vehicle position information; vehicle speed, vehicle position information, and attitude information constitute vehicle operation data.

[0081] In this embodiment of the application, in order to correct the positioning data, a high-precision map of the surrounding area of ​​the positioning data location can be obtained. A high-precision map is an electronic map with higher accuracy and more data dimensions. Higher accuracy is reflected in its centimeter-level precision, and more data dimensions are reflected in its inclusion of lane information, as well as surrounding static information related to traffic, such as traffic sign information.

[0082] By comparing image data and / or radar data with the corresponding locations on a high-precision map, the positioning data can be corrected to obtain vehicle location information. For example, the longitudinal distance between the location data and the road sign in the high-precision map can be calculated and compared with the longitudinal distance from the radar to the road sign to calculate the vehicle's longitudinal error, thereby correcting the positioning data.

[0083] Ultimately, vehicle speed, vehicle position information, and attitude information can be combined to form vehicle operation data. In this embodiment, the collected vehicle operation data is relatively accurate.

[0084] Since a large amount of vehicle operation data from different vehicles has been collected, preprocessing of the vehicle operation data can be performed to facilitate subsequent processing. In one possible implementation, before executing S202, the following may also be included:

[0085] B1: Divide vehicle operation data into multiple time periods.

[0086] Since continuous collection of vehicle operation data results in a large amount of data that is inconvenient for subsequent processing, the vehicle operation data can be segmented and stored in multiple time periods.

[0087] This application does not limit the segmentation principle; for example, it can be segmented according to the duration of time. If one vehicle operation data point is collected every 0.1 seconds, resulting in 6000 data points collected over 10 minutes, and the data is segmented every minute, it can be divided into 10 time periods, each containing 600 data points. Alternatively, segmentation can be based on the time point of a sudden change in vehicle posture. As long as the same database is maintained, the segmentation principle remains consistent.

[0088] B2: Align the vehicle operation data of each time period with the coordinate system to generate aligned vehicle operation data for each time period, and redefine the aligned vehicle operation data for each time period as vehicle motion data.

[0089] Then, the segmented data is aligned to ensure that vehicle movement data from different time periods are in the same coordinate system. The aligned vehicle movement data for each time period is then redefined as vehicle motion data, and then S202 is executed again.

[0090] Preprocessing vehicle operation data can improve its ease of use and accelerate the extraction of training samples from it.

[0091] The training process of the vehicle attitude angle prediction model will be explained below.

[0092] In one possible implementation, S204 uses training samples to train an initial LSTM temporal network to generate a vehicle attitude angle prediction model corresponding to the target Euler angles. The specific implementation of this model may include steps C1-C4:

[0093] C1: Input the vehicle operation data corresponding to the target Euler angles at the first consecutive time intervals into the initial LSTM time series network to obtain the predicted angle values ​​corresponding to the target Euler angles at the second consecutive time intervals.

[0094] For ease of understanding, this embodiment uses the target Euler angle as the roll angle as an example. It can be understood that the target Euler angle can also be the pitch angle or the yaw angle.

[0095] First, the training data is input into the initial LSTM temporal network. The initial LSTM temporal network can predict the angle value corresponding to the roll angle for the second consecutive number of time steps using the vehicle running data corresponding to the target Euler angles for the first consecutive number of time steps.

[0096] C2: Calculate the loss value based on the predicted angle value corresponding to the target Euler angle at the second consecutive number of time steps and the angle value corresponding to the target Euler angle at the second consecutive number of time steps.

[0097] Then, based on the predicted angle value corresponding to the roll angle at the second consecutive number of time steps and the angle value corresponding to the roll angle at the second consecutive number of time steps in the training sample, the loss value is calculated. This loss value can characterize the difference between the predicted angle value corresponding to the roll angle and the collected angle value.

[0098] C3: Adjust the initial LSTM temporal network based on the loss value.

[0099] The model parameters in the initial LSTM temporal network can be adjusted based on the loss value to optimize the initial LSTM temporal network.

[0100] C4: Repeat B1 to input the vehicle operation data corresponding to the target Euler angle at the first consecutive time step into the initial LSTM temporal network and subsequent steps until the preset conditions are met, and generate the vehicle attitude angle prediction model corresponding to the target Euler angle.

[0101] Before adjusting the initial LSTM temporal network, it's necessary to determine if preset conditions are met. When these conditions are met, the initial LSTM temporal network training ends, and the vehicle attitude angle prediction model corresponding to the trained roll angle is obtained. If the preset conditions are not met, the initial LSTM temporal network is adjusted based on the loss value until the preset conditions are met.

[0102] As one optional example, the preset condition is reaching a preset number of training iterations. As another optional example, the preset condition is that the loss value reaches a preset loss threshold. The preset number of training iterations and the preset loss threshold can be set according to actual conditions, and this application embodiment does not impose any limitations.

[0103] To illustrate the training process of a vehicle attitude angle prediction model corresponding to the roll angle in a practical context, the following example demonstrates the process. The vehicle attitude angle prediction model predicts the roll angle value from time T+1 to T+N based on the vehicle's operating data from the previous T time steps. Therefore, in the LSTM temporal network, the time step parameter is T, and N is the final output dimension of the LSTM temporal network, representing the roll angle value to be predicted for the next N time steps (N is a positive integer greater than 0).

[0104] We obtain K training samples. Each training sample includes an input feature X and a target variable Y. The input feature X consists of M-dimensional features such as vehicle speed, vehicle position information, and roll angle values ​​from the previous T time steps. The target variable Y consists of N-dimensional roll angle values ​​from time step T+1 to time step T+N. Therefore, the entire dimension of the input feature X is (K, T, M), and the entire dimension of the target variable Y is (K, N). Here, K, T, M, and N are all positive integers.

[0105] In practical applications, the LSTM temporal network structure can be stacked with Z layers, with each layer having P hidden neurons. The dropout parameter is set to D to prevent overfitting, the loss function is set to MSE (mean squared error), and the optimizer is the Adam optimizer. Here, Z and P are positive integers, and D is a floating-point number between 0 and 1. This application does not limit the relevant parameters of the LSTM temporal network; they can be selected according to the actual situation.

[0106] By inputting K training samples into an LSTM temporal network and training it through multiple rounds, the optimal network parameters can be obtained, thus generating a vehicle attitude angle prediction model corresponding to the roll angle.

[0107] Similarly, vehicle attitude angle prediction models corresponding to pitch angle and yaw angle can also be generated.

[0108] See Figure 3 As shown, this figure is a flowchart of a vehicle attitude angle prediction method provided in an embodiment of this application. Figure 3 As shown, the method may include S301-S302:

[0109] S301: Acquire real-time vehicle operation data for a first consecutive number of time intervals. The real-time vehicle operation data includes vehicle speed, vehicle position information, and attitude information. The attitude information includes at least one Euler angle value.

[0110] When vehicle attitude angles need to be predicted, real-time vehicle operation data can be collected via onboard equipment. In practical applications, real-time vehicle operation data can be collected continuously according to a sampling period. For example, if one piece of real-time vehicle operation data is collected every 0.1 seconds, then 600 pieces of real-time vehicle operation data can be collected per minute. Each piece of real-time vehicle operation data corresponds to a time value, which can be represented by a timestamp. Each piece of real-time vehicle operation data includes vehicle speed, vehicle position information, and attitude information, wherein the attitude information includes at least one Euler angle value.

[0111] S302: Input the real-time vehicle operation data of the first consecutive number of time steps into the vehicle attitude angle prediction model corresponding to the target Euler angle to obtain the predicted angle value of the target Euler angle of the second consecutive number of time steps; the target Euler angle is each Euler angle, and the consecutive time steps of the first consecutive number of time steps and the consecutive time steps of the second consecutive number of time steps constitute the consecutive time steps. The vehicle attitude angle prediction model corresponding to the target Euler angle is trained and generated according to the training method of the vehicle attitude angle prediction model mentioned above.

[0112] Once a first consecutive set of real-time vehicle operation data has been collected, the vehicle attitude angle prediction model corresponding to the target Euler angles can be invoked to obtain the predicted angle values ​​of the target Euler angles for the second consecutive set of time steps. When the target Euler angles are defined as individual Euler angles, the predicted angle values ​​of each Euler angle for the second consecutive set of time steps can be obtained.

[0113] The vehicle attitude angle prediction model corresponding to the target Euler angle is trained and generated according to the training method of the vehicle attitude angle prediction model in the above embodiment. For relevant explanations, please refer to the above embodiment, which will not be repeated here.

[0114] After receiving the predicted angle value of the target Euler angle, image stabilization processing can be performed on the vehicle projection system, such as to realize image stabilization of the AR-HUD system.

[0115] In one possible implementation, the predicted angle value of the target Euler angle at the current moment can also be obtained from the predicted angle values ​​of the target Euler angle at a consecutive second number of time moments; the projection matrix of the vehicle projection system is then adjusted based on the predicted angle value of the target Euler angle at the current moment.

[0116] When plotting the projection data, the predicted angle value of the target Euler angle for the current moment is obtained from the predicted angle values ​​of the target Euler angles at the second consecutive set of time steps. The display coordinate system parameters are changed in real time, the projection matrix is ​​adjusted, and anti-shake processing is performed to improve the display fit.

[0117] This application embodiment collects real-time vehicle operation data for T consecutive time periods (i.e., the first consecutive number of time periods) and inputs it into the vehicle attitude angle prediction model corresponding to each Euler angle. This allows for the prediction of the Euler angle values ​​for the next N time periods (i.e., the second consecutive number of time periods), achieving real-time prediction of vehicle attitude angles with high accuracy. The predicted angle values ​​of each Euler angle can also be used to implement image stabilization for the vehicle projection system.

[0118] Based on the training method for a vehicle attitude angle prediction model provided in the above-described method embodiments, this application also provides a training device for a vehicle attitude angle prediction model, which will be described below with reference to the accompanying drawings.

[0119] See Figure 4 As shown in the figure, this is a schematic diagram of the structure of a training device for a vehicle attitude angle prediction model provided in an embodiment of this application. Figure 4 As shown, the device includes:

[0120] The acquisition unit 401 is used to acquire vehicle operation data, which includes vehicle speed, vehicle position information and attitude information, and the attitude information includes at least one Euler angle value.

[0121] The decomposition unit 402 is used to decompose the vehicle operation data into vehicle operation data corresponding to target Euler angles, wherein the target Euler angles are each of the Euler angles.

[0122] The interception unit 403 is used to intercept the vehicle operation data corresponding to the target Euler angle for a first consecutive number of time moments and the angle value corresponding to the target Euler angle for a second consecutive number of time moments from the vehicle operation data corresponding to the target Euler angle as training samples; the first consecutive number of time moments and the second consecutive number of time moments constitute a continuous time moment.

[0123] Training unit 404 is used to train an initial LSTM temporal network using the training samples to generate a vehicle attitude angle prediction model corresponding to the target Euler angle.

[0124] In one possible implementation, the acquisition unit includes:

[0125] The first acquisition subunit is used to acquire vehicle speed and attitude information collected by the vehicle sensors;

[0126] The second acquisition subunit is used to acquire positioning data collected by the Global Positioning System, image data collected by the camera, and radar data collected by the radar.

[0127] The correction subunit is used to correct the positioning data based on the image data and / or the radar data to obtain vehicle position information; the vehicle speed, the vehicle position information, and the attitude information constitute vehicle operation data.

[0128] In one possible implementation, the device further includes:

[0129] A segmentation unit is used to segment the vehicle operation data into vehicle operation data for multiple time periods;

[0130] The alignment unit is used to align the vehicle operation data of each time period in the coordinate system, generate aligned vehicle operation data for each time period, and redefine the aligned vehicle operation data for each time period as vehicle motion data.

[0131] In one possible implementation, the training unit is specifically used for:

[0132] The vehicle operation data corresponding to the target Euler angles at the first consecutive number of time steps are input into the initial LSTM temporal network to obtain the predicted angle values ​​corresponding to the target Euler angles at the second consecutive number of time steps.

[0133] The loss value is calculated based on the predicted angle value corresponding to the target Euler angle at the second consecutive number of time steps and the angle value corresponding to the target Euler angle at the second consecutive number of time steps.

[0134] The initial LSTM timing network is adjusted based on the loss value;

[0135] The process of inputting the vehicle operation data corresponding to the target Euler angle at the first consecutive time step into the initial LSTM temporal network and subsequent steps is repeated until the preset conditions are met, thereby generating the vehicle attitude angle prediction model corresponding to the target Euler angle.

[0136] Based on the vehicle attitude angle prediction method provided in the above-described method embodiments, this application also provides a vehicle attitude angle prediction device, which will be described below with reference to the accompanying drawings.

[0137] See Figure 5 As shown, this figure is a schematic diagram of the structure of a vehicle attitude angle prediction device provided in an embodiment of this application. Figure 5 As shown, the device includes:

[0138] The first acquisition unit 501 is used to acquire real-time vehicle operation data for a first number of consecutive time intervals. The real-time vehicle operation data includes vehicle speed, vehicle position information and attitude information. The attitude information includes at least one Euler angle value.

[0139] The prediction unit 502 is used to input the real-time vehicle operation data of the first consecutive number of time steps into the vehicle attitude angle prediction model corresponding to the target Euler angle, and obtain the predicted angle value of the target Euler angle of the second consecutive number of time steps; the target Euler angle is each Euler angle, the first consecutive number of time steps and the second consecutive number of time steps constitute a consecutive time step, and the vehicle attitude angle prediction model corresponding to the target Euler angle is trained and generated according to the above-mentioned training method of the vehicle attitude angle prediction model.

[0140] In one possible implementation, the device further includes:

[0141] The second acquisition unit is used to obtain the predicted angle value of the target Euler angle at the current moment from the predicted angle values ​​of the target Euler angle at the second consecutive number of time moments;

[0142] An adjustment unit is used to adjust the projection matrix of the vehicle-mounted projection system according to the predicted angle value of the target Euler angle at the current moment.

[0143] In addition, this application also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the training method for the vehicle attitude angle prediction model as described in any of the above claims, or the vehicle attitude angle prediction method as described in any of the above claims.

[0144] In addition, embodiments of this application also provide a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform a training method for a vehicle attitude angle prediction model as described in any of the preceding claims, or a vehicle attitude angle prediction method as described in any of the preceding claims.

[0145] In this embodiment, a large amount of vehicle operation data, including vehicle speed, vehicle position information, and attitude information, is collected. This data is then broken down into vehicle operation data corresponding to each Euler angle. For each Euler angle, vehicle operation data from T consecutive time periods and the angle values ​​of that Euler angle from the following N time periods are extracted as training samples to train an initial LSTM temporal network, generating a vehicle attitude angle prediction model for that Euler angle. Then, real-time vehicle operation data from T consecutive time periods is input into the vehicle attitude angle prediction model corresponding to a specific Euler angle. This allows the model to predict the angle values ​​of that Euler angle for the following N time periods, achieving real-time prediction of vehicle attitude angles with high accuracy.

[0146] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems or apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0147] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0148] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0149] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0150] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A training method for a vehicle attitude angle prediction model, characterized in that, The method includes: Collect vehicle operation data, which includes vehicle speed, vehicle position information, and attitude information, wherein the attitude information includes at least one Euler angle value. The vehicle operation data is decomposed into vehicle operation data corresponding to target Euler angles, wherein the target Euler angles are each of the Euler angles. The vehicle operation data corresponding to the target Euler angle at a consecutive first number of time intervals and the angle values ​​corresponding to the target Euler angle at a consecutive second number of time intervals are extracted from the vehicle operation data corresponding to the target Euler angle as training samples; the consecutive first number of time intervals and the consecutive second number of time intervals constitute consecutive time intervals. The initial LSTM temporal network is trained using the training samples to generate a vehicle attitude angle prediction model corresponding to the target Euler angle.

2. The method according to claim 1, characterized in that, The collected vehicle operation data includes: Acquire vehicle speed and attitude information collected by onboard sensors; It acquires positioning data collected through the Global Positioning System, image data collected by cameras, and radar data collected by radar. The positioning data is corrected based on the image data and / or the radar data to obtain vehicle position information; the vehicle speed, the vehicle position information, and the attitude information constitute vehicle operation data.

3. The method according to claim 1 or 2, characterized in that, Before decomposing the vehicle operation data into vehicle operation data corresponding to the target Euler angles, the method further includes: The vehicle operation data is divided into vehicle operation data for multiple time periods; The vehicle operation data for each time period are aligned with the coordinate system to generate aligned vehicle operation data for each time period, and the aligned vehicle operation data for each time period is redefined as vehicle motion data.

4. The method according to claim 1, characterized in that, The step of training an initial LSTM temporal network using the training samples to generate a vehicle attitude angle prediction model corresponding to the target Euler angles includes: The vehicle operation data corresponding to the target Euler angles at the first consecutive number of time steps are input into the initial LSTM temporal network to obtain the predicted angle values ​​corresponding to the target Euler angles at the second consecutive number of time steps. The loss value is calculated based on the predicted angle value corresponding to the target Euler angle at the second consecutive number of time steps and the angle value corresponding to the target Euler angle at the second consecutive number of time steps. The initial LSTM timing network is adjusted based on the loss value; The process of inputting the vehicle operation data corresponding to the target Euler angle at the first consecutive time step into the initial LSTM temporal network and subsequent steps is repeated until the preset conditions are met, thereby generating the vehicle attitude angle prediction model corresponding to the target Euler angle.

5. A method for predicting vehicle attitude angles, characterized in that, The method includes: Acquire real-time vehicle operation data for a first consecutive number of time intervals. The real-time vehicle operation data includes vehicle speed, vehicle position information, and attitude information. The attitude information includes at least one Euler angle value. The vehicle real-time operation data of the first consecutive number of time moments are input into the vehicle attitude angle prediction model corresponding to the target Euler angle to obtain the predicted angle value of the target Euler angle of the second consecutive number of time moments; the target Euler angle is each Euler angle, the first consecutive number of time moments and the second consecutive number of time moments constitute a continuous time moment, and the vehicle attitude angle prediction model corresponding to the target Euler angle is generated by training the vehicle attitude angle prediction model according to any one of claims 1-4.

6. The method according to claim 5, characterized in that, The method further includes: The predicted angle value of the target Euler angle at the current moment is obtained from the predicted angle values ​​of the target Euler angle at the second consecutive number of time moments; The projection matrix of the vehicle projection system is adjusted based on the predicted angle value of the target Euler angle at the current moment.

7. A training device for a vehicle attitude angle prediction model, characterized in that, The device includes: The acquisition unit is used to acquire vehicle operation data, which includes vehicle speed, vehicle position information and attitude information, and the attitude information includes at least one Euler angle value. A decomposition unit is used to decompose the vehicle operation data into vehicle operation data corresponding to target Euler angles, wherein the target Euler angles are each of the Euler angles. The interception unit is used to intercept the vehicle operation data corresponding to the target Euler angle for a first consecutive number of time moments and the angle value corresponding to the target Euler angle for a second consecutive number of time moments from the vehicle operation data corresponding to the target Euler angle as training samples; the first consecutive number of time moments and the second consecutive number of time moments constitute a continuous time moment; The training unit is used to train an initial LSTM temporal network using the training samples to generate a vehicle attitude angle prediction model corresponding to the target Euler angle.

8. A vehicle attitude angle prediction device, characterized in that, The device includes: The acquisition unit is used to acquire real-time vehicle operation data for a first number of consecutive time intervals. The real-time vehicle operation data includes vehicle speed, vehicle position information, and attitude information. The attitude information includes at least one Euler angle value. The prediction unit is used to input the real-time vehicle operation data of the first consecutive number of time steps into the vehicle attitude angle prediction model corresponding to the target Euler angle, and obtain the predicted angle value of the target Euler angle for a second consecutive number of time steps; the target Euler angle is each Euler angle, the first consecutive number of time steps and the second consecutive number of time steps constitute a continuous time step, and the vehicle attitude angle prediction model corresponding to the target Euler angle is generated by training the vehicle attitude angle prediction model according to any one of claims 1-4.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the training method for the vehicle attitude angle prediction model as described in any one of claims 1-4, or the vehicle attitude angle prediction method as described in any one of claims 5-6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the training method for the vehicle attitude angle prediction model as described in any one of claims 1-4, or the vehicle attitude angle prediction method as described in any one of claims 5-6.

Images