A vehicle trajectory prediction method, device, equipment and storage medium

Abstract

Embodiments of the present application provide a vehicle track prediction method, device, equipment and storage medium. When the positioning system of the vehicle is abnormal, the electronic device can input the data of multiple sensors acquired in real time into a spatial vehicle motion state model to predict the spatial position and spatial motion state of the vehicle relative to an initial calculation time; and according to the spatial position vector and the spatial motion state vector, a track prediction model is used to predict the track of the vehicle. Through this implementation, the track of the vehicle can still be accurately predicted when the positioning system is in an abnormal state, and the accuracy and universality of track prediction are improved.

Description

Technical Field

[0001] This invention relates to the field of vehicle trajectory prediction technology, and in particular to a vehicle trajectory prediction method, apparatus, device, and storage medium. Background Technology

[0002] As the level of automotive intelligence continues to increase, vehicle trajectory prediction technology is also constantly evolving. Current technologies typically employ multiple sensors, such as GPS (Global Positioning System) sensors, to directly collect vehicle information and derive the vehicle's trajectory. However, this approach suffers from significantly reduced accuracy when sensors are obstructed or malfunction. Therefore, a solution that does not rely on GPS sensors to directly acquire vehicle positioning information urgently needs to be developed. Summary of the Invention

[0003] This application provides a vehicle trajectory prediction method, apparatus, device, and storage medium for predicting vehicle trajectories more accurately.

[0004] This application provides a vehicle trajectory prediction method, comprising: acquiring data from multiple sensors of the vehicle in real time; the multiple sensors include sensors for measuring the vehicle's body attitude and motion state; inputting the data from the multiple sensors into a preset spatial vehicle motion state model to predict the spatial position vector and spatial motion state vector of the vehicle at the current moment relative to the initial calculation time; and predicting the vehicle's trajectory using a trajectory prediction model based on the spatial position vector and the spatial motion state vector.

[0005] Further optionally, real-time acquisition of data from multiple sensors of the vehicle includes: real-time acquisition of initial data from multiple sensors of the vehicle; inputting the initial data from multiple sensors into a preset data filtering model to obtain filtered data from the multiple sensors; and inputting the filtered data from the multiple sensors into a preset data state analysis model for data filtering to obtain filtered data from the multiple sensors.

[0006] Optionally, the data from the multiple sensors are input into a preset space vehicle motion state model to predict the spatial position vector and spatial motion state vector of the vehicle at the current moment relative to the initial calculation moment. This includes: inputting data from multiple sensors corresponding to the current moment and the moment before the current moment into the space vehicle motion state model to obtain the change in spatial position and the change in spatial motion state at the current moment relative to the previous moment; calculating the spatial position vector at the current moment relative to the initial calculation moment based on the historical spatial position vector at the previous moment relative to the initial calculation moment and the change in spatial position; and calculating the spatial motion state vector at the current moment relative to the initial calculation moment based on the historical spatial position vector at the previous moment relative to the initial calculation moment and the change in spatial motion state.

[0007] Further optionally, based on the spatial position vector and the spatial motion state vector, a trajectory prediction model is used to predict the vehicle's trajectory, including: determining the vehicle coordinate system in which the spatial position vector and the spatial motion state vector reside; calculating the alignment relationship between the vehicle coordinate system and the world coordinate system corresponding to the vehicle's positioning system based on the vehicle's historical positioning data; the historical positioning data includes: GPS data of the vehicle at historical times; transforming the spatial position vector and the spatial motion state vector into the world coordinate system according to the alignment relationship; and using the trajectory prediction model to predict the vehicle's trajectory based on the spatial position vector and spatial motion state vector transformed into the world coordinate system.

[0008] Optionally, based on the spatial position vector and the spatial motion state vector, a trajectory prediction model is used to predict the trajectory of the vehicle, including: inputting the spatial position vector and the spatial motion state vector into the trajectory prediction model; in the trajectory prediction model, calculating a target road that matches the spatial position vector and the spatial motion state vector based on first road data; the first road data includes at least one of road position data, lane data, lane direction data, and road condition data of at least one road; and determining the trajectory based on the target road.

[0009] Further optionally, in the trajectory prediction model, calculating the target road matching the spatial position vector and the spatial motion state vector based on the first road data includes: inputting the spatial position vector, the spatial motion state vector, and the first road data into a first path matching model based on Hidden Markov Models; in the first path matching model, calculating the probability that the vehicle is located on at least one candidate road based on the spatial position vector, the spatial motion state vector, and the first road data; and inputting the spatial position vector, the spatial motion state vector, the first road data, the probability that the vehicle is located on at least one candidate road, and the vehicle's historical data into a second path matching model based on deep learning, so as to determine the target road from the at least one candidate road through the second path matching model.

[0010] Further optionally, after predicting the vehicle's trajectory, the method further includes: correcting the predicted spatial position vector based on the vehicle's trajectory to obtain the real-time position of the vehicle when the positioning system is in an abnormal state.

[0011] This application provides a vehicle trajectory prediction method, including: a memory and a processor; wherein, the memory is used to: store one or more computer instructions; the processor is used to execute the one or more computer instructions to: perform the steps in the vehicle trajectory prediction method.

[0012] This application provides a vehicle trajectory prediction device, comprising: a data acquisition module for acquiring data from multiple sensors of the vehicle in real time, including sensors for measuring the vehicle's body attitude and motion state; a vector prediction module for inputting the data from the multiple sensors into a preset spatial vehicle motion state model to predict the spatial position vector and spatial motion state vector of the vehicle at the current moment relative to an initial calculation time; and a trajectory prediction module for predicting the vehicle's trajectory based on the spatial position vector and the spatial motion state vector using the trajectory prediction model.

[0013] Further optionally, when the vector prediction module inputs the data from the multiple sensors into a preset spatial vehicle motion state model to predict the spatial position vector and spatial motion state vector of the vehicle at the current moment relative to the initial calculation time, it is specifically configured to: input the data from the multiple sensors corresponding to the current moment and the previous moment into the spatial vehicle motion state model to obtain the change in spatial position and the change in spatial motion state at the current moment relative to the previous moment; calculate the spatial position vector at the current moment relative to the initial calculation time based on the historical spatial position vector at the previous moment relative to the initial calculation time and the change in spatial position; and calculate the spatial motion state vector at the current moment relative to the initial calculation time based on the historical spatial motion state vector at the previous moment relative to the initial calculation time and the change in spatial motion state.

[0014] Further optionally, when the trajectory prediction module predicts the trajectory of the vehicle using a trajectory prediction model based on the spatial position vector and the spatial motion state vector, it is specifically configured to: input the spatial position vector and the spatial motion state vector into the trajectory prediction model; in the trajectory prediction model, calculate a target road that matches the spatial position vector and the spatial motion state vector based on first road data; the first road data includes at least one of road position data, lane data, lane direction data, and road condition data of at least one road; and determine the trajectory based on the target road.

[0015] This application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps in the vehicle trajectory prediction method.

[0016] In this embodiment, when the vehicle's positioning system malfunctions, the electronic device can input real-time data from various sensors into a spatial vehicle motion state model to predict the vehicle's spatial position and motion state relative to the initial calculation time. Based on the spatial position vector and the spatial motion state vector, a trajectory prediction model is used to predict the vehicle's trajectory. This implementation method allows for relatively accurate trajectory prediction even when the positioning system is malfunctioning, improving the accuracy and versatility of trajectory prediction. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1A flowchart illustrating a vehicle trajectory prediction method provided as an exemplary embodiment of this application;

[0019] Figure 2 A block diagram provided for a practical application of an exemplary embodiment of this application;

[0020] Figure 3 A schematic diagram of the structure of an electronic device provided in an exemplary embodiment of this application;

[0021] Figure 4 A schematic diagram of a vehicle trajectory prediction device provided for an exemplary embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] In existing technologies, the accuracy of trajectory prediction is significantly reduced when sensors are obstructed or malfunctioning. To address this technical problem, this application provides a vehicle trajectory prediction method in some embodiments.

[0024] In this vehicle trajectory prediction method, the spatial position and spatial motion state of a vehicle relative to a specified time can be predicted based on data from multiple sensors and a spatial vehicle motion state model. Furthermore, based on the spatial position vector, the spatial motion state vector, and first road data, a trajectory prediction model is used to predict the vehicle's trajectory. The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0025] Figure 1 A schematic flowchart of a vehicle trajectory prediction method provided in an exemplary embodiment of this application is shown below. Figure 1 As shown, the method includes:

[0026] Step 11: Acquire data from multiple sensors of the vehicle in real time; these sensors include those that measure the vehicle's attitude and motion.

[0027] Step 12: Input data from multiple sensors into a preset spatial vehicle motion state model to predict the vehicle's spatial position and vector space motion state vector at the current moment relative to the initial calculation moment.

[0028] Step 13: Based on the spatial position vector and spatial motion state vector, use the trajectory prediction model to predict the vehicle's trajectory.

[0029] This embodiment can be executed by an in-vehicle terminal device, which can be fixedly installed in a vehicle; in addition to an in-vehicle terminal, this embodiment can also be executed by a mobile terminal such as a smartphone or tablet computer, and this embodiment is not limited.

[0030] In this embodiment, the on-board terminal device can acquire data from multiple sensors in the vehicle in real time when the vehicle's positioning system malfunctions. This data includes data obtained from multiple sensor acquisition operations on various parts of the vehicle. These sensors may include, but are not limited to, two or more of the following: IMU (Inertial Measurement Unit) sensors, gyroscopes, angle sensors (used to calculate front wheel deflection angle, yaw angle, heading angle, etc.), and wheel speed sensors (wheel speed gauges). For example, if the sensors acquire data once each at times T0, T1, and T2, the data from these sensors may include: a 5° pitch angle corresponding to the acquisition operation at time T0, a 10° pitch angle corresponding to the acquisition operation at time T1, and a 15° pitch angle corresponding to the acquisition operation at time T2.

[0031] After acquiring data from multiple sensors, this data can be input into a pre-defined spatial vehicle motion state model to predict the vehicle's spatial position vector and spatial motion state vector relative to a specified time (i.e., the initial calculation time). The specified time can be the initial calculation time for prediction, which can be a time when the vehicle's positioning system is functioning normally or a time after an anomaly has occurred. The spatial position vector describes the vehicle's relative position to its location at the initial time. When multiple acquisitions and prediction operations are performed in real time, the predicted spatial position vector can include multiple spatial position vectors predicted based on the sensor data corresponding to the multiple acquisition operations. Connecting these multiple spatial position vectors in chronological order can preliminarily fit the approximate driving trajectory of the vehicle.

[0032] The spatial motion state vector refers to data indicating the change in the vehicle's current driving state in real space relative to its driving state at a specified time. When multiple data acquisitions are performed and multiple prediction operations are executed in real time, the predicted spatial motion state vector may include multiple spatial motion state vectors predicted from the data of the sensors corresponding to the multiple acquisition operations. In some optional embodiments, the spatial motion state vector may include, but is not limited to, at least one of the following motion state information: the vehicle's heading angle, acceleration, yaw angle, pitch angle, vehicle speed in at least one direction, vehicle acceleration in at least one direction, and vehicle angular velocity in at least one direction.

[0033] For example, assuming T3 is a specified time, the terminal device collects sensor data once at T4 and once at T5. The vehicle is located at point A at T3. Then, through the vehicle's spatial motion state model, the spatial position change of point B relative to point A corresponding to the data collection operation at T4 and the spatial position change of point C relative to point A corresponding to the data collection operation at T5 can be predicted. Furthermore, the yaw angle change of 50° relative to T3 and the yaw angle change of 60° relative to T3 can be predicted for the data collection operation at T4.

[0034] After predicting the spatial position vector and the spatial motion state vector, a trajectory prediction model can be used to predict the vehicle's trajectory based on the spatial position vector and the spatial motion state vector.

[0035] The trajectory prediction model can be pre-trained using training data, which may include spatial position sample data and spatial motion state sample data of the vehicle. During training, the spatial position sample data and spatial motion state sample data are input into the trajectory prediction model, and the predicted trajectory is output. Comparing the predicted trajectory with the vehicle's actual trajectory determines the prediction error of the trajectory prediction model. Optimizing the parameters of the depth model with the goal of continuously converging this prediction error yields the fully trained trajectory prediction model.

[0036] In this embodiment, when the vehicle's positioning system malfunctions, the electronic device can input real-time data from various sensors into a spatial vehicle motion state model to predict the vehicle's spatial position vector and spatial motion state vector relative to a specified time. Based on the spatial position vector and spatial motion state vector, a trajectory prediction model is used to predict the vehicle's trajectory. This implementation method allows for relatively accurate trajectory prediction even when the positioning system is in an abnormal state, improving the accuracy and universality of trajectory prediction.

[0037] In some optional embodiments, when the vehicle's positioning system malfunctions and data from multiple vehicle sensors are acquired in real time, the initial data collected from these sensors can be filtered and selected. Specifically, this can be achieved according to the following steps:

[0038] The system acquires initial data from various vehicle sensors in real time. This initial data may include at least one of the following: wheel speed, motor speed signal, IMU acceleration, pitch angle, yaw angle, and steering wheel angle. For example, the acquired initial sensor data may be: yaw angle 15° and yaw angle 10°.

[0039] The initial data from the sensor is input into a preset data filtering model to obtain filtered data. Since the initial data acquired by the sensor may contain noise, this step aims to filter the sensor data to reduce noise interference.

[0040] After obtaining filtered data from multiple sensors, this data can be input into a preset data state analysis model for data filtering, resulting in filtered data from the various sensors. Since the initial data collected by the sensors may contain useless data, this step aims to filter out this useless data to reduce the computational load in subsequent steps.

[0041] In some optional embodiments, the operation of inputting data from multiple sensors into a preset spatial vehicle motion state model in step 12 above to predict the spatial position vector and spatial motion state vector of the vehicle at the current moment relative to the initial calculation time can be achieved through the following steps:

[0042] Step 121: Input the data from multiple sensors corresponding to the current moment and the moment before the current moment into the spatial vehicle motion state model to obtain the change in spatial position and the change in spatial motion state between the current moment and the moment before the current moment.

[0043] When data from multiple sensors corresponding to two adjacent time points (i.e., the current time and the previous time) are input into the space vehicle motion state model, the model can output the spatial position change and spatial motion state change between the two adjacent time points. For example, taking the time series {t0, t1, t2, t3} as an example, when data from multiple sensors corresponding to t0 and t1 are input into the model, the model can output the spatial position change ΔP1(t0, t1) and spatial motion state change ΔC1(t0, t1) at time t1 relative to t0; when data from multiple sensors corresponding to t1 and t2 are input, the model can output the spatial position change ΔP2(t1, t2) and spatial motion state change ΔC2(t1, t2) at time t2 relative to t1. Similarly, ΔP3(t2, t3) and ΔC3(t2, t3) can be obtained.

[0044] Step 122: Calculate the spatial position vector of the current time relative to the initial calculation time based on the historical spatial position vector of the previous time relative to the initial calculation time and the change in spatial position.

[0045] Here, the historical spatial position vector refers to the spatial position vector calculated before the current moment. The historical spatial position vector of the previous moment is used to describe the positional change of the previous moment relative to the initial calculation moment. Assume that t0 is the initial calculation moment, t1 is the previous moment, and the current moment is t2; the historical spatial position vector (P0, P1) at time t1 is calculated from the initial spatial position vector at time t0 and the spatial positional change ΔP1 at time t1 relative to time t0. The historical spatial position vector (P0, P2) at time t2 is calculated from the historical spatial position vector (P0, P1) at time t1 and the spatial positional change ΔP2 at time t2 relative to time t1.

[0046] If the current time is t3, then the historical spatial position vector at time t3 is calculated from the historical spatial position vector at time t2 and the change in spatial position at time t3 relative to time t2. That is, if the current time is t3, then the historical spatial position vector at the previous time t2 can be represented as (P0, P2). By performing vector calculation on (P0, P2) based on ΔP3, the spatial position vector at time t3 relative to time t0 (P0, P3) can be obtained.

[0047] Step 123: Calculate the spatial motion state vector of the current moment relative to the initial calculation moment based on the historical spatial motion state vector of the previous moment relative to the initial calculation moment and the change in spatial motion state.

[0048] The historical spatial motion state vector refers to the spatial motion state vector calculated up to the current moment. The historical spatial motion state vector of the previous moment describes the change in motion state relative to the initial calculation moment. Assuming t0 is the initial calculation moment, t1 is the previous moment, and the current moment is t2; the historical spatial motion state vector (C0, C1) at time t1 is calculated from the initial spatial motion state vector at time t0 and the change in spatial motion state at time t1 relative to time t0, ΔC1. The historical spatial motion state vector (C0, C2) at time t2 is calculated from the historical spatial motion state vector (C0, C1) at time t1 and the change in spatial motion state at time t2 relative to time t1, ΔC2.

[0049] If the current time is t3, then the historical spatial motion state vector at time t3 is calculated from the historical spatial motion state vector at time t2 and the change in spatial motion state at time t3 relative to time t2. That is, if the current time is t3, then the historical spatial motion state vector at the previous time t2 can be represented as (C0, C2). By performing vector calculation on (C0, C2) using ΔC3, the spatial motion state vector at time t3 relative to time t0 can be obtained as (C0, C3).

[0050] Steps 122 and 123 above can be executed synchronously or asynchronously in a specific order; this embodiment does not impose any restrictions.

[0051] In some optional embodiments, when GPS fails, the calculation of the spatial position vector and the spatial motion state vector can be performed in the vehicle coordinate system. To output the track in the GPS coordinate system, the operation in step 13 above, which predicts the vehicle's track using a track prediction model based on the spatial position vector and the spatial motion state vector, can be achieved through the following steps:

[0052] Step 131: Determine the vehicle coordinate system in which the spatial position vector and the spatial motion state vector are located.

[0053] Step 132: Calculate the alignment relationship between the vehicle coordinate system and the world coordinate system corresponding to the vehicle's positioning system based on the vehicle's historical positioning data; the historical positioning data includes: the vehicle's GPS data at historical times.

[0054] Step 133: Based on the alignment relationship, transform the spatial position vector and the spatial motion state vector into the world coordinate system.

[0055] Step 134: Based on the spatial position vector and spatial motion state vector transformed into the world coordinate system, the trajectory prediction model is used to predict the trajectory of the vehicle.

[0056] Steps 131 to 133 above are mainly used to transform the spatial position vector and spatial motion state vector calculated in the vehicle coordinate system to the world coordinate system. Typically, when a vehicle uses GPS for navigation, this world coordinate system refers to the world coordinate system corresponding to the GPS (i.e., the absolute coordinate system).

[0057] In step 132, the vehicle's historical positioning data can be valid positioning data from at least one moment before the vehicle failed. Continuing with the aforementioned time series {t0, t1, t2, t3} as an example, assuming time t0 is the initial calculation time and the GPS is not faulty at time t0, a relative world coordinate system can be established with the GPS position at time t0 as its origin. Based on the GPS position at time t0, the relative world coordinate system with time t0 as its origin can be aligned with the world coordinate system corresponding to the GPS. For example, this alignment can be achieved by rotating, translating, or scaling the relative world coordinate system. At time t0, the vehicle's position in the vehicle body coordinate system can be determined, and the mapping relationship between the vehicle body coordinate system and the relative world coordinate system can be determined based on the vehicle's position in the vehicle body coordinate system at time t0 and the GPS position at time t0.

[0058] Based on the mapping relationship between the vehicle coordinate system and the relative world coordinate system, and the alignment relationship between the relative world coordinate system and the world coordinate system corresponding to GPS, the alignment relationship between the vehicle coordinate system and the world coordinate system corresponding to GPS can be obtained.

[0059] Based on the alignment between the vehicle coordinate system and the GPS-corresponding world coordinate system, the spatial position vector and the spatial motion state vector can be transformed into the world coordinate system for subsequent calculations.

[0060] In some optional embodiments, the spatial position vector and spatial motion state vector in the GPS world coordinate system can be directly calculated using a space vehicle motion state model. Optionally, the space vehicle motion state model can be pre-trained. Specifically, this can be achieved based on the following steps:

[0061] When the vehicle's positioning system is functioning normally, the system acquires the vehicle's GPS coordinates and data samples from various sensors. These sensor data samples refer to the data from the sensors used to train the model. For example, it acquires the positioning coordinates (39°N, 116°E) and data samples from sensors with an acceleration of 5 m / s² and a steering wheel angle of 30°.

[0062] After obtaining the above data, data samples from various sensors can be input into the space vehicle motion state model to obtain the vehicle's estimated spatial position. The term "estimated" is used to define the spatial position, solely for differentiation from the previously mentioned "spatial position".

[0063] It should be noted that after obtaining the estimated spatial position of the vehicle, since the estimated spatial position is in the vehicle coordinate system corresponding to the spatial vehicle motion state model, while the GPS coordinates are in the GPS coordinate system, the two coordinate systems are in different coordinate systems. Therefore, the aforementioned two coordinate systems can be transformed and aligned. This transformation and alignment includes: coordinate scale transformation, coordinate origin alignment, and spatial coordinate correction. Coordinate scale transformation refers to unifying the scale of the two coordinate systems; coordinate origin alignment refers to setting the origin of both coordinate systems to the same point; and spatial coordinate correction refers to updating both the GPS coordinate system (where the GPS coordinates are located) and the coordinate system corresponding to the spatial vehicle motion state model to the same coordinate system after the coordinate system transformation and alignment.

[0064] After the coordinate system transformation and alignment described above, the vehicle's GPS coordinates and the predicted spatial position can be converted to the same coordinate system, thus providing them with the same basis for comparison. Given this shared basis, the difference between the vehicle's GPS coordinates and the predicted spatial position can be calculated. This difference can then be used to train a spatial vehicle motion state model until the difference between the predicted spatial position and the GPS coordinates converges to a specified range. This specified range can be 1% to 3% or other values; this embodiment does not impose any limitations.

[0065] For example, if the vehicle's GPS coordinates are (60°N, 50°E), and the sensor data sample Y1 is input into the space vehicle motion state model, the output predicted spatial position 1 is (70°N, 60°E), which differs significantly from the vehicle's GPS coordinates. Therefore, the space vehicle motion state model can be adjusted. The sensor data sample Y1 is then input into the adjusted space vehicle motion state model again, and the output predicted spatial position 2 is (65°N, 55°E). This predicted spatial position 2 is then compared with the GPS coordinates. If the difference between the two still has not converged to the specified range, the space vehicle motion state model can be adjusted again, i.e., the above training process is repeated until the difference between the predicted spatial position and the GPS coordinates converges to the specified range. If the specified range is 0-2%, the above steps are repeated 100 times. When the predicted spatial position 100 is (60.1°N, 50.2°E), the difference between the predicted spatial position 100 and the GPS coordinates has converged to the specified range.

[0066] By training the space vehicle motion state model as described above, the error between the predicted spatial position of the vehicle relative to a specified time and the actual GPS coordinates can be reduced, thereby improving the accuracy of the space vehicle motion state model.

[0067] In some optional embodiments, when predicting the vehicle's trajectory using a trajectory prediction model based on the spatial position vector and the spatial motion state vector, the spatial position vector and the spatial motion state vector can be input into the trajectory prediction model. In the trajectory prediction model, a target road matching the spatial position vector and the spatial motion state vector can be calculated based on first road data, and the trajectory can be determined based on the target road.

[0068] In this implementation, first road data corresponding to the spatial location can be obtained. The first road data refers to the map road data of the area where the spatial location is located, and this first road data may include at least one of the following: road location data, lane data, lane direction data, and road condition data for at least one road. For example, if it is predicted that a vehicle is located after point D, and the area corresponding to point D is Q1, the first road data of road location data L1 within area Q1 can be obtained.

[0069] In some optional embodiments, the trajectory prediction model may include: a first path matching model based on Hidden Markov Models and a second path matching model based on deep learning. In the trajectory prediction model, the operation of calculating the target road matching the spatial position vector and the spatial motion state vector based on the first road data can be implemented based on the following steps:

[0070] Step 141: Input the spatial location vector, spatial motion state vector, and first road data into the first path matching model based on Hidden Markov Model.

[0071] Hidden Markov Models (HMMs) are a road matching method that associates a series of ordered trajectory points with the actual road network. Specifically, this algorithm compares a path composed of ordered trajectory points with other paths on a map, calculating the similarity between the path and other paths. This allows the algorithm to determine which road on the map the trajectory points are most likely to appear on. In other words, this algorithm can deduce the true location from the trajectory points.

[0072] Step 142: In the first path matching model, calculate the probability that the vehicle is located on at least one candidate road based on the spatial position vector, the spatial motion state vector, and the first road data.

[0073] Among them, at least one candidate road refers to a road in the map road data of the area corresponding to the spatial location vector.

[0074] For example, the first path matching model based on Hidden Markov Model can calculate the probability that a vehicle is located on candidate road 1 is 70% and the probability that it is located on candidate road 2 is 80%, based on the input spatial motion state vector with a yaw angle of 10° and an acceleration of 1m / s2, spatial position 1, and first road data L2.

[0075] Step 143: Input the spatial location vector, spatial motion state vector, first road data, probability of the vehicle being located on at least one candidate road, and historical data of the vehicle into the second path matching model based on deep learning, so as to determine the target road from at least one candidate road through the second path matching model.

[0076] The target road refers to the road selected from the candidate roads that most closely resembles the vehicle's trajectory. The vehicle's historical data refers to the data collected by sensors up to a specified time.

[0077] Continuing with the previous example distance, the second path matching model can be based on the input second yaw angle of 10° and acceleration of 1m / s². 2 Based on the spatial motion state vector, spatial position 1, first road data L2, the probability that the vehicle is located on candidate road 1 is 70% and the probability that it is located on candidate road 2 is 80%, candidate road 2 is determined as the target road.

[0078] Using the above method, the vehicle's location can still be predicted relatively accurately even when the positioning system is in an abnormal state.

[0079] In some optional embodiments, after the trajectory is predicted, the predicted spatial position can be corrected based on the vehicle's trajectory to obtain the vehicle's real-time position when the positioning system is in an abnormal state. For example, based on the predicted trajectory H1, the predicted spatial position 3 can be corrected to spatial position 4, which is the vehicle's real-time position when the positioning system is in an abnormal state.

[0080] The following will combine Figure 2 The vehicle trajectory prediction method provided in this application embodiment will be further explained in terms of practical application scenarios.

[0081] like Figure 2As shown, after the vehicle acquires data from various sensors such as wheel speedometers and IMUs, the sensor data is filtered and selected before being input into a preset spatial vehicle motion state model. This yields the vehicle's spatial position and motion state in the model's coordinate system. Using the GPS signal before GPS failure, the vehicle's GPS coordinates in the GPS coordinate system are obtained. Then, the spatial vehicle motion state model coordinate system and the GPS coordinate system are transformed and aligned to obtain the same coordinate system. The vehicle's GPS coordinates and the predicted spatial position are then transformed to the same coordinate system, providing a common basis for comparison. With this common basis, the difference between the vehicle's GPS coordinates and the predicted spatial position can be calculated, and the spatial vehicle motion state model can be trained based on this difference to improve its prediction accuracy. Finally, combining the information obtained from the preceding steps, a path matching algorithm and a deep learning matching algorithm are used to calculate the vehicle's trajectory. This implementation method allows for relatively accurate trajectory prediction even when the positioning system is in an abnormal state, improving the accuracy and universality of trajectory prediction.

[0082] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 11 to 14 can be device A; or the execution subject of steps 11 and 12 can be device A, and the execution subject of steps 13 and 14 can be device B; and so on.

[0083] Furthermore, some processes described in the above embodiments and accompanying drawings include multiple operations that appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 11, 12, etc., are merely used to distinguish different operations and do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel.

[0084] It should be noted that the terms "first" and "second" in this article are used to distinguish different messages, devices, modules, etc., and do not represent a chronological order, nor do they limit "first" and "second" to different types.

[0085] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an exemplary embodiment of this application, such as... Figure 3 As shown, the electronic device includes a memory 301 and a processor 302.

[0086] Memory 301 is used to store computer programs and can be configured to store various other data to support operation on the terminal device. Examples of this data include instructions for any application or method used to operate on the terminal device, contact data, phone book data, messages, pictures, videos, etc.

[0087] The memory 301 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0088] The processor 302, coupled to the memory 301, is used to execute the computer program in the memory 301 for: acquiring data from multiple sensors of the vehicle in real time; the multiple sensors include sensors that measure the vehicle's body attitude and motion state; inputting the data from the multiple sensors into a preset spatial vehicle motion state model to predict the spatial position vector and spatial motion state vector of the vehicle at the current moment relative to the initial calculation moment; and predicting the vehicle's trajectory using a trajectory prediction model based on the spatial position vector and the spatial motion state vector.

[0089] Further optionally, the processor 302 acquires data from multiple sensors of the vehicle in real time, specifically for: acquiring initial data from multiple sensors of the vehicle in real time; inputting the initial data from multiple sensors into a preset data filtering model to obtain filtered data from the multiple sensors; and inputting the filtered data from the multiple sensors into a preset data state analysis model for data filtering to obtain filtered data from the multiple sensors.

[0090] Further optionally, when the processor 302 inputs the data from the multiple sensors into a preset space vehicle motion state model to predict the spatial position vector and spatial motion state vector of the vehicle at the current moment relative to the initial calculation time, it specifically performs the following steps: inputting the data from the multiple sensors corresponding to the current moment and the previous moment into the space vehicle motion state model to obtain the change in spatial position and the change in spatial motion state at the current moment relative to the previous moment; calculating the spatial position vector at the current moment relative to the initial calculation time based on the historical spatial position vector at the previous moment relative to the initial calculation time and the change in spatial position; and calculating the spatial motion state vector at the current moment relative to the initial calculation time based on the historical spatial position vector at the previous moment relative to the initial calculation time and the change in spatial motion state.

[0091] Optionally, when the processor 302 predicts the vehicle's trajectory using a trajectory prediction model based on the spatial position vector and the spatial motion state vector, it specifically performs the following steps: determining the vehicle coordinate system in which the spatial position vector and the spatial motion state vector reside; calculating the alignment relationship between the vehicle coordinate system and the world coordinate system corresponding to the vehicle's positioning system based on the vehicle's historical positioning data; the historical positioning data including the vehicle's GPS data at historical times; transforming the spatial position vector and the spatial motion state vector into the world coordinate system based on the alignment relationship; and predicting the vehicle's trajectory using the trajectory prediction model based on the spatial position vector and spatial motion state vector transformed into the world coordinate system.

[0092] Further optionally, when the processor 302 predicts the trajectory of the vehicle using a trajectory prediction model based on the spatial position vector and the spatial motion state vector, it specifically performs the following steps: inputting the spatial position vector and the spatial motion state vector into the trajectory prediction model; calculating a target road matching the spatial position vector and the spatial motion state vector based on first road data; the first road data includes at least one of road position data, lane data, lane direction data, and road condition data for at least one road; and determining the trajectory based on the target road.

[0093] Further optionally, when the processor 302 calculates the target road matching the spatial position vector and the spatial motion state vector based on the first road data in the trajectory prediction model, it is specifically configured to: input the spatial position vector, the spatial motion state vector, and the first road data into a first path matching model based on Hidden Markov Model; in the first path matching model, calculate the probability that the vehicle is located on at least one candidate road based on the spatial position vector, the spatial motion state vector, and the first road data; input the spatial position vector, the spatial motion state vector, the first road data, the probability that the vehicle is located on at least one candidate road, and the vehicle's historical data into a second path matching model based on deep learning, so as to determine the target road from the at least one candidate road through the second path matching model.

[0094] Further optionally, the processor 302 is also configured to: correct the predicted spatial position vector based on the vehicle's trajectory to obtain the real-time position of the vehicle when the positioning system is in an abnormal state.

[0095] The above Figure 3The memory in the memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0096] The above Figure 3 The display 303 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation.

[0097] Furthermore, such as Figure 3 As shown, the electronic device also includes other components such as a communication component 304 and a power supply component 305. Figure 3 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 3 The components shown.

[0098] The above Figure 3 The communication component 304 is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G, or 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component may be implemented based on Near Field Communication (NFC), Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), Bluetooth (BT), and other technologies.

[0099] The power supply component 305 provides power to various components of the device in which it resides. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which it resides.

[0100] In this embodiment, when the vehicle's positioning system malfunctions, the electronic device can input real-time data from various sensors into a spatial vehicle motion state model to predict the vehicle's spatial position and motion state relative to the initial calculation time. Based on the spatial position vector and the spatial motion state vector, a trajectory prediction model is used to predict the vehicle's trajectory. This implementation method allows for relatively accurate trajectory prediction even when the positioning system is malfunctioning, improving the accuracy and versatility of trajectory prediction.

[0101] Figure 4 A schematic diagram of a vehicle trajectory prediction device provided in an exemplary embodiment of this application is shown below. Figure 4 As shown, the vehicle trajectory prediction device includes: a data acquisition module 401, a vector prediction module 402, and a trajectory prediction module 403.

[0102] The data acquisition module 401 is used to acquire data from various sensors of the vehicle in real time; the various sensors include sensors that measure the vehicle's body posture and motion state.

[0103] The vector prediction module 402 is used to input the data from the various sensors into a preset spatial vehicle motion state model to predict the spatial position vector and spatial motion state vector of the vehicle at the current moment relative to the initial calculation time.

[0104] The trajectory prediction module 403 is used to predict the trajectory of the vehicle based on the spatial position vector and the spatial motion state vector, using a trajectory prediction model.

[0105] Further optionally, the data acquisition module 401 acquires data from multiple sensors of the vehicle in real time, specifically for: acquiring initial data from multiple sensors of the vehicle in real time; inputting the initial data from multiple sensors into a preset data filtering model to obtain filtered data from the multiple sensors; and inputting the filtered data from the multiple sensors into a preset data state analysis model for data filtering to obtain filtered data from the multiple sensors.

[0106] Further optionally, when the vector prediction module 402 inputs the data from the multiple sensors into a preset spatial vehicle motion state model to predict the spatial position vector and spatial motion state vector of the vehicle at the current moment relative to the initial calculation time, it is specifically configured to: input the data from the multiple sensors corresponding to the current moment and the previous moment into the spatial vehicle motion state model to obtain the change in spatial position and the change in spatial motion state at the current moment relative to the previous moment; calculate the spatial position vector at the current moment relative to the initial calculation time based on the historical spatial position vector at the previous moment relative to the initial calculation time and the change in spatial position; and calculate the spatial motion state vector at the current moment relative to the initial calculation time based on the historical spatial position vector at the previous moment relative to the initial calculation time and the change in spatial motion state.

[0107] Optionally, when the trajectory prediction module 403 predicts the vehicle's trajectory using a trajectory prediction model based on the spatial position vector and the spatial motion state vector, it specifically performs the following steps: determining the vehicle coordinate system in which the spatial position vector and the spatial motion state vector reside; calculating the alignment relationship between the vehicle coordinate system and the world coordinate system corresponding to the vehicle's positioning system based on the vehicle's historical positioning data; the historical positioning data includes the vehicle's GPS data at historical times; transforming the spatial position vector and the spatial motion state vector into the world coordinate system based on the alignment relationship; and predicting the vehicle's trajectory using the trajectory prediction model based on the spatial position vector and spatial motion state vector transformed into the world coordinate system.

[0108] Optionally, when the trajectory prediction module 403 predicts the trajectory of the vehicle using a trajectory prediction model based on the spatial position vector and the spatial motion state vector, it specifically performs the following steps: inputting the spatial position vector and the spatial motion state vector into the trajectory prediction model; calculating a target road matching the spatial position vector and the spatial motion state vector based on first road data; the first road data includes at least one of road position data, lane data, lane direction data, and road condition data for at least one road; and determining the trajectory based on the target road.

[0109] Further optionally, when the trajectory prediction module 403 calculates the target road matching the spatial position vector and the spatial motion state vector based on the first road data in the trajectory prediction model, it is specifically configured to: input the spatial position vector, the spatial motion state vector, and the first road data into a first path matching model based on Hidden Markov Model; in the first path matching model, calculate the probability that the vehicle is located on at least one candidate road based on the spatial position vector, the spatial motion state vector, and the first road data; input the spatial position vector, the spatial motion state vector, the first road data, the probability that the vehicle is located on at least one candidate road, and the vehicle's historical data into a second path matching model based on deep learning, so as to determine the target road from the at least one candidate road through the second path matching model.

[0110] Optionally, the trajectory prediction module 403 is further configured to: correct the predicted spatial position vector based on the trajectory of the vehicle to obtain the real-time position of the vehicle when the positioning system is in an abnormal state.

[0111] In this embodiment, when the vehicle's positioning system malfunctions, the electronic device can input real-time data from various sensors into a spatial vehicle motion state model to predict the vehicle's spatial position and motion state relative to the initial calculation time. Based on the spatial position vector and the spatial motion state vector, a trajectory prediction model is used to predict the vehicle's trajectory. This implementation method allows for relatively accurate trajectory prediction even when the positioning system is malfunctioning, improving the accuracy and versatility of trajectory prediction.

[0112] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps in the vehicle trajectory prediction method.

[0113] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0114] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0115] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0116] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0117] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0118] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0119] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0120] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0121] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for predicting vehicle trajectories, characterized in that, include: Real-time acquisition of data from multiple sensors of the vehicle; The various sensors include sensors that measure the vehicle's body posture and motion state; The data from the various sensors are input into a preset space vehicle motion state model to predict the spatial position vector and spatial motion state vector of the vehicle at the current moment relative to the initial calculation time. Based on the spatial position vector and the spatial motion state vector, a trajectory prediction model is used to predict the trajectory of the vehicle. The step of inputting data from the various sensors into a preset spatial vehicle motion state model to predict the vehicle's spatial position vector and spatial motion state vector at the current moment relative to the initial calculation moment includes: The data from multiple sensors corresponding to the current moment and the previous moment are input into the space vehicle motion state model to obtain the change in spatial position and the change in spatial motion state of the current moment relative to the previous moment. Based on the historical spatial position vector of the previous moment relative to the initial calculation moment and the change in spatial position, calculate the spatial position vector of the current moment relative to the initial calculation moment; and... Based on the historical spatial motion state vector of the previous moment relative to the initial calculation moment and the change in spatial motion state, the spatial motion state vector of the current moment relative to the initial calculation moment is calculated.

2. The method according to claim 1, characterized in that, Real-time acquisition of data from multiple sensors of the vehicle, including: Acquire initial data from multiple sensors of the vehicle in real time; The initial data from the various sensors are input into a preset data filtering model to obtain filtered data from the various sensors after filtering. The filtered data from the various sensors are input into a preset data state analysis model for data filtering to obtain the filtered data from the various sensors.

3. The method according to claim 1, characterized in that, Based on the spatial position vector and the spatial motion state vector, a trajectory prediction model is used to predict the trajectory of the vehicle, including: Determine the vehicle coordinate system in which the spatial position vector and the spatial motion state vector are located; Based on the vehicle's historical positioning data, the alignment relationship between the vehicle coordinate system and the world coordinate system corresponding to the vehicle's positioning system is calculated; the historical positioning data includes: the vehicle's GPS data at historical times; Based on the alignment relationship, the spatial position vector and the spatial motion state vector are transformed into the world coordinate system; Based on the spatial position vector and spatial motion state vector transformed into the world coordinate system, the trajectory prediction model is used to predict the trajectory of the vehicle.

4. The method according to claim 1, characterized in that, Based on the spatial position vector and the spatial motion state vector, a trajectory prediction model is used to predict the trajectory of the vehicle, including: The spatial position vector and the spatial motion state vector are input into the trajectory prediction model; In the trajectory prediction model, a target road matching the spatial position vector and the spatial motion state vector is calculated based on the first road data; the first road data includes at least one of the following: road position data, lane data, lane direction data, and road condition data of at least one road. The flight path is determined based on the target road.

5. The method according to claim 4, characterized in that, In the trajectory prediction model, based on the first road data, the target road that matches the spatial position vector and the spatial motion state vector is calculated, including: The spatial location vector, the spatial motion state vector, and the first road data are input into the first path matching model based on Hidden Markov Model. In the first path matching model, the probability that the vehicle is located on at least one candidate road is calculated based on the spatial location vector, the spatial motion state vector, and the first road data. The spatial location vector, the spatial motion state vector, the first road data, the probability that the vehicle is located on at least one candidate road, and the vehicle's historical data are input into a second path matching model based on deep learning, so as to determine the target road from the at least one candidate road through the second path matching model.

6. The method according to any one of claims 1-5, characterized in that, After predicting the vehicle's trajectory, the method further includes: Based on the vehicle's trajectory, the predicted spatial position vector is corrected to obtain the vehicle's real-time position.

7. An electronic device, characterized in that, include: Memory and processor; The memory is used to store one or more computer instructions; The processor is configured to execute one or more computer instructions for performing the steps of the method according to any one of claims 1-6.

8. A vehicle trajectory prediction device, characterized in that, include: The data acquisition module is used to acquire data from various sensors of the vehicle in real time; The various sensors include sensors that measure the vehicle's body posture and motion state; The vector prediction module is used to input the data from the various sensors into a preset space vehicle motion state model to predict the spatial position vector and spatial motion state vector of the vehicle at the current moment relative to the initial calculation time. The trajectory prediction module is used to predict the trajectory of the vehicle based on the spatial position vector and the spatial motion state vector, using a trajectory prediction model. The vector prediction module, when inputting data from the various sensors into a preset spatial vehicle motion state model to predict the vehicle's spatial position vector and spatial motion state vector at the current moment relative to the initial calculation moment, is specifically used for: The data from multiple sensors corresponding to the current moment and the previous moment are input into the space vehicle motion state model to obtain the change in spatial position and the change in spatial motion state of the current moment relative to the previous moment. Calculate the spatial position vector of the current time relative to the initial calculation time based on the historical spatial position vector of the previous time relative to the initial calculation time and the spatial position change. as well as, Based on the historical spatial motion state vector of the previous moment relative to the initial calculation moment and the change in spatial motion state, the spatial motion state vector of the current moment relative to the initial calculation moment is calculated.

9. The apparatus according to claim 8, characterized in that, When the trajectory prediction module predicts the vehicle's trajectory based on the spatial position vector and the spatial motion state vector using a trajectory prediction model, it is specifically used for: The spatial position vector and the spatial motion state vector are input into the trajectory prediction model; In the trajectory prediction model, a target road matching the spatial position vector and the spatial motion state vector is calculated based on the first road data; the first road data includes at least one of the following: road position data, lane data, lane direction data, and road condition data of at least one road. The flight path is determined based on the target road.

10. A computer-readable storage medium storing a computer program, characterized in that, When a computer program is executed by a processor, it causes the processor to perform the steps of the method according to any one of claims 1-6.

Images