A collision risk prediction method and related apparatus

Abstract

The application discloses a collision risk prediction method and related device, and relates to the technical field of artificial intelligence, which comprises the following steps: obtaining vehicle driving track data and operation track data of at least one obstacle, processing the operation track data of the obstacle and the vehicle driving track data, and obtaining distance-time sequences corresponding to the obstacle and the vehicle. Since the distance-time sequences comprise distances between the obstacle and the vehicle arranged in time sequence and represent relative motion trends between the obstacle and the vehicle, the distance-time sequences corresponding to the obstacle and the vehicle are input into a collision risk prediction model based on hidden Markov, and the collision risk prediction model based on hidden Markov can accurately predict the collision risk according to the relative motion trends between the obstacle and the vehicle, thereby improving driving safety and providing data support for automatic driving.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a collision risk prediction method and related apparatus. Background Technology

[0002] With the development of automotive safety technology, how to identify the collision risk between vehicles and obstacles has become a research focus in this field.

[0003] Currently, the prediction of potential collision risks is mainly based on the current positions of the vehicle and obstacles, ignoring the dynamic changes in the positions of the vehicle and obstacles, resulting in low accuracy. Summary of the Invention

[0004] In view of the above problems, this application provides a collision risk prediction method and related apparatus. The collision risk prediction model based on Hidden Markov Models can accurately predict collision risk based on the relative motion trend between the obstacle and the vehicle. The specific solution is as follows:

[0005] The first aspect of this application provides a collision risk prediction method, including:

[0006] Acquire vehicle trajectory data and the trajectory data of at least one obstacle;

[0007] A temporal relationship analysis is performed on the trajectory data of the obstacle and the trajectory data of the vehicle to obtain a sequence of trajectory point combinations corresponding to the obstacle and the vehicle. The sequence of trajectory point combinations includes multiple trajectory point combinations arranged in temporal order. Each trajectory point combination includes: a vehicle trajectory point and an obstacle trajectory point that matches the vehicle trajectory point.

[0008] Calculate the distance between the vehicle trajectory point and the obstacle trajectory point that matches the vehicle trajectory point in each combination of trajectory points to obtain the time series of distances between the obstacle and the vehicle;

[0009] The time series of distances between the obstacle and the vehicle are input into a collision risk prediction model based on Hidden Markov Model (HMM) to obtain the collision risk prediction results between the obstacle and the vehicle.

[0010] In one possible implementation, the method for training the Hidden Markov-based collision risk prediction model includes:

[0011] Acquire historical data samples for the target user, including vehicle trajectory data samples within a preset time range before and after the vehicle exits autonomous driving in multiple scenarios, and trajectory data samples of at least one obstacle.

[0012] For each scenario, a time-series relationship analysis is performed on the obstacle trajectory data samples and vehicle driving trajectory data samples to obtain trajectory point combination sequence samples corresponding to obstacles and vehicles. The trajectory point combination sequence samples include multiple trajectory point combinations arranged in time sequence. Each trajectory point combination includes: vehicle trajectory point and obstacle trajectory point matching the vehicle trajectory point. The distance between the vehicle trajectory point and the obstacle trajectory point matching the vehicle trajectory point in each trajectory point combination is calculated to obtain the distance time series samples corresponding to obstacles and vehicles.

[0013] In scenarios marked as having dynamic collision risk, the distance time series samples corresponding to obstacles and vehicles with dynamic collision risk are identified as positive samples, while the distance time series samples corresponding to other obstacles and vehicles in the scenario are identified as negative samples.

[0014] In scenarios that have been marked as having no dynamic collision risk, the time series samples of distances to obstacles and vehicles are determined as the negative samples;

[0015] The Hidden Markov Model is trained using a training dataset consisting of the positive and negative samples to obtain the Hidden Markov Collision Risk Prediction Model.

[0016] In one possible implementation, training the Hidden Markov Model using a training dataset consisting of the positive and negative samples to obtain the Hidden Markov-based collision risk prediction model includes:

[0017] A hidden Markov model is constructed, which includes two hidden states, one with collision risk and one without collision risk; the observation states corresponding to the two hidden states are time series samples of distances to obstacles and vehicles.

[0018] The model parameters of the Hidden Markov Model are initialized;

[0019] The Hidden Markov Model is iteratively trained using the training dataset, and the model parameters are updated during each iteration to obtain the changes in the model parameters.

[0020] If the change in the model parameters is less than a first threshold and / or the number of iterations of the Hidden Markov Model reaches a preset maximum number of iterations, the training of the Hidden Markov Model ends, and the Hidden Markov-based collision risk prediction model is obtained.

[0021] In one possible implementation, after obtaining the collision risk prediction result between the obstacle and the vehicle, the collision risk prediction method further includes:

[0022] When the vehicle is in autonomous driving mode, if the collision risk prediction result indicates that there is a dynamic collision risk between the obstacle and the vehicle, then the relative motion trend between the obstacle and the vehicle is determined based on the distance time series corresponding to the obstacle and the vehicle.

[0023] The vehicle's direction and speed are controlled based on the relative motion trend between the obstacle and the vehicle.

[0024] In one possible implementation, after obtaining the collision risk prediction result between the obstacle and the vehicle, the collision risk prediction method further includes:

[0025] When the vehicle is in manual driving mode, if the collision risk prediction result indicates that there is a dynamic collision risk between the obstacle and the vehicle, a collision risk warning message will be issued.

[0026] In one possible implementation, a time-series relationship analysis is performed on the trajectory data of the obstacle and the trajectory data of the vehicle to obtain a sequence of trajectory point combinations corresponding to the obstacle and the vehicle, including:

[0027] For each vehicle trajectory point in the vehicle trajectory data, the obstacle trajectory point with the smallest time difference from the vehicle trajectory point in the obstacle's trajectory data is determined as the obstacle trajectory point that matches the vehicle trajectory point.

[0028] Vehicle trajectory points with a time difference less than or equal to a second threshold and matching obstacle trajectory points are determined as the trajectory point combinations, and each trajectory point combination is sorted according to time sequence to obtain the trajectory point combination sequence corresponding to the obstacle and the vehicle.

[0029] A second aspect of this application provides a collision risk prediction device, comprising:

[0030] The first acquisition unit is used to acquire vehicle trajectory data and the trajectory data of at least one obstacle.

[0031] The analysis unit is used to perform time-series relationship analysis on the trajectory data of the obstacle and the trajectory data of the vehicle to obtain a sequence of trajectory point combinations corresponding to the obstacle and the vehicle. The sequence of trajectory point combinations includes multiple trajectory point combinations arranged in time sequence. Each trajectory point combination includes: a vehicle trajectory point and an obstacle trajectory point that matches the vehicle trajectory point.

[0032] The calculation unit is used to calculate the distance between the vehicle trajectory point and the obstacle trajectory point that matches the vehicle trajectory point in each combination of trajectory points, and to obtain the time series of the distance between the obstacle and the vehicle.

[0033] The prediction unit is used to input the distance time series corresponding to the obstacle and the vehicle into the collision risk prediction model based on Hidden Markov Model to obtain the collision risk prediction result between the obstacle and the vehicle.

[0034] A third aspect of this application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0035] The memory is used to store computer programs;

[0036] The processor is used to execute the computer program so that the electronic device can implement the collision risk prediction method of the first aspect or any implementation thereof.

[0037] The fourth aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the collision risk prediction method of the first aspect or any implementation thereof.

[0038] The fifth aspect of this application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the collision risk prediction method described in the first aspect or any implementation thereof.

[0039] By employing the above technical solution, this application provides a collision risk prediction method and related apparatus that performs time-series relationship analysis on obstacle trajectory data and vehicle driving trajectory data to obtain a sequence of trajectory point combinations corresponding to the obstacle and the vehicle. Then, based on the vehicle trajectory points and the obstacle trajectory points matching the vehicle trajectory points in the multiple trajectory point combinations arranged in time sequence within the trajectory point combination sequence, a distance time series corresponding to the obstacle and the vehicle is obtained. Since the distance time series includes the distances between the obstacle and the vehicle arranged in time sequence, representing the relative motion trend between the obstacle and the vehicle, by inputting the distance time series corresponding to the obstacle and the vehicle into a Hidden Markov Model (HMM)-based collision risk prediction model, the HMM-based collision risk prediction model can accurately predict collision risks based on the relative motion trend between the obstacle and the vehicle, improving driving safety and providing data support for autonomous driving. Attached Figure Description

[0040] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0041] Figure 1A system architecture diagram provided for an embodiment of this application;

[0042] Figure 2 This is a schematic diagram of the structure of a vehicle-mounted device provided in an embodiment of this application;

[0043] Figure 3 This is a schematic diagram of the structure of a network-side device provided in an embodiment of this application;

[0044] Figure 4 A flowchart illustrating a collision risk prediction method provided in an embodiment of this application;

[0045] Figure 5 A flowchart illustrating a method for training a collision risk prediction model based on Hidden Markov Models, provided in an embodiment of this application;

[0046] Figure 6 This is a schematic diagram of a collision risk prediction device provided in an embodiment of this application. Detailed Implementation

[0047] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0048] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0049] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0050] See Figure 1 , Figure 1 A schematic diagram of a system architecture is shown. The system may include an in-vehicle device 100 and a network-side device 200. The network-side device 200 may include one or more servers (…). Figure 1(Taking a server as an example for illustration), the network-side device 200 can provide the collision risk prediction method provided in the embodiments of this application to one or more vehicle-mounted devices 100.

[0051] The vehicle-mounted device 100 periodically acquires vehicle driving trajectory data and the running trajectory data of at least one obstacle, and sends the vehicle driving trajectory data and the running trajectory data of at least one obstacle to the network-side device 200. The network-side device 200 performs collision risk prediction based on the received data, obtains the collision risk prediction result, and sends the collision risk prediction result to the vehicle-mounted device 100.

[0052] It should be understood that in some optional implementations, the vehicle-mounted device 100 may also perform the action of collision risk prediction and obtain collision risk prediction results by itself based on the vehicle driving trajectory data and the running trajectory data of at least one obstacle, without the cooperation of the network-side device 200. This application embodiment is not limited to this.

[0053] The following description Figure 1 The product form of the vehicle-mounted equipment 100;

[0054] The vehicle-mounted device 100 in this application embodiment is an electronic device in a vehicle, and this application embodiment does not impose any limitations on it.

[0055] Figure 2 A schematic diagram of an optional hardware structure for the vehicle-mounted device 100 is shown.

[0056] refer to Figure 2 As shown, the vehicle-mounted device 100 may include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160 (optional), a speaker 161 (optional), a microphone 162 (optional), a headphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190, and other components. Those skilled in the art will understand that... Figure 2 This is merely an example of vehicle-mounted device 100 and does not constitute a limitation on vehicle-mounted device 100. It may include more or fewer components than shown, or combine certain components, or use different components.

[0057] The input unit 130 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the vehicle-mounted device 100. Specifically, the input unit 130 may include a touchscreen 131 (optional) and / or other input devices 132. The touchscreen 131 can collect touch operations performed by the user on or near it (such as operations performed by the user using fingers, knuckles, styluses, or any suitable object on or near the touchscreen), and drive the corresponding connection devices according to a pre-set program. The touchscreen can detect the user's touch actions, convert the touch actions into touch signals and send them to the processor 170, and can receive and execute commands sent by the processor 170; the touch signal includes at least touch point coordinate information. The touchscreen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, various types of touchscreens, such as resistive, capacitive, infrared, and surface acoustic wave, can be used to implement the touchscreen. In addition to the touchscreen 131, the input unit 130 may also include other input devices. Specifically, other input devices 132 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0058] Among them, the input device 132 can receive input data, etc.

[0059] The display unit 140 can be used to display information input by the user or information provided to the user, various menus of the in-vehicle device 100, interactive interfaces, file display, and / or playback of any multimedia file. In this embodiment, the display unit 140 can be used to display collision risk warning information, etc.

[0060] The memory 120 can be used to store instructions and data. The memory 120 may primarily include an instruction storage area and a data storage area. The data storage area can store various types of data, such as multimedia files and text. The instruction storage area can store software units such as operating systems, applications, and instructions required for at least one function, or subsets or extended sets thereof. It may also include non-volatile random access memory. It provides the processor 170 with hardware, software, and data resources for managing the computing device, supporting control software and applications. It is also used for storing multimedia files, as well as storing running programs and applications.

[0061] The processor 170 is the control center of the terminal 100. It connects various parts of the vehicle-mounted device 100 via various interfaces and lines. By running or executing instructions stored in the memory 120 and calling data stored in the memory 120, it performs various functions and processes data of the vehicle-mounted device 100, thereby providing overall control of the vehicle-mounted device 100. Optionally, the processor 170 may include one or more processing units; preferably, the processor 170 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 170. In some embodiments, the processor and memory can be implemented on a single chip; in some embodiments, they can also be implemented on separate chips. The processor 170 can also be used to generate corresponding operation control signals, send them to the corresponding components of the computing processing device, read and process data in the software, especially read and process data and programs in the memory 120, so that the various functional modules therein perform corresponding functions, thereby controlling the corresponding components to act according to the instructions.

[0062] The memory 120 can be used to store software code related to the collision risk prediction method, and the processor 170 can execute the steps of the collision risk prediction method. It can also schedule other units (such as the above-mentioned input unit 130 and display unit 140) to achieve the corresponding functions.

[0063] The radio frequency unit 110 (optional) can be used for receiving and transmitting signals during information transmission or calls. For example, it can receive downlink information from the base station and process it for the processor 170; additionally, it can transmit uplink data to the base station. Typically, the RF circuit includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, the radio frequency unit 110 can also communicate wirelessly with network devices and other devices. This wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0064] In this embodiment of the application, the radio frequency unit 110 can send data to the network-side device 200 and receive the processing results sent by the network-side device 200.

[0065] It should be understood that the radio frequency unit 110 is optional and can be replaced with other communication interfaces, such as a network port.

[0066] The vehicle-mounted equipment 100 also includes a power supply 190 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the processor 170 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system.

[0067] The vehicle-mounted device 100 also includes an external interface 180, which can be a standard Micro USB interface or a multi-pin connector, which can be used to connect the vehicle-mounted device 100 to communicate with other devices, or to connect a charger to charge the vehicle-mounted device 100.

[0068] Although not shown, the vehicle-mounted device 100 may also include a strobe light, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with various functions, etc., which will not be described in detail here. Some or all of the methods described below can be applied to, for example... Figure 2 The vehicle-mounted equipment 100 shown.

[0069] The following description Figure 1 The product form of the network-side equipment 200;

[0070] Figure 3 A schematic diagram of the structure of a network-side device 200 is provided, such as... Figure 3 As shown, the network-side device 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate with each other via the bus 201.

[0071] Bus 201 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0072] The processor 202 can be any one or more of the following processors: central processing unit (CPU), graphics processing unit (GPU), microprocessor (MP), or digital signal processor (DSP).

[0073] Memory 204 may include volatile memory, such as random access memory (RAM). Memory 204 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0074] The memory 204 can be used to store software code related to the collision risk prediction method, and the processor 202 can execute the steps of the collision risk prediction method, or schedule other units to achieve the corresponding functions.

[0075] It should be understood that the network-side device 200 can be a centralized or distributed device. The processors (e.g., processor 170 and processor 202) in the vehicle-mounted device 100 and the network-side device 200 can be hardware circuits (such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), general-purpose processors, digital signal processors (DSPs), microprocessors or microcontrollers, etc.) or combinations of these hardware circuits. For example, the processor can be a hardware system with instruction execution capabilities, such as a CPU or DSP, or a hardware system without instruction execution capabilities, such as an ASIC or FPGA, or a combination of the aforementioned hardware systems without instruction execution capabilities and hardware systems with instruction execution capabilities.

[0076] This application provides a collision risk prediction method. The collision risk prediction method of this application embodiment will be described in detail below with reference to the accompanying drawings.

[0077] Reference Figure 4 , Figure 4 This is a flowchart illustrating a collision risk prediction method provided in an embodiment of this application, as shown below. Figure 4 As shown in the figure, the collision risk prediction method provided in this application embodiment may include steps 401 to 404, which are described in detail below.

[0078] 401: Acquire vehicle trajectory data and the trajectory data of at least one obstacle;

[0079] The vehicle trajectory data includes multiple vehicle trajectory points, and each obstacle corresponds to a separate trajectory data. The obstacle trajectory data includes multiple obstacle trajectory points. Each vehicle trajectory point and obstacle trajectory point includes various attributes, such as timestamp, position, direction angle, speed, acceleration, etc.

[0080] Vehicle trajectory data can be obtained based on data collected by various sensors in the vehicle, such as positioning devices and speed sensors. Obstacle trajectory data can be obtained based on data collected by devices such as cameras and lidar installed on the vehicle.

[0081] In different application scenarios, the vehicle's trajectory data and the trajectory data of at least one obstacle are different.

[0082] In application scenarios where collision risk prediction is performed periodically during vehicle operation, the vehicle's driving trajectory data and the trajectory data of at least one obstacle are periodically acquired. This data can be collected when the user is driving the vehicle or when the user is using the autonomous driving function.

[0083] In the application scenario of automatically labeled data samples, vehicle driving trajectory data and the running trajectory data of at least one obstacle are historical data samples to be labeled.

[0084] In application scenarios where autonomous driving functions are optimized using historical collision risk data, the vehicle's driving trajectory data and the trajectory data of at least one obstacle are obtained within a preset time period before and after the vehicle exits the autonomous driving function. For example, the vehicle's driving trajectory data and the trajectory data of at least one obstacle are obtained within a range of 10 seconds before the vehicle exits the autonomous driving function and 5 seconds after the vehicle exits the autonomous driving function.

[0085] Among them, obstacles are dynamic obstacles, such as vehicles and pedestrians.

[0086] 402: Perform time-series relationship analysis on obstacle trajectory data and vehicle trajectory data to obtain the trajectory point combination sequence corresponding to the obstacle and the vehicle. The trajectory point combination sequence includes multiple trajectory point combinations arranged in time sequence. Each trajectory point combination includes: vehicle trajectory point and obstacle trajectory point matching the vehicle trajectory point.

[0087] The obstacle trajectory data includes multiple obstacle trajectory points, and the vehicle trajectory data includes multiple vehicle trajectory points.

[0088] Temporal relationship analysis is performed on the trajectory data of obstacles and the trajectory data of vehicles. Specifically, the temporal relationship between obstacle trajectory points and vehicle trajectory points is analyzed to obtain multiple trajectory point combinations. These multiple trajectory point combinations arranged in temporal order form a sequence of trajectory point combinations corresponding to obstacles and vehicles.

[0089] In one possible implementation, in order to accurately describe the relative motion characteristics between the obstacle and the vehicle, the obstacle trajectory points matched with the vehicle trajectory points are the obstacle trajectory points within the same sampling period as the vehicle trajectory points.

[0090] 403: Calculate the distance between the vehicle trajectory point and the obstacle trajectory point matched by the vehicle trajectory point in each trajectory point combination to obtain the time series of distances between the obstacle and the vehicle;

[0091] Specifically, based on the positions of the vehicle trajectory points and the obstacle trajectory points matched by the vehicle trajectory points in the trajectory point combination, the distance between the vehicle trajectory points and the obstacle trajectory points matched by the vehicle trajectory points in the trajectory point combination is calculated.

[0092] Distance time series includes the distances between obstacles and vehicles arranged in chronological order.

[0093] Since the time series of distances between obstacles and vehicles includes the distances between obstacles and vehicles arranged in chronological order, it represents the change in the distance between obstacles and vehicles over time, such as the distance between obstacles and vehicles getting closer or farther.

[0094] 404: Input the time series of distances between obstacles and vehicles into a collision risk prediction model based on Hidden Markov Model to obtain the collision risk prediction results between obstacles and vehicles.

[0095] Since the time series of distances between obstacles and vehicles represents the relative motion trend between them, by inputting the time series of distances between obstacles and vehicles into a collision risk prediction model based on Hidden Markov Models, it is possible to analyze the changes in the relative motion between obstacles and vehicles over time, effectively capture complex driving scenarios, and predict whether there is a dynamic collision risk between obstacles and vehicles in the next moment.

[0096] The collision risk prediction results between obstacles and vehicles include: the presence of dynamic collision risk and the absence of dynamic collision risk.

[0097] In applications where collision risk prediction is periodically performed during vehicle operation, if the predicted collision risk between the obstacle and the vehicle indicates a dynamic collision risk, different actions are taken depending on the vehicle's driving mode. In manual driving mode, a collision risk warning is issued, such as a voice warning or an animated warning displayed on the central control screen. In autonomous driving mode, the relative motion trend between the obstacle and the vehicle is determined based on the time series of their distances. Then, based on this trend, the vehicle's direction and speed are controlled to move it away from the obstacle. For example, if the relative motion trend is to the left and forward, the vehicle's direction is controlled to deviate from this left-forward direction, and its speed is reduced to improve driving safety.

[0098] In the application scenario of automatically labeled data samples, sample labels are added to the time series of distances between obstacles and vehicles based on the collision risk prediction results. The sample labels include those indicating whether there is dynamic collision risk and those indicating no dynamic collision risk. This not only improves the labeling efficiency of data samples, but also ensures the objectivity and consistency of the data sample labeling results.

[0099] In application scenarios where autonomous driving functions are optimized using historical collision risk data, vehicle trajectory data and the trajectory data of at least one obstacle, the time series of distances between the obstacle and the vehicle, and collision risk prediction results are sent to data application parties such as simulation testing platforms to provide data support for the optimization of autonomous driving functions.

[0100] In one possible implementation, step 402 in the above embodiments includes the following steps A1-A2:

[0101] A1: For each vehicle trajectory point in the vehicle driving trajectory data, the obstacle trajectory point with the smallest time difference from the vehicle trajectory point in the obstacle running trajectory data is determined as the obstacle trajectory point that matches the vehicle trajectory point.

[0102] The time difference between the obstacle trajectory point and the vehicle trajectory point is the difference between the timestamp of the obstacle trajectory point and the timestamp of the vehicle trajectory point.

[0103] Through this processing step, for each vehicle trajectory point in the vehicle driving trajectory data, there is an obstacle trajectory point that matches the vehicle trajectory point in the trajectory data of each obstacle. Taking two obstacles as an example, the trajectory data of the two obstacles are B1 and B2 respectively. Each vehicle trajectory point must have a matching obstacle trajectory point in B1 and also a matching obstacle trajectory point in B2.

[0104] A2: The vehicle trajectory points with a time difference less than or equal to the second threshold and the matching obstacle trajectory points are determined as trajectory point combinations, and each trajectory point combination is sorted according to the time sequence to obtain the trajectory point combination sequence corresponding to the obstacle and the vehicle.

[0105] To avoid excessively long time differences between vehicle trajectory points and matched obstacle trajectory points, which could cause them to be outside the same sampling period, vehicle trajectory points and matched obstacle trajectory points with time differences greater than a second threshold are removed. In other words, only vehicle trajectory points and matched obstacle trajectory points with time differences less than or equal to the second threshold are identified as trajectory point combinations. The second threshold is set according to the sampling period, for example, it can be set to 0.5s.

[0106] It should be noted that the Hidden Markov Model (HMM)-based collision prediction model needs to be trained before application. This application provides a method for training a HMM-based collision risk prediction model; please refer to [link to relevant documentation]. Figure 5 , Figure 5 A flowchart illustrating a method for training a collision risk prediction model based on Hidden Markov Models, as provided in this application embodiment, is shown below. Figure 5 As shown, the specific steps include 501-505:

[0107] 501: Obtain historical data samples for the target user. The historical data samples include vehicle trajectory data samples within a preset time range before and after the vehicle exits autonomous driving in multiple scenarios, and trajectory data samples of at least one obstacle.

[0108] The scenarios here refer to the vehicle driving scenarios for the target user. Each scenario corresponds to a unique identifier, vid. Scenarios can be scenarios with dynamic collision risks or scenarios without dynamic collision risks.

[0109] 502: For each scenario, perform time-series relationship analysis on obstacle trajectory data samples and vehicle trajectory data samples to obtain trajectory point combination sequence samples corresponding to obstacles and vehicles. The trajectory point combination sequence samples include multiple trajectory point combinations arranged in time sequence. Each trajectory point combination includes: vehicle trajectory point and obstacle trajectory point matching the vehicle trajectory point; calculate the distance between the vehicle trajectory point and the obstacle trajectory point matching the vehicle trajectory point in each trajectory point combination to obtain the distance time series samples corresponding to obstacles and vehicles.

[0110] The specific implementation of step 502 is the same as that of steps 402-403 in the above embodiments, and will not be repeated here.

[0111] 503: In scenarios marked as having dynamic collision risk, the distance time series samples corresponding to obstacles and vehicles with dynamic collision risk are identified as positive samples, and the distance time series samples corresponding to other obstacles and vehicles in the scenario are identified as negative samples.

[0112] It should be noted that even in scenarios marked as having a dynamic collision risk, not every obstacle poses a dynamic collision risk to the vehicle. For example, in a certain scenario, there are three obstacles in front of the vehicle. There is a dynamic collision risk between the vehicle and one of the obstacles, but no dynamic collision risk between the vehicle and the other two obstacles.

[0113] In one possible implementation, the obstacle with the smallest average distance to the vehicle is identified as the obstacle posing a dynamic collision risk. Specifically, the average distance between the obstacle and the vehicle in the time series of distances is determined as the average distance between the obstacle and the vehicle.

[0114] 504: In scenarios that have been marked as having no dynamic collision risk, the time series samples of distances to obstacles and vehicles are identified as negative samples;

[0115] 505: Using a training dataset consisting of positive and negative samples, a Hidden Markov Model is trained to obtain a collision risk prediction model based on Hidden Markov Model.

[0116] Since different users react and take over autonomous driving to the risk of dynamic obstacle collisions, their reactions and the degree to which they take over autonomous driving vary significantly due to factors such as personal experience, driving habits, and risk perception ability, this embodiment uses historical data samples for the target user to train the model. Since the historical data samples for the target user include vehicle trajectory data samples within a preset time range before and after the vehicle exits autonomous driving in multiple scenarios, as well as trajectory data samples of at least one obstacle, the collision risk prediction model based on Hidden Markov Models can learn the relationship between distance time series samples and collision risk prediction results according to the individual characteristics of the target user, and adapt to the real collision risk prediction needs of the target user.

[0117] In one possible implementation, the Hidden Markov Model is trained using a training dataset, specifically including the following steps 5051-5054:

[0118] 5051: Constructing a Hidden Markov Model;

[0119] The Hidden Markov Model consists of two hidden states: one with collision risk and one without. The observation states corresponding to the two hidden states are time series samples of distances to obstacles and vehicles.

[0120] 5052: Initialize the model parameters of the Hidden Markov Model;

[0121] The model parameters of a Hidden Markov Model include: the transition probability matrix (A), the observation probability distribution (B), and the initial state probability vector (π).

[0122] The transition probability matrix (A) is of size N×(N+1) and represents the probability of transitioning from one hidden state to another. It is initialized using a uniform distribution or empirical data, where N is the number of hidden states.

[0123] The observation probability distribution (B) is of size N×M, representing the probability of observing different outcomes in each hidden state. For Gaussian mixture models, initial estimates of the mean and variance can be used, where M is the number of distances in the distance time series.

[0124] Initial state probability vector (π): of size 1×(N), representing the probability of being in each hidden state at the beginning.

[0125] 5053: Iteratively train the Hidden Markov Model using the training dataset, and update the model parameters during each iteration to obtain the change in the model parameters;

[0126] Specifically, the training of the Hidden Markov Model uses the forward-backward algorithm (Baum-Ettinger-Jensen algorithm), with the goal of maximizing the probability of the observed data.

[0127] The recurrence relation for the forward algorithm is as follows:

[0128]

[0129] in, Let i represent the probability of being in hidden state i at time t, and let j represent the probability of being in hidden state i at time t. Also, let j be the number of times the time series is t before the time series. Let N=2, and let i and j represent two hidden states respectively.

[0130] This represents the transition probability, which is the probability of transitioning from the previous state to the current state.

[0131] It represents the probability of observation, reflecting the correlation between the hidden state and the observation.

[0132] The recursive formula for the backward algorithm is as follows:

[0133]

[0134] in, Let i represent the probability of being in hidden state i at time t, and let j represent two hidden states. The observation distance time series is from t to T times, N=2.

[0135] The expected values ​​of the state transition probability and the observation probability are calculated based on the forward and backward probabilities mentioned above. The model parameters are then updated based on these expected values ​​to obtain the change between the updated model parameters and the original model parameters.

[0136] 5054: If the change in model parameters is less than the first threshold and / or the number of iterations of the Hidden Markov Model reaches the preset maximum number of iterations, the training of the Hidden Markov Model is terminated, and a collision risk prediction model based on Hidden Markov Model is obtained.

[0137] The above describes a collision risk prediction method provided by the embodiments of this application. The following describes the apparatus for performing the above-described collision risk prediction method.

[0138] Please see Figure 6 , Figure 6 This is a schematic diagram of a collision risk prediction device provided in an embodiment of this application. Figure 6 As shown, the collision risk prediction device includes:

[0139] The first acquisition unit 601 is used to acquire vehicle driving trajectory data and the running trajectory data of at least one obstacle;

[0140] Analysis unit 602 is used to perform time-series relationship analysis on the running trajectory data of the obstacle and the driving trajectory data of the vehicle to obtain the trajectory point combination sequence corresponding to the obstacle and the vehicle. The trajectory point combination sequence includes multiple trajectory point combinations arranged in time sequence. Each trajectory point combination includes: vehicle trajectory point and obstacle trajectory point matching the vehicle trajectory point.

[0141] The calculation unit 603 is used to calculate the distance between the vehicle trajectory point and the obstacle trajectory point that matches the vehicle trajectory point in each combination of trajectory points, so as to obtain the distance time series between the obstacle and the vehicle.

[0142] The prediction unit 604 is used to input the distance time series corresponding to the obstacle and the vehicle into the collision risk prediction model based on Hidden Markov Model to obtain the collision risk prediction result between the obstacle and the vehicle.

[0143] In one possible implementation, the collision risk prediction device further includes:

[0144] The second acquisition unit is used to acquire historical data samples for the target user. The historical data samples include vehicle driving trajectory data samples within a preset time range before and after the vehicle exits autonomous driving in multiple scenarios, and at least one obstacle's running trajectory data sample.

[0145] The preprocessing unit is used to perform time-series relationship analysis on obstacle trajectory data samples and vehicle driving trajectory data samples for each scenario, to obtain trajectory point combination sequence samples corresponding to obstacles and vehicles. The trajectory point combination sequence samples include multiple trajectory point combinations arranged in time sequence. Each trajectory point combination includes: vehicle trajectory point and obstacle trajectory point matching the vehicle trajectory point; the distance between the vehicle trajectory point and the obstacle trajectory point matching the vehicle trajectory point in each trajectory point combination is calculated to obtain distance time series samples corresponding to obstacles and vehicles.

[0146] The data segmentation unit is used to, in scenarios marked as having a dynamic collision risk, identify the distance time series samples corresponding to obstacles and vehicles with a dynamic collision risk as positive samples, and identify the distance time series samples corresponding to other obstacles and vehicles in the scenario as negative samples; in scenarios marked as not having a dynamic collision risk, the distance time series samples corresponding to obstacles and vehicles are identified as the negative samples.

[0147] The training unit is used to train the Hidden Markov Model using a training dataset consisting of the positive and negative samples to obtain the Hidden Markov-based collision risk prediction model.

[0148] In one possible implementation, the training unit is specifically used to construct a Hidden Markov Model (HMM), which includes two hidden states: one indicating a collision risk and the other indicating a lack of collision risk. The observed states corresponding to the two hidden states are time-series distance samples of obstacles and vehicles. The model parameters of the HMM are initialized. The HMM is iteratively trained using the training dataset, and the model parameters are updated during each iteration to obtain the change in the model parameters. The training of the HMM ends when the change in the model parameters is less than a first threshold and / or the number of iterations of the HMM reaches a preset maximum number of iterations, thus obtaining the collision risk prediction model based on the Hidden Markov Model.

[0149] In one possible implementation, the collision risk prediction device further includes:

[0150] The control unit is configured to, when the vehicle is in autonomous driving mode, determine the relative motion trend of the obstacle and the vehicle based on the distance-time series corresponding to the obstacle and the vehicle if the collision risk prediction result indicates that there is a dynamic collision risk between the obstacle and the vehicle; and control the running direction and speed of the vehicle based on the relative motion trend of the obstacle and the vehicle.

[0151] In one possible implementation, the collision risk prediction device further includes:

[0152] The alert unit is used to issue a collision risk alert message when the vehicle is in manual driving mode and the collision risk prediction result indicates that there is a dynamic collision risk between the obstacle and the vehicle.

[0153] In one possible implementation, the analysis unit 602 is specifically used to, for each vehicle trajectory point in the vehicle driving trajectory data, determine the obstacle trajectory point in the obstacle's running trajectory data that has the smallest time difference with the vehicle trajectory point as the obstacle trajectory point that matches the vehicle trajectory point; determine the vehicle trajectory points and matching obstacle trajectory points with a time difference less than or equal to a second threshold as the trajectory point combination, and sort each trajectory point combination according to time sequence to obtain the trajectory point combination sequence corresponding to the obstacle and the vehicle.

[0154] This embodiment discloses a collision risk prediction device that performs time-series relationship analysis on obstacle trajectory data and vehicle driving trajectory data to obtain a sequence of trajectory point combinations corresponding to the obstacle and the vehicle. Then, based on the vehicle trajectory points and the obstacle trajectory points matching the vehicle trajectory points in the multiple trajectory point combinations arranged in time sequence, a distance time series corresponding to the obstacle and the vehicle is obtained. Since the distance time series includes the distances between the obstacle and the vehicle arranged in time sequence, it represents the relative motion trend between the obstacle and the vehicle. By inputting the distance time series corresponding to the obstacle and the vehicle into a Hidden Markov Model (HMM), the HMM-based collision risk prediction model can accurately predict collision risks based on the relative motion trend between the obstacle and the vehicle, improving driving safety and providing data support for autonomous driving.

[0155] This application embodiment also provides an electronic device, including at least one processor and a memory connected to the processor, wherein:

[0156] The memory is used to store computer programs;

[0157] The processor is used to execute the computer program so that the electronic device can implement any of the collision risk prediction methods provided in the embodiments of this application.

[0158] The electronic device provided in this embodiment can be the vehicle-mounted device 100 or the network-side device 200 in the above embodiments.

[0159] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the collision risk prediction methods provided in this application.

[0160] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the collision risk prediction methods provided in this application.

[0161] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0162] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0163] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0164] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A collision risk prediction method, characterized in that, include: Acquire vehicle trajectory data and the trajectory data of at least one obstacle; A temporal relationship analysis is performed on the trajectory data of the obstacle and the trajectory data of the vehicle to obtain a sequence of trajectory point combinations corresponding to the obstacle and the vehicle. The sequence of trajectory point combinations includes multiple trajectory point combinations arranged in temporal order. Each trajectory point combination includes: a vehicle trajectory point and an obstacle trajectory point that matches the vehicle trajectory point. Calculate the distance between the vehicle trajectory point and the obstacle trajectory point that matches the vehicle trajectory point in each combination of trajectory points to obtain the time series of distances between the obstacle and the vehicle; The time series of distances between the obstacle and the vehicle are input into a collision risk prediction model based on Hidden Markov Model (HMM) to obtain the collision risk prediction results between the obstacle and the vehicle.

2. The collision risk prediction method according to claim 1, characterized in that, The method for training the Hidden Markov Model-based collision risk prediction model includes: Acquire historical data samples for the target user, including vehicle trajectory data samples within a preset time range before and after the vehicle exits autonomous driving in multiple scenarios, and trajectory data samples of at least one obstacle. For each scenario, a time-series relationship analysis is performed on the obstacle trajectory data samples and vehicle driving trajectory data samples to obtain trajectory point combination sequence samples corresponding to obstacles and vehicles. The trajectory point combination sequence samples include multiple trajectory point combinations arranged in time sequence. Each trajectory point combination includes: vehicle trajectory point and obstacle trajectory point matching the vehicle trajectory point. The distance between the vehicle trajectory point and the obstacle trajectory point matching the vehicle trajectory point in each trajectory point combination is calculated to obtain the distance time series samples corresponding to obstacles and vehicles. In scenarios marked as having dynamic collision risk, the distance time series samples corresponding to obstacles and vehicles with dynamic collision risk are identified as positive samples, while the distance time series samples corresponding to other obstacles and vehicles in the scenario are identified as negative samples. In scenarios that have been marked as having no dynamic collision risk, the time series samples of distances to obstacles and vehicles are determined as the negative samples; The Hidden Markov Model is trained using a training dataset consisting of the positive and negative samples to obtain the Hidden Markov Collision Risk Prediction Model.

3. The collision risk prediction method according to claim 2, characterized in that, The step of training a Hidden Markov Model using a training dataset consisting of the positive and negative samples to obtain the Hidden Markov-based collision risk prediction model includes: A hidden Markov model is constructed, which includes two hidden states, one with collision risk and one without collision risk; the observation states corresponding to the two hidden states are time series samples of distances to obstacles and vehicles. The model parameters of the Hidden Markov Model are initialized; The Hidden Markov Model is iteratively trained using the training dataset, and the model parameters are updated during each iteration to obtain the changes in the model parameters. If the change in the model parameters is less than a first threshold and / or the number of iterations of the Hidden Markov Model reaches a preset maximum number of iterations, the training of the Hidden Markov Model ends, and the Hidden Markov-based collision risk prediction model is obtained.

4. The collision risk prediction method according to any one of claims 1-3, characterized in that, After obtaining the collision risk prediction result between the obstacle and the vehicle, the collision risk prediction method further includes: When the vehicle is in autonomous driving mode, if the collision risk prediction result indicates that there is a dynamic collision risk between the obstacle and the vehicle, then the relative motion trend between the obstacle and the vehicle is determined based on the distance time series corresponding to the obstacle and the vehicle. The vehicle's direction and speed are controlled based on the relative motion trend between the obstacle and the vehicle.

5. The collision risk prediction method according to any one of claims 1-3, characterized in that, After obtaining the collision risk prediction result between the obstacle and the vehicle, the collision risk prediction method further includes: When the vehicle is in manual driving mode, if the collision risk prediction result indicates that there is a dynamic collision risk between the obstacle and the vehicle, a collision risk warning message will be issued.

6. The collision risk prediction method according to claim 1, characterized in that, A time-series relationship analysis is performed on the trajectory data of the obstacle and the trajectory data of the vehicle to obtain a sequence of trajectory point combinations corresponding to the obstacle and the vehicle, including: For each vehicle trajectory point in the vehicle trajectory data, the obstacle trajectory point with the smallest time difference from the vehicle trajectory point in the obstacle's trajectory data is determined as the obstacle trajectory point that matches the vehicle trajectory point. Vehicle trajectory points with a time difference less than or equal to a second threshold and matching obstacle trajectory points are determined as the trajectory point combinations, and each trajectory point combination is sorted according to time sequence to obtain the trajectory point combination sequence corresponding to the obstacle and the vehicle.

7. A collision risk prediction device, characterized in that, include: The first acquisition unit is used to acquire vehicle trajectory data and the trajectory data of at least one obstacle. The analysis unit is used to perform time-series relationship analysis on the trajectory data of the obstacle and the trajectory data of the vehicle to obtain a sequence of trajectory point combinations corresponding to the obstacle and the vehicle. The sequence of trajectory point combinations includes multiple trajectory point combinations arranged in time sequence. Each trajectory point combination includes: a vehicle trajectory point and an obstacle trajectory point that matches the vehicle trajectory point. The calculation unit is used to calculate the distance between the vehicle trajectory point and the obstacle trajectory point that matches the vehicle trajectory point in each combination of trajectory points, and to obtain the time series of the distance between the obstacle and the vehicle. The prediction unit is used to input the distance time series corresponding to the obstacle and the vehicle into the collision risk prediction model based on Hidden Markov Model to obtain the collision risk prediction result between the obstacle and the vehicle.

8. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the collision risk prediction method as described in any one of claims 1 to 6.

9. A computer program product, characterized in that, Includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the collision risk prediction method as described in any one of claims 1 to 6.

10. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the collision risk prediction method as described in any one of claims 1 to 6.

Images