Driving risk prediction method and device, electronic equipment and storage medium

Abstract

The application discloses a driving risk prediction method and device, electronic equipment and a storage medium. The driving risk prediction method is applied to a first vehicle and comprises the following steps: acquiring vehicle monitoring data, extracting risk features from the vehicle monitoring data through an encoder; inputting the risk features into a risk identification model to determine at least one high-risk target and risk information corresponding to each high-risk target; and inputting the risk features corresponding to each high-risk target into a risk prediction model to predict a low-risk driving area of the first vehicle in a preset future time period. The application solves the technical problem of low driving risk prediction accuracy in the related art.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a method, device, electronic device and storage medium for predicting driving risks. Background Technology

[0002] With the increasing number of cars and the growing number of drivers in China, the road safety situation is becoming increasingly serious. Driving risk prediction algorithms play a crucial role in vehicle active safety systems, and are of great significance for improving traffic safety and protecting the lives and property of passengers.

[0003] The driving risk prediction methods in related technologies are mainly based on the dynamic parameters of traffic participants. These methods use parameters such as distance, speed, and acceleration to calculate risk assessment indicators. They are computationally efficient, but they are difficult to cope with changing real-world situations. For example, when a vehicle is moving forward, there may be no other vehicles within a certain distance ahead, so the driving risk is judged to be low. However, if vehicles suddenly change lanes on either side or a vehicle suddenly moves in front of the vehicle at an intersection, these sudden changes cannot be predicted by the vehicle's dynamic parameters, resulting in low accuracy in driving risk prediction. Summary of the Invention

[0004] The main objective of this application is to provide a method, device, electronic device, and storage medium for predicting driving risks, aiming to solve the technical problem of low accuracy in driving risk prediction in related technologies.

[0005] To achieve the above objectives, this application provides a driving risk prediction method, which is applied to a first vehicle and includes the following steps:

[0006] Acquire vehicle monitoring data, and extract risk characteristics from the vehicle monitoring data using an encoder;

[0007] By inputting the risk characteristics into a risk identification model, at least one high-risk target and the risk information corresponding to each of the high-risk targets are identified.

[0008] The risk characteristics corresponding to each of the high-risk targets are input into the risk prediction model to predict the low-risk driving area of ​​the first vehicle in a preset future time period.

[0009] This application also provides a driving risk prediction device, which is applied to a first vehicle and includes:

[0010] The acquisition module is used to acquire vehicle monitoring data and extract risk features from the vehicle monitoring data through an encoder;

[0011] The risk identification module is used to identify at least one high-risk target and the risk information corresponding to each of the high-risk targets by inputting the risk characteristics into the risk identification model.

[0012] The risk prediction module is used to input the risk characteristics corresponding to each of the high-risk targets into the risk prediction model to predict the low-risk driving area of ​​the first vehicle in a preset future time period.

[0013] This application also provides an electronic device, which is a physical device, comprising: a memory, a processor, and a program of the driving risk prediction method stored in the memory and executable on the processor. When the program of the driving risk prediction method is executed by the processor, it can implement the steps of the driving risk prediction method as described above.

[0014] This application also provides a storage medium, which is a computer-readable storage medium, on which a program for implementing the driving risk prediction method is stored. When the program for the driving risk prediction method is executed by a processor, it implements the steps of the driving risk prediction method as described above.

[0015] This application provides a driving risk prediction method, device, electronic device, and storage medium. The driving risk prediction method is applied to a first vehicle. First, vehicle monitoring data is acquired, and risk features are extracted from the vehicle monitoring data through an encoder, thus realizing the extraction of risk features. Then, by inputting the risk features into a risk identification model, at least one high-risk target and the risk information corresponding to each of the high-risk targets are identified, thus realizing the identification of high-risk targets. Then, the risk features corresponding to each of the high-risk targets are input into the risk prediction model to predict the low-risk driving area of ​​the first vehicle in a preset future time period, thus realizing the prediction of the low-risk driving area of ​​the first vehicle in the preset future time period. Therefore, a phased, progressive, multi-target driving risk prediction is realized. The first stage realizes the rapid identification of high-risk targets, and the second stage realizes the accurate prediction of low-risk driving areas. In this way, on the one hand, compared with the method of predicting driving risks based on the dynamic parameters of traffic participants, this application can make full use of various available vehicle monitoring data, such as vehicle speed, vehicle acceleration, driver attention distribution, external road condition data, external environment data, pedestrian walking speed, pedestrian position, other vehicle position, other vehicle speed, etc. These vehicle monitoring data carry information that may change, such as the state of other traffic participants, the state of the driver, and the state of road conditions. The encoder can extract this information that may change and encode it into risk features for high-risk target identification and low-risk driving area prediction. Thus, the potentially changing information can be fully considered in the driving risk prediction process, thereby effectively improving the accuracy of driving risk prediction. On the other hand, compared with the method of risk identification by predicting movement trajectory, the phased, progressive, multi-target driving risk prediction does not need to predict the movement trajectory or position information of all risk targets in the first stage. Therefore, it can effectively reduce the amount of computation in the first stage, shorten the time to identify high-risk targets, and give the driver more time to respond, thereby improving driving safety. Therefore, it overcomes the technical shortcomings of predicting sudden changes by predicting the vehicle's dynamic parameters, which cannot predict these changes and thus leads to low accuracy in predicting travel risks. It improves the accuracy of driving risk prediction and thus enhances the safety of vehicle driving. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the first embodiment of the driving risk prediction method of this application;

[0019] Figure 2 This is a schematic diagram illustrating one possible implementation of a deep learning model based on the Transformer structure in this application.

[0020] Figure 3 This is a flowchart illustrating the second embodiment of the driving risk prediction method of this application;

[0021] Figure 4 This is a schematic diagram of the driving risk prediction device in the embodiments of this application;

[0022] Figure 5 This is a schematic diagram of the equipment structure of the hardware operating environment involved in the driving risk prediction method in the embodiments of this application.

[0023] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0024] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] With the increasing number of cars and the growing number of drivers in China, the road safety situation is becoming increasingly serious. Driving risk prediction algorithms play a crucial role in vehicle active safety systems, and are of great significance for improving traffic safety and protecting the lives and property of passengers.

[0026] The driving risk prediction methods in related technologies are mainly based on the dynamic parameters of traffic participants. These methods use parameters such as distance, speed, and acceleration to calculate risk assessment indicators. They are computationally efficient, but they are difficult to cope with changing real-world situations. For example, when a vehicle is moving forward, there may be no other vehicles within a certain distance ahead, so the driving risk is judged to be low. However, if vehicles suddenly change lanes on either side or a vehicle suddenly moves in front of the vehicle at an intersection, these sudden changes cannot be predicted by the vehicle's dynamic parameters, resulting in low accuracy in driving risk prediction.

[0027] In related technologies, artificial intelligence technology has been proposed to predict the trajectories of traffic participants in order to predict driving risks. However, such methods are largely limited by the accuracy of the prediction algorithm. Improving the accuracy of the prediction algorithm requires a large number of training samples. However, there are relatively few cases of collision accidents, so there is a lack of training samples. This is especially true for complex traffic accident scenarios with many traffic participants, where trajectory prediction is even more difficult.

[0028] With the rapid development of large-scale deep learning models, especially those based on the Transformer architecture, these models, leveraging attention mechanisms, possess strong multi-scenario and multi-objective fusion capabilities, enabling them to understand rich semantic information and handle more complex tasks. Therefore, the Transformer architecture can be used to fuse traffic scene information, information from other traffic participants, and vehicle monitoring data such as the driver's state, to more accurately and quickly identify high-risk targets and more precisely predict low-risk driving areas. This model will be better able to cope with changing traffic environments, improving the accuracy and practicality of predictions.

[0029] This application constructs a deep learning model based on the Transformer architecture for end-to-end driving risk prediction. The deep learning model based on the Transformer architecture includes at least an encoder, a risk identification model, and a risk prediction model. The encoder is used to extract, fuse, and encode features from various modalities and types of vehicle monitoring data. This vehicle monitoring data carries information that may change, such as the states of other traffic participants, the driver's state, and road conditions. The encoder can extract this potentially changing information and encode it into risk features for high-risk target identification and low-risk driving area prediction. This allows for full consideration of potentially changing information during driving risk prediction, thus effectively improving the accuracy of driving risk prediction. Both the risk identification model and the risk prediction model can apply attention mechanisms or other methods that can simultaneously focus on multiple targets. The target technology enables phased, progressive, and multi-target vehicle risk prediction. The first stage allows for rapid identification of high-risk targets, while the second stage enables accurate prediction of low-risk driving areas. Therefore, compared to risk identification based on predicted movement trajectories, this phased, progressive, and multi-target vehicle risk prediction method eliminates the need to predict the movement trajectories or location information of all risk targets in the first stage. This effectively reduces the computational load in the first stage, shortens the time to identify high-risk targets, and provides drivers with more time to react, thus improving driving safety. Furthermore, compared to methods that only identify or predict the movement trajectories of risk targets, in complex scenarios involving multiple high-risk targets, by fusing the movement trajectories of multiple high-risk targets to determine low-risk driving areas, the driver's manual judgment is reduced, allowing them to focus more on driving and improving safety in complex scenarios involving multiple high-risk targets. Therefore, this technology overcomes the technical shortcomings of methods that predict vehicle dynamics parameters, which cannot predict these sudden changes, resulting in lower accuracy in risk prediction. This improves the accuracy of vehicle risk prediction, thereby enhancing driving safety.

[0030] Example 1

[0031] This application provides a method for predicting driving risks, referring to... Figure 1 In the first embodiment of the driving risk prediction method of this application, the driving risk prediction method is applied to a first vehicle, and the driving risk prediction method includes the following steps:

[0032] Step S10: Obtain vehicle monitoring data and extract risk features from the vehicle monitoring data using an encoder;

[0033] The subject executing the method in this embodiment can be a driving risk prediction device, a driving risk prediction terminal device, or a server. This embodiment takes a driving risk prediction device as an example. The driving risk prediction device can be integrated into terminal devices such as vehicles, in-vehicle terminals, smartphones, and computers with data processing functions.

[0034] In this embodiment, it should be noted that the driving risk prediction method is applied to the first vehicle. The driving risk prediction device is equipped with a deep learning model based on the Transformer structure, which is used to perform end-to-end driving risk prediction for the first vehicle. The deep learning model based on the Transformer structure includes at least an encoder, a risk identification model, and a risk prediction model.

[0035] In one feasible approach, refer to Figure 2 The deep learning model based on the Transformer architecture includes a perception module, a risk identification module, a risk prediction module, and a communication module. The perception module includes an environmental perception module and a cockpit perception module. The environmental perception module includes a bird's-eye view encoding unit, a map encoding unit, and a tracking encoding unit. The cockpit perception module includes a driver attention distribution encoding unit and a vehicle driving time sequence encoding unit. The risk identification module includes a high-risk target identification unit and a risk type identification unit. The risk prediction module includes a high-risk target trajectory prediction unit and a low-risk driving area prediction unit. The communication module is used to collect a set of high-risk targets and low-risk drivable areas and output corresponding prompts.

[0036] The vehicle monitoring data refers to the data required for predicting driving risks, including one or more of the following: the speed of the first vehicle, the acceleration of the first vehicle, the driver's attention distribution in the first vehicle's cabin, the state or actions of the occupants in the first vehicle's cabin, external road condition data, external environment data, pedestrian walking speed, pedestrian position, the position of other vehicles besides the first vehicle, and the speed of other vehicles besides the first vehicle. The vehicle monitoring data can be one or more modalities such as image data, text data, sound data, and signal data. For example, it can collect external image data and cabin image data through a camera, external sound data and cabin sound data through a microphone, convert the collected speech into text data, extract text data from the collected images, and collect signal data transmitted between various modules of the first vehicle and signal data generated by vehicle movement. This vehicle monitoring data can be collected in real time, thus carrying real-time information on the state of other road users, the driver's state, road conditions, and other potentially changing information at the current moment.

[0037] The encoder is used to extract, fuse, and encode features from various modalities and types of vehicle monitoring data. Through the encoder, real-time information on potentially changing data can be extracted and encoded into risk features for high-risk target identification and low-risk driving area prediction. This allows for full consideration of potentially changing information during driving risk prediction, thus effectively improving the accuracy of driving risk prediction. One or more encoders can be used, and the specific number can be determined according to actual needs; this embodiment does not impose any limitations on this.

[0038] The risk characteristics can include external environmental characteristics and cabin characteristics. Cabin characteristics can further include vehicle driving characteristics, driver attention distribution characteristics, driver state characteristics, and occupant state characteristics. Changes in driver attention distribution characteristics have a significant impact on driving risk. For example, when a driver turns their head to the right, their attention is focused on the right side, making it difficult to detect risks on the left, thus greatly increasing the probability of driving risks on the left. Therefore, this can increase the probability of identifying objects on the left (such as fast-moving or small objects) as high-risk targets, promptly alerting the driver. Occupant state characteristics and driver state characteristics may also influence risk formation. For example, in emergency situations, some drivers remain calm while others may react aggressively. Similarly, if all occupants are asleep, the driver may also be affected. These factors all influence driving risk and can therefore be used as risk characteristics to predict driving risks.

[0039] As an example, step S10 includes: acquiring vehicle monitoring data at the current moment, inputting the vehicle monitoring data into the encoder for feature extraction, feature fusion and feature encoding to obtain an encoding matrix, wherein the risk feature refers to the encoding matrix.

[0040] Optionally, the vehicle monitoring data includes an external environment image sequence, a cabin image sequence, and a vehicle driving data sequence; the risk features include external environment features and cabin features, wherein the cabin features include driver attention distribution features and vehicle driving features; the encoder includes a bird's-eye view encoder, a cabin image encoder, and a vehicle driving time sequence encoder.

[0041] The step of extracting risk features from the vehicle monitoring data using an encoder includes:

[0042] The bird's-eye view encoder extracts external environment features from the external environment image sequence, the cockpit image encoder extracts driver attention distribution features from the cockpit image sequence, and the vehicle driving time sequence encoder extracts vehicle driving features from the vehicle driving data sequence.

[0043] In this embodiment, it should be noted that different encoders can be used for feature extraction of vehicle monitoring data of different modalities. For example, convolutional neural networks can be used for feature extraction of image data, and long short-term memory networks can be used for feature extraction of time-series data. Different risk features can also be extracted using different encoders. For example, map features can be extracted by an encoder to extract map elements, and driver attention distribution features can be extracted by an encoder to extract driver information.

[0044] The risk characteristics include external environment characteristics and cabin characteristics. The external environment characteristics can be extracted from one or more frames of external environment image data captured by a camera. These external environment characteristics may include road information and information about other traffic participants. While road information changes relatively little, it can be altered by sudden accidents, road construction, or other unforeseen circumstances. Since traffic participants are more likely to be on the road, information about other traffic participants changes significantly. Changes in the information of other traffic participants can alter the probability of risk to the first vehicle. For example, a vehicle initially traveling straight in the adjacent lane may have a lower risk probability, but if it suddenly turns and merges in front of the first vehicle, its risk probability increases considerably. The cabin characteristics include driver attention distribution characteristics and vehicle movement characteristics. These cabin characteristics can... The system acquires one or more frames of cockpit image data via a camera, and then extracts the data from these frames. Changes in the driver's attention distribution characteristics have a significant impact on driving risk. For example, when the driver turns their head to the right, their attention is focused on the right side, making it difficult to detect potential risks on the left. This greatly increases the probability of identifying objects on the left (such as fast-moving or small objects) as high-risk targets, allowing for timely driver alerts. The vehicle driving characteristics can be obtained from vehicle driving data at one or more time points acquired from the vehicle controller or controller area network bus, and then extracted from this data. The vehicle driving data includes speed, acceleration, and steering information. The vehicle driving characteristics characterize the vehicle's driving motivation, such as acceleration, steering, and parking. In one feasible implementation, the vehicle driving timing encoder can be a long short-term memory network.

[0045] As an example, the collected external environment image data can be arranged sequentially into an external environment image sequence, and the external environment image sequence can be input sequentially into the bird's-eye view encoder to extract the external environment features in the form of a bird's-eye view. Simultaneously, the collected cabin image data can also be arranged sequentially into a cabin image sequence, and the cabin image sequence can be input sequentially into a cabin image encoder to extract driver action information, thereby determining the driver's attention distribution characteristics. Simultaneously, the collected vehicle driving data can also be arranged sequentially into a vehicle driving data sequence, and the vehicle driving data sequence can be input sequentially into the vehicle driving time sequence encoder to extract vehicle driving features.

[0046] Optionally, the encoder further includes a map encoder and a tracking encoder; the risk features further include map features and tracking features;

[0047] Following the step of extracting exterior environment features from the vehicle exterior image sequence using the bird's-eye view encoder, the method further includes:

[0048] The map encoder extracts map features from the external environment features, and the tracking encoder performs target detection and multi-target tracking based on the external environment features to obtain tracking features of at least one potential risk target.

[0049] In this embodiment, it should be noted that driving risk prediction is usually based on trends. For example, if a potential risk target shows a tendency to approach the first vehicle, the risk probability of the potential risk target can be considered to have increased. However, for potential risk targets that are subject to change, information at a single moment is often insufficient to reflect their changing trends. Therefore, using information from a single moment for driving risk prediction will result in lower accuracy. Tracking features are used to characterize the trajectory of a target over a certain time range, containing information about the target's movement trends. Therefore, obtaining the tracking features of each potential risk target through target tracking and using these tracking features to identify high-risk targets and predict low-risk driving areas yields higher accuracy.

[0050] As an example, the map encoder can extract sparsely represented road elements from the external environment features, thereby converting them into map features. The map features encode location and structural information, which can support subsequent recognition and prediction tasks. Simultaneously, the external environment features can be input into the tracking encoder. First, target detection is performed through image recognition to identify at least one potential risk target. Then, a cross-attention mechanism is used to perform multi-target tracking on each potential risk target to determine the tracking features of each potential risk target.

[0051] Step S20: By inputting the risk characteristics into the risk identification model, at least one high-risk target and the risk information corresponding to each of the high-risk targets are determined.

[0052] In this embodiment, it should be noted that the risk identification model refers to a model used to identify high-risk targets. In one feasible approach, since the various risk targets are interconnected and influence each other, and each data point in the vehicle monitoring data may be incomplete (e.g., obstructions in the acquired images prevent the identification of some features), the risk identification model adopts a transformer structure. This structure can effectively track high-risk targets, thereby improving the accuracy of high-risk target identification. The transformer-structured risk identification model may include one or more structures such as attention layers, multilayer perceptron layers, and feedforward network layers, which can be determined according to actual needs. This embodiment does not impose any limitations on this. The risk information refers to the relevant information generated during the process of identifying each high-risk target as a high-risk target. In one feasible approach, the risk identification model is a classification model. The risk information may include risk type and risk probability. The risk type can be a classification label for high-risk targets, such as forward collisions, rear-end collisions, crossing collisions, collisions with fixed objects, left-turn traffic conflicts, right-turn traffic conflicts, and crossing traffic conflicts. The risk probability refers to the probability of being identified as a high-risk target. After inputting the risk features into the risk identification model, the model can classify each potential risk target based on the risk features, determine the classification label for each potential risk target, and calculate the risk probability of each potential risk target being identified as its corresponding category. Potential risk targets with a risk probability higher than a preset probability threshold are identified as high-risk targets, and the classification label with a risk probability higher than the preset probability threshold is determined as the risk type of the corresponding high-risk target. This allows for the identification of high-risk targets and the determination of their risk type and risk probability. The high-risk targets include one or more of traffic participants, traffic infrastructure, etc.

[0053] In one feasible approach, the risk identification model includes a multi-head attention layer, which can simultaneously focus on multiple risk targets, thereby facilitating the detection of high-risk targets and the tracking of multiple high-risk targets.

[0054] As an example, step S20 includes: inputting the risk features into the risk identification model to identify multiple potential risk targets; determining whether there is a high-risk target at the current moment by comparing whether the confidence level of each potential risk target is higher than a confidence threshold; if there is no high-risk target at the current moment, the step of obtaining vehicle monitoring data can be returned to continuously predict driving risks; if there is at least one high-risk target at the current moment, the risk information corresponding to each high-risk target is obtained so that the user can respond to potential driving risks in a timely manner based on the risk information corresponding to each high-risk target.

[0055] Optionally, after the step of determining at least one high-risk target and the risk information corresponding to each of the high-risk targets by inputting the risk characteristics into the risk identification model, the method further includes:

[0056] Based on the risk information corresponding to each of the high-risk targets, corresponding risk warning information is generated and output, wherein the risk information includes risk type and risk probability.

[0057] In this embodiment, after identifying high-risk targets in the first stage, corresponding risk warning information can be generated immediately based on the risk information corresponding to each of the high-risk targets. The risk warning information is then output in the form of images, sounds, text, etc., so that users can be aware of the current driving risks in a timely manner. Based on the risk information corresponding to each of the high-risk targets, users can respond to and resolve the current driving risks in a timely manner, which increases the interaction with the driver and provides more comprehensive and accurate protection for driving safety.

[0058] In one feasible approach, the risk warning information can be output on a preset bird's-eye view map. For example, each of the high-risk targets can be highlighted on the preset bird's-eye view map, such as by highlighting or filling it with different colors. The risk type and risk probability can be directly marked near each of the high-risk targets on the preset bird's-eye view map, or they can be distinguished by different display methods, such as darker colors indicating higher risk probabilities, and different colors representing different risk types.

[0059] Step S30: Input the risk characteristics corresponding to each of the high-risk targets into the risk prediction model to predict the low-risk driving area of ​​the first vehicle in a preset future time period.

[0060] In this embodiment, it should be noted that the risk prediction model refers to a model used to predict the low-risk driving area of ​​the first vehicle in a preset future time period. The risk identification model can be a time series model, such as a recurrent neural network or a long short-term memory network. The risk identification model can also adopt a transformer structure. The transformer structure can better integrate risk features of different models and types, more accurately track high-risk targets, and thus more accurately predict the movement trajectory of high-risk targets in the preset future time period. Therefore, it can more accurately determine the low-risk driving area. The risk identification model of the transformer structure can include one or more of attention layers, multilayer perceptron layers, feedforward network layers, etc., which can be determined according to actual needs. This embodiment does not impose any restrictions on this. The risk information refers to the relevant information generated during the process of identifying each high-risk target as a high-risk target.

[0061] In one feasible approach, the risk identification model is a classification model. The risk information may include risk type and risk probability. The risk type can be a classification label for high-risk targets, such as forward collision, rear-end collision, crossing collision, collision with a fixed object, left-turn traffic collision, right-turn traffic collision, and crossing traffic collision. The risk probability refers to the probability of being identified as a high-risk target. After inputting the risk features into the risk identification model, the model can classify each potential risk target based on the risk features, determine the classification label for each potential risk target, and calculate the risk probability of each potential risk target being identified as the corresponding category. Potential risk targets with a risk probability higher than a preset probability threshold are identified as high-risk targets, and the classification label with a risk probability higher than the preset probability threshold is determined as the risk type of the corresponding high-risk target. Thus, high-risk targets can be identified, and the risk type and risk probability of high-risk targets can be determined.

[0062] In one feasible approach, the risk prediction model includes a multi-head attention layer, which can simultaneously focus on multiple risk targets, thereby facilitating the detection of high-risk targets and the tracking of multiple high-risk targets. By integrating the driving trajectories of multiple high-risk targets over a preset future time period, low-risk driving areas can be determined more accurately and quickly.

[0063] As an example, step S30 includes: inputting the risk characteristics corresponding to each of the high-risk targets into the risk prediction model, predicting the location information of each of the high-risk targets in a preset future time period through the risk prediction model, such as the area where they are located and their movement trajectory, and thus determining the area that each of the high-risk targets is less likely to reach in the preset future time period as the low-risk driving area of ​​the first vehicle in the preset future time period based on the location information of each of the high-risk targets; alternatively, the risk prediction model can be used to predict the low-risk driving area that each of the high-risk targets is less likely to reach in the preset future time period.

[0064] In one feasible approach, after the low-risk driving area is predicted and determined in the second stage, the low-risk driving area can be output in the form of images, sound, text, etc., for users to refer to in order to deal with and resolve the current driving risks. This increases the interaction with the driver and provides more comprehensive and accurate protection for driving safety.

[0065] In one feasible approach, the low-risk driving area can be output on a preset bird's-eye view map. For example, the low-risk driving area can be highlighted on the preset bird's-eye view map, such as by highlighting it or filling it with different colors.

[0066] In this embodiment, a deep learning model based on the Transformer architecture is constructed to achieve end-to-end driving risk prediction. This deep learning model includes at least an encoder, a risk identification model, and a risk prediction model. The encoder extracts and encodes features from various modalities and types of vehicle monitoring data. This vehicle monitoring data carries information that may change, such as the states of other traffic participants, the driver's state, and road conditions. The encoder extracts this potentially changing information and encodes it into risk features for high-risk target identification and low-risk driving area prediction. This allows for full consideration of potentially changing information during driving risk prediction, thus effectively improving the accuracy of driving risk prediction. Both the risk identification model and the risk prediction model can apply attention mechanisms or other mechanisms that can simultaneously focus on specific information. This technology, targeting multiple targets, enables phased, progressive, and multi-target vehicle risk prediction. The first stage allows for rapid identification of high-risk targets, while the second stage enables accurate prediction of low-risk driving areas. Therefore, compared to risk identification based on predicted movement trajectories, this phased, progressive, and multi-target vehicle risk prediction method eliminates the need to predict the movement trajectories or location information of all risk targets in the first stage. This effectively reduces the computational load in the first stage, shortens the time to identify high-risk targets, and provides drivers with more time to react, thus improving driving safety. Furthermore, compared to methods that only identify or predict the movement trajectories of risk targets, in complex scenarios involving multiple high-risk targets, by fusing the movement trajectories of multiple high-risk targets to determine low-risk driving areas, the driver's manual judgment is reduced, allowing them to focus more on driving and improving safety in complex scenarios involving multiple high-risk targets. Therefore, this method overcomes the technical shortcomings of methods that predict sudden changes in vehicle dynamics parameters, which often result in low accuracy in risk prediction, thus improving the accuracy of vehicle risk prediction and ultimately enhancing driving safety.

[0067] Example 2

[0068] Furthermore, in the second embodiment of this application, content that is the same as or similar to the above embodiments can be referred to the above description, and will not be repeated hereafter. Based on this, refer to... Figure 3 The risk identification model includes a multi-head self-attention layer, a first multi-head cross-attention layer, a second multi-head cross-attention layer, a first multi-head deformable attention layer, and a first multilayer perceptron.

[0069] The step of determining at least one high-risk target and the risk information corresponding to each of the high-risk targets by inputting the risk characteristics into a risk identification model includes:

[0070] Step S21: Input the tracking features of each potential risk target into the multi-head self-attention layer to obtain tracking interaction features. Input the tracking interaction features and the map features into the first multi-head cross-attention layer for feature fusion to determine the risk target encoding features and risk target location features of each potential risk target. Input the vehicle external environment features and the cabin features into the second multi-head cross-attention layer for feature fusion to obtain the first fused features.

[0071] In this embodiment, it should be noted that driving risk prediction is usually based on trends. For example, if a potential risk target shows a trend of approaching the first vehicle, the risk probability of the potential risk target can be considered to increase. However, for potential risk targets that are subject to change, static information at a single moment is often insufficient to reflect their changing trends. Therefore, using static information at a single moment for driving risk prediction will result in low accuracy. In complex traffic accident scenarios with many road users, the interaction between multiple road users is often a significant factor affecting the accuracy of driving risk prediction.

[0072] The risk identification model adopts a transformer structure and includes at least a multi-head self-attention layer, a first multi-head cross-attention layer, a second multi-head cross-attention layer, a first multi-head deformable attention layer, and a first multi-layer perceptron. Each of these layers can be a single layer or multiple layers, depending on actual needs; this embodiment does not impose any limitations. The risk target location features of each potential risk target characterize the impact of other traffic participants on the driving risk of the first vehicle. The first fusion feature characterizes the impact of the first vehicle's own state and its environment on the driving risk of the first vehicle. Fusing these two features allows for driving risk prediction by combining real-time scenarios and other traffic participants. This improves the real-time performance of risk prediction while considering the impact of other traffic participants, enhancing robustness in handling complex and sudden traffic scenarios.

[0073] The tracking interaction features are used to characterize the actions of each potential risk target in a preset future time period after being affected by other potential risk targets. For example, it can be whether it encounters the first vehicle, whether it will obstruct the first vehicle's movement, etc. The risk target location features are used to characterize the location information of each potential risk target in the preset future time period. It can be the location information at a certain point in time in the preset future time period, or the movement trajectory in the preset future time period, etc.

[0074] As an example, step S21 includes: forming a tracking feature sequence from the tracking features of each potential risk target; inputting the tracking feature sequence into a multi-head self-attention layer, so that the multi-head self-attention layer captures the dependency between the tracking features of each potential risk target and the tracking features of other potential risk targets in the tracking feature sequence, determines the mutual influence between each potential risk target and other potential risk targets, generates tracking interaction features, and then inputs the tracking interaction features and the map features into a first multi-head cross-attention layer for feature fusion, determines the risk target location features of each potential risk target, and encodes each potential risk target to obtain the risk target encoding features of each potential risk target. Simultaneously, the external environment features and cabin features related to the current scene of the first vehicle can also be input into a second multi-head cross-attention layer for feature fusion to obtain a first fused feature.

[0075] Step S22: Input the first fusion feature, the risk target encoding feature and the risk target location feature of each potential risk target into the first multi-head deformable attention layer to obtain the risk target interaction feature;

[0076] Step S23: By inputting the interactive features of the risk targets into the first multilayer perceptron, at least one high-risk target and the risk information corresponding to each high-risk target are determined from the potential risk targets.

[0077] As an example, steps S22-S23 include: inputting the first fused feature, along with the risk target encoding features and risk target location features of each potential risk target, into the first multi-head deformable attention layer; utilizing a deformable attention mechanism, the first multi-head deformable attention layer focuses on the interaction relationship between the location of the potential risk target and scene information in the first fused feature to obtain risk target interaction features; then, the first multilayer perceptron classifies each potential risk target based on the risk target interaction features, identifies at least one high-risk target from among the potential risk targets, and obtains the risk information corresponding to each high-risk target.

[0078] Optionally, the risk prediction model includes a second multi-head deformable attention layer and a second multilayer perceptron;

[0079] The step of inputting the risk characteristics corresponding to each of the high-risk targets into the risk prediction model to predict the low-risk driving area of ​​the first vehicle in a preset future time period includes:

[0080] Step S31: Input the map features, the risk target encoding features and risk target location features corresponding to each of the high-risk targets into the second multi-head deformable attention layer to obtain the second fusion feature;

[0081] Step S32: Input the second fusion feature, the risk target encoding features corresponding to each of the high-risk targets, and the first fusion feature into the second multilayer perceptron to predict the low-risk driving area of ​​the first vehicle in a preset future time period.

[0082] In this embodiment, it should be noted that the risk prediction model adopts a transformer structure, which includes at least a second multi-head deformable attention layer and a second multi-layer perceptron. The second multi-head deformable attention layer and the second multi-layer perceptron can both be a single layer or a multi-layer structure, which can be determined according to actual needs. This embodiment does not impose any restrictions on this.

[0083] As an example, steps S31-S32 include: inputting the map features, the risk target encoding features and risk target location features corresponding to each of the high-risk targets into the second multi-head deformable attention layer; the second multi-head deformable attention layer focuses on the features of the location of each of the high-risk targets in the map features in a future preset time period, i.e., the second fusion feature; then, after concatenating the second fusion feature with the first fusion feature and the risk target encoding features corresponding to each of the high-risk targets, the multilayer perceptron can decode the low-risk driving area where the probability of each of the high-risk targets reaching is low, or it can decode the movement trajectory of each of the high-risk targets in a preset future time period. The area corresponding to the movement trajectory of each of the high-risk targets in the preset future time period is the high-risk driving area. Alternatively, the driving area can be divided into multiple risk levels according to the probability that the high-risk targets may reach, for example, it can be divided into high-risk driving areas, medium-risk driving areas and low-risk driving areas.

[0084] In one feasible approach, after determining the low-risk driving area, the low-risk driving area can be output, and driving areas of different levels can also be output, and driving areas of different risk levels can be displayed separately.

[0085] Optionally, the risk prediction model further includes a third multilayer perceptron; after the step of inputting the map features and the risk target encoding features and risk target location features corresponding to each of the high-risk targets into the second multi-head deformable attention layer to obtain the second fusion features, the model further includes:

[0086] Step S33: Input the second fusion feature, the risk target encoding feature corresponding to each of the high-risk targets and the first fusion feature into the third multilayer perceptron to predict the movement trajectory of each of the high-risk targets in a preset future time period, and determine the driving area corresponding to the movement trajectory of each of the high-risk targets in the preset future time period as the high-risk driving area of ​​the first vehicle in the preset future time period.

[0087] Step S34: Output the movement trajectory of the low-risk driving area, the high-risk driving area and / or each of the high-risk targets in a preset future time period.

[0088] As an example, steps S33-S34 include: concatenating the second fusion feature, the risk target encoding features corresponding to each of the high-risk targets, and the first fusion feature, and inputting the result into the third multilayer perceptron. The multilayer perceptron can decode the movement trajectories of each of the high-risk targets within a preset future time period, thus defining the areas corresponding to the movement trajectories of each of the high-risk targets within the preset future time period as high-risk driving areas. Furthermore, the low-risk driving areas, the high-risk driving areas, and / or the movement trajectories of each of the high-risk targets within the preset future time period can be output, allowing users to promptly respond to potential driving risks, avoid high-risk targets, and improve driving safety.

[0089] In this embodiment, the tracking features include target movement trend information. By utilizing a multi-head self-attention mechanism to capture the dependencies between multiple potential risk targets, the accuracy of predicting the location information of each potential risk target in a preset future time period can be improved, thereby enhancing the accuracy of identifying high-risk targets. Furthermore, since the risk target location features of each potential risk target represent the impact of other traffic participants on the driving risk of the first vehicle, and the first fused feature represents the impact of the first vehicle's own state and environment on the driving risk of the first vehicle, by fusing the two, driving risk prediction can be performed by combining real-time scenarios and other traffic participants. This improves the real-time performance of risk prediction while considering the impact of other traffic participants, enhancing robustness in dealing with complex and sudden traffic scenarios.

[0090] Example 3

[0091] Furthermore, embodiments of this application also provide a driving risk prediction device, referring to... Figure 4 The driving risk prediction method is applied to a first device, on which a first encoder is deployed; the driving risk prediction device includes:

[0092] The acquisition module 10 is used to acquire vehicle monitoring data and extract risk features from the vehicle monitoring data through an encoder.

[0093] Risk identification module 20 is used to identify at least one high-risk target and the risk information corresponding to each of the high-risk targets by inputting the risk characteristics into the risk identification model;

[0094] The risk prediction module 30 is used to input the risk characteristics corresponding to each of the high-risk targets into the risk prediction model to predict the low-risk driving area of ​​the first vehicle in a preset future time period.

[0095] Optionally, the vehicle monitoring data includes an external environment image sequence, a cabin image sequence, and a vehicle driving data sequence; the risk features include external environment features and cabin features, wherein the cabin features include driver attention distribution features and vehicle driving features; the encoder includes a bird's-eye view encoder, a cabin image encoder, and a vehicle driving time sequence encoder.

[0096] The acquisition module 10 is further configured to:

[0097] The bird's-eye view encoder extracts external environment features from the external environment image sequence, the cockpit image encoder extracts driver attention distribution features from the cockpit image sequence, and the vehicle driving time sequence encoder extracts vehicle driving features from the vehicle driving data sequence.

[0098] Optionally, the encoder further includes a map encoder and a tracking encoder; the risk features further include map features and tracking features;

[0099] The acquisition module 10 is further configured to:

[0100] The map encoder extracts map features from the external environment features, and the tracking encoder performs target detection and multi-target tracking based on the external environment features to obtain tracking features of at least one potential risk target.

[0101] Optionally, the risk identification model includes a multi-head self-attention layer, a first multi-head cross-attention layer, a second multi-head cross-attention layer, a first multi-head deformable attention layer, and a first multilayer perceptron;

[0102] The risk identification module 20 is also used for:

[0103] The tracking features of each potential risk target are input into a multi-head self-attention layer to obtain tracking interaction features. The tracking interaction features and the map features are input into a first multi-head cross-attention layer for feature fusion to determine the risk target encoding features and risk target location features of each potential risk target. The vehicle external environment features and cabin features are input into a second multi-head cross-attention layer for feature fusion to obtain a first fused feature.

[0104] The first fusion feature, along with the risk target encoding features and risk target location features of each potential risk target, are input into the first multi-head deformable attention layer to obtain the risk target interaction features.

[0105] By inputting the interactive features of the risk targets into the first multilayer perceptron, at least one high-risk target and the risk information corresponding to each high-risk target are determined from the potential risk targets.

[0106] Optionally, the risk prediction model includes a second multi-head deformable attention layer and a second multilayer perceptron;

[0107] The risk prediction module 30 is also used for:

[0108] The map features, along with the risk target encoding features and risk target location features corresponding to each of the high-risk targets, are input into the second multi-head deformable attention layer to obtain the second fusion feature.

[0109] The second fusion feature, the risk target encoding features corresponding to each of the high-risk targets, and the first fusion feature are input into the second multilayer perceptron to predict the low-risk driving area of ​​the first vehicle in a preset future time period.

[0110] Optionally, the risk prediction model further includes a third multilayer perceptron; the risk prediction module 30 is further configured to:

[0111] The second fusion feature, the risk target encoding feature corresponding to each of the high-risk targets, and the first fusion feature are input into the third multilayer perceptron to predict the movement trajectory of each of the high-risk targets in a preset future time period, and the driving area corresponding to the movement trajectory of each of the high-risk targets in the preset future time period is determined as the high-risk driving area of ​​the first vehicle in the preset future time period.

[0112] Output the movement trajectories of the low-risk driving area, the high-risk driving area, and / or each of the high-risk targets in a preset future time period.

[0113] Optionally, after the operation of determining at least one high-risk target and the risk information corresponding to each of the high-risk targets by inputting the risk characteristics into the risk identification model, the driving risk prediction device further includes an output module, the output module being used for:

[0114] Based on the risk information corresponding to each of the high-risk targets, corresponding risk warning information is generated and output, wherein the risk information includes risk type and risk probability.

[0115] The driving risk prediction device provided by this invention employs the driving risk prediction method in the above embodiments, solving the technical problem of low accuracy in driving risk prediction in related technologies. Compared with related technologies, the driving risk prediction device provided by this invention has the same advantages as the driving risk prediction method provided in the above embodiments, and other technical features in this driving risk prediction device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0116] Example 4

[0117] Furthermore, embodiments of the present invention provide an electronic device, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the driving risk prediction method in the above embodiments.

[0118] The following is for reference. Figure 5 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as Bluetooth headsets, mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0119] like Figure 5 As shown, an electronic device may include a processing unit (such as a central processing unit, graphics processing unit, etc.) that can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) or a program loaded from a storage device into random access memory (RAM). The RAM also stores various programs and arrays required for the operation of the electronic device. The processing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0120] Typically, the following systems can be connected to the I / O interface: input devices including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices including, for example, magnetic tapes, hard disks, etc.; and communication devices. Communication devices allow electronic devices to communicate wirelessly or wiredly with other devices to exchange arrays. Although electronic devices with various systems are shown in the figures, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems may be implemented alternatively.

[0121] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, it performs the functions defined above in the methods of embodiments of this disclosure.

[0122] The electronic device provided by this invention employs the driving risk prediction method in the above embodiments, solving the technical problem of low accuracy in driving risk prediction in related technologies. Compared with related technologies, the electronic device provided by this invention has the same advantages as the driving risk prediction method provided in the above embodiments, and other technical features in this electronic device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0123] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0124] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0125] Example 5

[0126] Furthermore, this embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon, the computer-readable program instructions being used to execute the driving risk prediction method in the above embodiment.

[0127] The computer-readable storage medium provided in this embodiment of the invention may be, for example, a USB flash drive, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0128] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0129] The aforementioned computer-readable storage medium carries one or more programs that, when executed by an electronic device, cause the electronic device to: acquire vehicle monitoring data; extract risk features from the vehicle monitoring data using an encoder; determine at least one high-risk target and risk information corresponding to each of the high-risk targets by inputting the risk features into a risk identification model; and input the risk features corresponding to each of the high-risk targets into a risk prediction model to predict the low-risk driving area of ​​the first vehicle in a preset future time period.

[0130] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0131] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0132] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0133] The computer-readable storage medium provided by this invention stores computer-readable program instructions for executing the above-described driving risk prediction method, thus solving the technical problem of low accuracy in driving risk prediction in related technologies. Compared with related technologies, the advantages of the computer-readable storage medium provided in this invention are the same as those of the driving risk prediction method provided in the above-described embodiments, and will not be repeated here.

[0134] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent scope of this application.

Claims

1. A method for predicting driving risks, characterized in that, The driving risk prediction method adopts a phased, progressive, multi-objective driving risk prediction approach, applied to the first vehicle, and includes the following steps: Vehicle monitoring data is acquired, and risk features are extracted from the vehicle monitoring data through an encoder. The vehicle monitoring data includes a sequence of images of the external environment of the vehicle. The encoder includes a map encoder, a tracking encoder, and a bird's-eye view encoder. The risk features include map features, tracking features, and external environment features. The driving risk prediction method further includes: extracting external environment features from the external environment image sequence through the bird's-eye view encoder, extracting map features from the external environment features through the map encoder, and performing target detection and multi-target tracking based on the external environment features through the tracking encoder to obtain tracking features of at least one potential risk target; By inputting the risk characteristics into a risk identification model, at least one high-risk target and the risk information corresponding to each of the high-risk targets are identified. The risk information corresponding to each of the high-risk targets is input into the risk prediction model to predict the low-risk driving area of ​​the first vehicle in a preset future time period.

2. The driving risk prediction method as described in claim 1, characterized in that, The vehicle monitoring data includes cockpit image sequences and vehicle driving data sequences; the risk features include cockpit features, which include driver attention distribution features and vehicle driving features; the encoder includes a cockpit image encoder and a vehicle driving time sequence encoder; the driving risk prediction method further includes: The bird's-eye view encoder extracts external environment features from the external environment image sequence, the cockpit image encoder extracts driver attention distribution features from the cockpit image sequence, and the vehicle driving time sequence encoder extracts vehicle driving features from the vehicle driving data sequence.

3. The driving risk prediction method as described in claim 1, characterized in that, The risk identification model includes a multi-head self-attention layer, a first multi-head cross-attention layer, a second multi-head cross-attention layer, a first multi-head deformable attention layer, and a first multilayer perceptron. The step of determining at least one high-risk target and the risk information corresponding to each of the high-risk targets by inputting the risk characteristics into a risk identification model includes: The tracking features of each potential risk target are input into a multi-head self-attention layer to obtain tracking interaction features. The tracking interaction features and the map features are input into a first multi-head cross-attention layer for feature fusion to determine the risk target encoding features and risk target location features of each potential risk target. The vehicle external environment features and cabin features are input into a second multi-head cross-attention layer for feature fusion to obtain a first fused feature. The first fusion feature, along with the risk target encoding features and risk target location features of each potential risk target, are input into the first multi-head deformable attention layer to obtain the risk target interaction features. By inputting the interactive features of the risk targets into the first multilayer perceptron, at least one high-risk target and the risk information corresponding to each high-risk target are determined from the potential risk targets.

4. The driving risk prediction method as described in claim 3, characterized in that, The risk prediction model includes a second multi-head deformable attention layer and a second multi-layer perceptron. The step of inputting the risk information corresponding to each of the high-risk targets into the risk prediction model to predict the low-risk driving area of ​​the first vehicle in a preset future time period includes: The map features, along with the risk target encoding features and risk target location features corresponding to each of the high-risk targets, are input into the second multi-head deformable attention layer to obtain the second fusion feature. The second fusion feature, the risk target encoding features corresponding to each of the high-risk targets, and the first fusion feature are input into the second multilayer perceptron to predict the low-risk driving area of ​​the first vehicle in a preset future time period.

5. The driving risk prediction method as described in claim 4, characterized in that, The risk prediction model further includes a third multilayer perceptron; after the step of inputting the map features and the risk target encoding features and risk target location features corresponding to each of the high-risk targets into the second multi-head deformable attention layer to obtain the second fusion features, it further includes: The second fusion feature, the risk target encoding feature corresponding to each of the high-risk targets, and the first fusion feature are input into the third multilayer perceptron to predict the movement trajectory of each of the high-risk targets in a preset future time period, and the driving area corresponding to the movement trajectory of each of the high-risk targets in the preset future time period is determined as the high-risk driving area of ​​the first vehicle in the preset future time period. Output the movement trajectories of the low-risk driving area, the high-risk driving area, and / or each of the high-risk targets in a preset future time period.

6. The driving risk prediction method as described in claim 1, characterized in that, After the step of inputting the risk characteristics into the risk identification model to determine at least one high-risk target and the risk information corresponding to each of the high-risk targets, the method further includes: Based on the risk information corresponding to each of the high-risk targets, corresponding risk warning information is generated and output, wherein the risk information includes risk type and risk probability.

7. A driving risk prediction device, characterized in that, The driving risk prediction device employs a phased, progressive, multi-objective driving risk prediction method, applied to the first vehicle, and includes: The acquisition module is used to acquire vehicle monitoring data and extract risk features from the vehicle monitoring data through an encoder. The vehicle monitoring data includes a sequence of images of the external environment of the vehicle. The encoder includes a map encoder, a tracking encoder, and a bird's-eye view encoder. The risk features include map features, tracking features, and external environment features. Specifically, the acquisition module is used to extract external environment features from the sequence of images of the external environment of the vehicle through the bird's-eye view encoder. The risk identification module is used to identify at least one high-risk target and the risk information corresponding to each of the high-risk targets by inputting the risk characteristics into the risk identification model. The risk prediction module is used to input the risk information corresponding to each of the high-risk targets into the risk prediction model to predict the low-risk driving area of ​​the first vehicle in a preset future time period. The driving risk prediction device further includes: extracting map features from the external environment features through the map encoder, and performing target detection and multi-target tracking based on the external environment features through the tracking encoder to obtain the tracking features of at least one potential risk target.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by at least one of the processors to enable the at least one processor to perform the steps of the driving risk prediction method according to any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and the computer-readable storage medium stores a program that implements the driving risk prediction method. The program that implements the driving risk prediction method is executed by a processor to implement the steps of the driving risk prediction method as described in any one of claims 1 to 6.

Images