Driving system safety assessment method, device and system and storage medium

Abstract

The invention relates to the technical field of intelligent driving, and discloses a driving system safety assessment method, device and system and a storage medium, which are used for accurately measuring the safety of a driving system through multi-modal trajectory prediction in combination with the relative position of a main vehicle and an obstacle. The driving system safety assessment method comprises the following steps: performing multi-modal trajectory prediction on trajectories of a main vehicle and an obstacle in a first preset duration in the future to obtain a target main vehicle trajectory set and a target obstacle trajectory set; selecting one track from each of the target main vehicle track set and the target obstacle track set for combination to obtain multiple pairs of track combinations; determining an initial risk score of each pair of track combinations in a second preset time length in the future, and determining a candidate risk score corresponding to the current frame according to the relative position of the main vehicle and the obstacle in the current frame; and selecting a maximum value from the candidate risk scores corresponding to the frames in the first preset time length to obtain a target risk score corresponding to the first preset time length.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to a driving system safety assessment method, device, system and storage medium. Background Art

[0002] With the development of artificial intelligence technology, intelligent driving technology that applies artificial intelligence technology to vehicles has emerged. The intelligent driving system used in this scenario integrates control technology, environmental perception, and multi-level assisted driving functions, enabling vehicles to travel in a safer, more comfortable and convenient state. Among them, safety assessment is the most critical indicator of the vehicle's driving system, especially the autonomous driving system.

[0003] Evaluating system performance through the safety of each frame during driving helps to respond to environmental changes in a timely manner, take risk avoidance measures in a timely manner, and ensure driving safety. The safety assessment of each frame can also be used to further optimize and iterate the system.

[0004] In related technologies, the safety of a certain frame can usually be defined as the possibility of collision with other obstacles in the future. The most commonly used method is to measure safety through the time to collision (TTC), which is the relative distance between the main vehicle and the obstacle in the current frame divided by the relative speed to obtain the collision time. The shorter the collision time, the greater the possibility of collision in the future. This method is simple to calculate and has a clear physical meaning. The disadvantage is that it only considers the situation where the speed and direction of the main vehicle and the obstacle do not change in the future. This assumption is often not true in actual scenarios, because the behavior of vehicles and obstacles is usually dynamic, and traditional safety assessment methods cannot accurately assess safety in complex situations. Summary of the invention

[0005] The present application provides a driving system safety assessment method, device, system and storage medium, which are used to solve the problem that traditional driving systems perform safety assessments based on static driving behaviors and cannot adapt to dynamic driving behavior changes that may occur in actual driving scenarios. Traditional safety assessment methods cannot accurately assess safety in complex situations.

[0006] The first aspect of the present application provides a driving system safety assessment method, comprising: performing multimodal trajectory prediction on the trajectories of a host vehicle and an obstacle within a first preset time period in the future to obtain a target host vehicle trajectory set and a target obstacle trajectory set;

[0007] Select one track from each of the target vehicle track set and the target obstacle track set to combine, to obtain multiple track combinations;

[0008] Determine the initial risk score of each pair of trajectory combinations within a second preset time period in the future, and determine a candidate risk score corresponding to the current frame from a plurality of initial risk scores according to the relative positions of the host vehicle and the obstacle in the current frame;

[0009] A maximum value is selected from the candidate risk scores corresponding to each frame within the first preset duration to obtain a target risk score corresponding to the first preset duration.

[0010] A second aspect of the present application provides a driving system safety assessment device, comprising: a prediction module, configured to perform multimodal trajectory prediction on the trajectories of a host vehicle and an obstacle within a first preset time period in the future, and obtain a target host vehicle trajectory set and a target obstacle trajectory set;

[0011] A combination module, used to select one track from each of the target vehicle track set and the target obstacle track set for combination, to obtain multiple track combinations;

[0012] A first evaluation module is used to determine the initial risk score of each pair of trajectory combinations within a second preset time period in the future, and determine a candidate risk score corresponding to the current frame from a plurality of initial risk scores according to the relative positions of the host vehicle and the obstacle in the current frame;

[0013] The second evaluation module is used to select a maximum value among the candidate risk scores corresponding to each frame within the first preset time length to obtain a target risk score corresponding to the first preset time length.

[0014] The third aspect of the present application provides a driving system, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the driving system executes the above-mentioned driving system safety assessment method.

[0015] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned driving system safety assessment method.

[0016] In the technical solution provided by the present application, a method of multimodal trajectory prediction and multiple pairs of trajectory combinations is adopted to solve the problem that the traditional driving system performs safety assessment based on static driving behavior and cannot adapt to the dynamic driving behavior changes that may occur in actual driving scenarios. By combining multiple trajectory combinations to assess driving safety, the safety of the driving system in complex traffic environments can be more accurately and comprehensively assessed; by selecting a target trajectory combination from the predicted trajectory set of the main vehicle and the obstacle based on the relative position of the main vehicle and the obstacle to determine the candidate hazard score corresponding to the current frame, a more appropriate trajectory combination can be selected to better adapt to the behavior mode of the main vehicle and the obstacle to improve the accuracy of the safety assessment; by selecting the maximum value from the candidate hazard scores corresponding to each frame within the first preset time length, the target hazard score corresponding to the first preset time length can be obtained, which can comprehensively consider various possible dangerous situations, help to identify the most likely dangerous scenes in advance, enhance the system's sensitivity to potential risks, and thus better provide guidance for vehicle decision-making and avoid possible safety accidents in the future. The candidate hazard scores of multiple frames within the first preset time length can enhance the early warning capability of potential hazards in the future time period, which helps to prevent accidents more accurately. Through multimodal trajectory prediction, combined with the relative positions of the main vehicle and obstacles, the safety of the driving system in various scenarios can be accurately measured, providing a reliable decision-making data basis for subsequent decision-making processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a schematic diagram of the first embodiment of the driving system safety assessment method in this application;

[0018] Figure 2 This is a schematic diagram of a second embodiment of the driving system safety assessment method in this application;

[0019] Figure 3 A schematic diagram of an embodiment of the distance-score function in this application;

[0020] Figure 4 This is a schematic diagram of an embodiment of the time-weight function in this application;

[0021] Figure 5 A schematic diagram of an embodiment of the obstacle relative yaw angle-percentage function in this application;

[0022] Figure 6 This is a schematic diagram of a third embodiment of the driving system safety assessment method in this application;

[0023] Figure 7 A schematic diagram of an embodiment of a driving system safety assessment device in the present application;

[0024] Figure 8 A schematic diagram of another embodiment of the driving system safety assessment device in the present application;

[0025] Fig. 9 A schematic diagram of an embodiment of a driving system in the present application. DETAILED DESCRIPTION

[0026] The present application provides a driving system safety assessment method, device, system and storage medium for accurately measuring the driving system safety through multi-modal trajectory prediction combined with the relative positions of the main vehicle and obstacles.

[0027] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 , an embodiment of the driving system safety assessment method in the present application includes:

[0029] 101. Perform multimodal trajectory prediction on the trajectories of the host vehicle and the obstacles within a first preset time period in the future to obtain a target host vehicle trajectory set and a target obstacle trajectory set.

[0030] It is understandable that the execution subject of the present application can be a driving system safety assessment device, or a vehicle terminal, a driving system, an automatic driving system or a server, which is not limited here. The present application embodiment is described by taking the automatic driving system as the execution subject as an example.

[0031] In this embodiment, Multimodal Trajectory Prediction (MTP) is an advanced prediction method for providing multiple predictions for a target object to cover all modes of future trajectories, i.e., multiple possible motion trajectories in the future, wherein the target object is the main vehicle and obstacles indicated in this application. The core of multimodal trajectory prediction technology is to utilize multiple sensor data, such as laser radar (LIDAR), camera, millimeter wave radar, ultrasonic sensor and other multimodal information for fusion to improve the accuracy and robustness of the prediction. A variety of models such as noise-based MTP framework, anchor condition MTP framework, grid-based MTP framework, etc. can be used for prediction, and this embodiment does not make specific restrictions.

[0032] It should be further explained that the number of obstacles in the present application may be one, two or more, without specific limitation. Obstacles may be intelligent agents such as pedestrians, other vehicle terminals or other road users in the same driving environment as the main vehicle. Multimodal trajectory prediction is usually used to generate a diverse, acceptable and explainable future prediction distribution for each intelligent agent. In special cases, the obstacles of the present application may also be other non-intelligent agents such as roadblocks. The position of non-intelligent agents is usually fixed, that is, their trajectory is usually a point. Non-intelligent agents may move due to collisions with other intelligent agents. The multimodal trajectory prediction of the present application predicts the trajectories of all obstacles in the current frame when appropriate to evaluate the safety in various possible scenarios.

[0033] Specifically, the autonomous driving system outputs the multimodal predicted trajectories of the main vehicle and obstacles within a first preset time period in the future based on multimodal data such as the main vehicle information, obstacle information, map information, traffic light information, etc. in the current frame and the past period of time, and obtains the target main vehicle trajectory set and the target obstacle trajectory set.

[0034] In this embodiment, the target main vehicle trajectory set includes multiple predicted trajectories of the main vehicle within the first preset time in the future, which are called main vehicle trajectories for easy distinction; the target obstacle trajectory set includes multiple predicted trajectories of the obstacle within the first preset time in the future, which are called obstacle trajectories for easy distinction. The output form of the predicted trajectory can be the coordinates, direction, speed, acceleration, etc. every third preset time in the future.

[0035] 102. Select one track from each of the target main vehicle track set and the target obstacle track set to combine, and obtain multiple track combinations.

[0036] In this embodiment, for an obstacle (obstacle, obs), all the predicted trajectories of the main vehicle in the target main vehicle trajectory set and all the predicted trajectories of the obstacle in the target obstacle trajectory set corresponding to the obstacle are grouped together to obtain multiple pairs of trajectory combinations, each pair of trajectory combinations includes a main vehicle trajectory and an obstacle trajectory.

[0037] Specifically, suppose a frame includes a main vehicle and an obstacle, and suppose the main vehicle has m predicted trajectories and an obstacle obs has n predicted trajectories. By combining these predicted trajectories in pairs, we can get m*n combinations. For example, suppose the main vehicle has 3 predicted trajectories (V_1, V_2, V_3) and the obstacle obs has 2 predicted trajectories (O_1, O_2), then there are 6 combinations: (V_1, O_1), (V_1, O_2), (V_2, O_1), (V_2, O_2), (V_3, O_1), (V_3, O_2).

[0038] 103. Determine the initial risk score of each pair of trajectory combinations within a second preset time period in the future, and determine a candidate risk score corresponding to the current frame from a plurality of initial risk scores according to the relative positions of the host vehicle and the obstacle in the current frame.

[0039] Specifically, the minimum distance between the main vehicle and the obstacle within the second preset time period in the future is determined based on each pair of trajectory combinations, and the minimum distance is converted into a danger score, and all trajectory combinations are traversed to obtain the initial danger score corresponding to each pair of trajectory combinations; the target trajectory combination is selected according to the relative position of the main vehicle and the obstacle in the current frame, and the initial danger score corresponding to the target trajectory combination is determined as the candidate danger score corresponding to the current frame.

[0040] Among them, the danger score is used to indicate the possibility of a collision between the main vehicle and the obstacle within the second preset time in the future. The higher the score, the greater the risk and the lower the safety of the main vehicle. The distance between the main vehicle and the obstacle is negatively correlated with the danger score, that is, the smaller the distance, the greater the danger score.

[0041] In this embodiment, the relative position of the main vehicle and the obstacle in the current frame is used to indicate the spatial position relationship between the main vehicle and the obstacle in the current frame, for example, the obstacle is located directly in front of the main vehicle, the obstacle is located directly behind the main vehicle, or the obstacle is located between directly behind and directly in front of the main vehicle, etc.

[0042] It is understandable that the multimodal trajectory prediction of the main vehicle and obstacles generates a large number of possible trajectory combinations, which represent the movement patterns of the main vehicle and obstacles at different time points in the future. How to select the most representative trajectory combination that best reflects the actual risk among these trajectories is the key to improving the accuracy of safety assessment.

[0043] The present application selects a suitable trajectory combination by the relative position of the main vehicle and the obstacle, ensuring that during the risk assessment, the selected trajectory can accurately reflect the dynamic changes of the main vehicle and the obstacle under different positional relationships.

[0044] Take the case where both the main vehicle and the obstacle are intelligent entities, the main vehicle is the first vehicle, and the obstacle is the second vehicle. During driving, the relative position of the two is strongly correlated with whether there is an avoidance intention. If the second vehicle is located directly behind the first vehicle, since the first vehicle is within its field of vision, the second vehicle will usually have a corresponding avoidance intention to the driving behavior change of the first vehicle. At this time, a trajectory combination with a smaller initial risk score can be selected, that is, the safest trajectory combination can better simulate the avoidance behavior of the main vehicle to the obstacle. This method can effectively avoid over-evaluation of the risk and reduce the system's false alarms. Similarly, when the second vehicle is located just in front of the first vehicle, it is necessary to select the trajectory combination with the largest initial risk score, that is, select the most unsafe trajectory combination, to ensure that the system reasonably warns of potential dangers. If the relative position relationship is judged based on the main vehicle, the selected trajectory combination is exactly the opposite, and this embodiment does not make specific restrictions.

[0045] 104. Select a maximum value among the candidate risk scores corresponding to the frames within the first preset duration to obtain a target risk score corresponding to the first preset duration.

[0046] It can be understood that the second preset duration and the first preset duration can be set according to actual conditions. The first preset duration is greater than the second preset duration. Through step 103, the candidate hazard scores corresponding to each frame within the first preset duration can be determined to obtain multiple candidate hazard scores; the maximum value among the multiple candidate hazard scores is selected as the target hazard score corresponding to the first preset duration.

[0047] In this embodiment, the method of multimodal trajectory prediction and multiple pairs of trajectory combinations is adopted to solve the problem that the traditional driving system performs safety assessment based on static driving behavior and cannot adapt to the dynamic driving behavior changes that may occur in actual driving scenarios. By combining multiple trajectory combinations to evaluate driving safety, the safety of the driving system in complex traffic environments can be more accurately and comprehensively evaluated; by selecting a target trajectory combination from the predicted trajectory set of the main vehicle and the obstacle according to the relative position of the main vehicle and the obstacle to determine the candidate hazard score corresponding to the current frame, a more appropriate trajectory combination can be selected to better adapt to the behavior mode of the main vehicle and the obstacle to improve the accuracy of safety assessment; by selecting the maximum value from the candidate hazard scores corresponding to each frame within the first preset time length, the target hazard score corresponding to the first preset time length can be obtained, which can comprehensively consider various possible dangerous situations, help to identify the most likely dangerous scenes in advance, enhance the system's sensitivity to potential risks, and thus better provide guidance for vehicle decision-making and avoid possible safety accidents in the future. The candidate hazard scores of multiple frames within the first preset time length can enhance the early warning capability of potential hazards in the future time period, which helps to prevent accidents more accurately. Through multimodal trajectory prediction, combined with the relative positions of the main vehicle and obstacles, the safety of the driving system in various scenarios can be accurately measured, providing a reliable decision-making data basis for subsequent decision-making processes.

[0048] In practical applications, since the behaviors of the main vehicle and obstacles are usually dynamic, they may have multiple possible running trajectories in the future. Especially when there are multiple obstacles or complex road conditions, there are many trajectory combinations. How to select the most representative trajectory combination that best reflects the actual risk is the key to improving the accuracy of risk assessment. Figure 2 , the second embodiment of the driving system safety assessment method in the present application includes:

[0049] 201. Perform multimodal trajectory prediction on the host vehicle and the obstacles to obtain an initial host vehicle trajectory set and an initial obstacle trajectory set within a first preset time period in the future.

[0050] Step 201 can be performed with reference to step 101 and will not be described in detail here.

[0051] 202. Perform trajectory filtering on the initial main vehicle trajectory set and the initial obstacle trajectory set based on preset constraint rules to obtain a target main vehicle trajectory set and a target obstacle trajectory set.

[0052] Specifically, unreasonable trajectories in the initial main vehicle trajectory set and the initial obstacle trajectory set are screened according to preset constraint rules to improve the rationality of the target main vehicle trajectory set and the target obstacle trajectory set. By reducing unreasonable trajectories, the calculation amount of subsequent safety assessment is reduced, and the safety assessment efficiency is improved while taking into account a comprehensive assessment of multiple possible trajectories.

[0053] In this embodiment, the constraint rule is used to indicate the rule for filtering the predicted trajectory set, which can be a combination of one or more rules of road constraint rules, dynamic constraint rules, traffic rule constraint rules, etc., without specific limitation.

[0054] It is understandable that the constraint rules for screening the initial main vehicle trajectory set may be the same as or different from the constraint rules for screening the initial obstacle trajectory set. Corresponding constraint rules may be selected according to different types of obstacles, that is, the constraint rules corresponding to each obstacle may be the same or different.

[0055] 203. Select one track from each of the target main vehicle track set and the target obstacle track set to combine, and obtain multiple track combinations.

[0056] Step 203 may be performed with reference to step 102 and will not be described in detail here.

[0057] 204. Determine the initial risk score of each pair of trajectory combinations within a second preset time period in the future.

[0058] Specifically, the change of the outline distance between the main vehicle and the obstacle within the preset second preset time period is determined according to each pair of trajectory combinations; and the initial danger scores of the main vehicle and the obstacle within the second preset time period are determined according to the change of the outline distance.

[0059] The above-mentioned method of determining the initial danger score of the main vehicle and the obstacle in the second preset time period according to the change of the contour distance includes: determining the minimum contour distance between the main vehicle and the obstacle every third preset time period and converting it into a danger score, obtaining the danger score corresponding to every third preset time period, and the minimum contour distance is negatively correlated with the danger score; determining the maximum value of the product of the danger score corresponding to every third preset time period and the preset time weight as the initial danger score of the main vehicle and the obstacle in the second preset time period.

[0060] Exemplarily, the risk score is obtained according to the minimum contour distance and a preset distance-score function. The distance-score function is used to indicate the functional relationship between the distance and the risk score, such as Figure 3 As shown, it can be understood that the function shown in the figure is only one possible example.

[0061] For the combination of the i-th predicted trajectory of the main vehicle and the j-th predicted trajectory of the obstacle, calculate the contour distance dt between the main vehicle and the obstacle every 0.1 seconds in the next 1 second, and convert dt into Figure 3 The relationship shown is converted into the danger level st, that is, the smaller the distance between the main vehicle and the outline of the obstacle, the more dangerous it is. When the distance is 0, the danger level reaches 1, and when the distance is greater than or equal to 1 meter, the danger level is 0.

[0062] according to Figure 4 The time-weight function calculates the time-related weight wt. The closer to the current moment, the larger the wt. That is, the closer the prediction, the more reliable it is. t *w t The maximum value of is taken as the risk s of the combination of the i-th host vehicle prediction trajectory and the j-th obstacle prediction trajectory (i,j) It can be understood that the function described in the figure is only one possible example.

[0063] 205. Determine the percentile according to the relative position of the host vehicle and the obstacle in the current frame.

[0064] In this embodiment, the percentile is a position correlation factor indicated by the relative position between the obstacle and the main vehicle. The percentile can select the candidate hazard score of the corresponding position as the candidate hazard score from multiple initial hazard scores sorted by size, that is, the target trajectory combination pair can be determined according to the percentile.

[0065] The relative position of this embodiment can be represented by the relative yaw angle between the obstacle and the main vehicle. The percentile can be determined according to the relative yaw angle-percentage function. The obstacle relative yaw angle-percentage function can refer to Figure 5 As shown, it can be understood that the function described in the figure is only a possible example.

[0066] Optionally, when the position of the obstacle relative to the main vehicle is between the rear and the front, a percentile q is calculated according to the relative position and the preset relative yaw angle-percentage function, q∈[0,1]; when the obstacle is in front of the main vehicle, that is, yaw is 0, q is 1; when the obstacle is behind the main vehicle, that is, yaw is 180 degrees, q is 0.

[0067] It is understandable that when the obstacle is located directly behind the main vehicle, the obstacle will generally make corresponding avoidance to the various trajectories of the main vehicle. For example, the obstacle behind the main vehicle will respond to the deceleration trajectory of the main vehicle by decelerating and avoiding. At this time, the danger of the main vehicle is relatively small, so the minimum value of all sij is selected, that is, when yaw is 180 degrees, the percentile q is 0. When the obstacle is located directly in front of the main vehicle, the obstacle generally will not respond to the various trajectories of the main vehicle. At this time, the danger of the main vehicle is greater than when the obstacle is behind, so all sij are selected.(i,j) The maximum value in .

[0068] It should be further explained that the relationship between the above relative yaw angle-percentage function is related to the subject used as the comparison benchmark and the subsequent ranking method of multiple initial risk scores. Figure 5 The percentage shown is determined by the criterion of whether the obstacle will react to the various trajectories of the main vehicle. The danger score is described by the danger caused by the obstacle in the current frame. In actual situations, it can also be set according to whether the main vehicle will react to the various trajectories of the obstacle as the judgment criterion, and the corresponding steps can be adaptively adjusted.

[0069] 206. Determine, according to the percentile, a candidate hazard score corresponding to the obstacle from a plurality of initial hazard scores sorted by score size.

[0070] Optionally, multiple initial hazard scores are sorted from small to large, and the initial hazard score at the percentile is taken as the candidate hazard score corresponding to the obstacle.

[0071] For example, there are m*n trajectory combinations, and m*n s can be obtained according to step 205. (i,j) , sort them from small to large, and take the qth percentile s (i,j) As the danger score s caused by the obstacle obs in this frame obs , assuming that there are 6 trajectory combinations, the obstacle is directly behind the main vehicle in the current frame, q = 0, and the danger scores of the 6 trajectory combinations are ranked from small to large: s (V_3,O_1) (V_2,O_2) (V_ 1,O _1) (V_2,O_1) (V_1,O_2) (V_3,O_2) , then s (V_3,O_1) Determine the danger score s caused by the obstacle obs in this frame obs .

[0072] It can be understood that if multiple initial risk scores are sorted from large to small, the initial risk score at the percentile (1-q) can be used as the candidate risk score corresponding to the obstacle.

[0073] 207. Select a maximum value among the candidate risk scores corresponding to the frames within the first preset duration to obtain a target risk score corresponding to the first preset duration.

[0074] Step 207 can be performed with reference to step 104 and will not be described in detail here.

[0075] ​​​​​In this embodiment, a method of multimodal trajectory prediction and multiple trajectory combinations is adopted to solve the problem that the traditional driving system performs safety assessment based on static driving behavior and cannot adapt to the dynamic driving behavior changes that may occur in actual driving scenarios. By combining multiple trajectory combinations to assess driving safety, the safety of the driving system in a complex traffic environment can be more accurately and comprehensively assessed. The initial predicted trajectory set is screened by preset constraint rules to improve the rationality of the target trajectory set, and the calculation amount of subsequent safety assessment is reduced by reducing unreasonable trajectories. The percentile is determined by the relative position of the main vehicle and the obstacle, and the possibility of the existence of the avoidance intention can be assessed according to the relative position by selecting the candidate hazard score from multiple initial hazard scores sorted by the score size. The most appropriate target trajectory combination can be selected for safety assessment, effectively avoiding over-assessment of hazard, reducing the system's false alarms and avoiding omissions, and improving the accuracy of the safety assessment. The maximum value is selected from the candidate hazard scores corresponding to each frame within the first preset time length to obtain the target hazard score corresponding to the first preset time length, which can comprehensively consider various possible dangerous situations, help to identify the most likely dangerous scenes in advance, enhance the system's sensitivity to potential risks, and better provide guidance for vehicle decision-making to avoid possible safety accidents in the future. The candidate hazard scores of multiple frames within the first preset time period can enhance the early warning capability of potential hazards in the future time period, which helps to prevent accidents more accurately. Through multi-modal trajectory prediction, combined with the relative position of the main vehicle and obstacles, the safety of the driving system in various scenarios can be accurately measured, providing a reliable decision-making data basis for subsequent decision-making processes.

[0076] In actual driving, there may be two or more obstacles in each frame. Please refer to Figure 3 A third embodiment of the driving system safety assessment method of the present application is provided:

[0077] 601. Perform multimodal trajectory prediction on the trajectories of the host vehicle and each obstacle within a first preset time period in the future to obtain an initial host vehicle trajectory set corresponding to the host vehicle and an initial obstacle trajectory set corresponding to each obstacle.

[0078] It can be understood that, since there are at least two obstacles at present, the motion trajectories between each obstacle and the main vehicle may affect each other, and the motion trajectories between the obstacles may also affect each other. For example, the original running trajectories of the first obstacle and the main vehicle are independent of each other, and the first obstacle may also change its own trajectory under the influence of the running trajectory of the second obstacle and affect the running trajectory of the main vehicle. The multimodal trajectory prediction of this embodiment comprehensively considers various current situations to predict various driving trajectories of the main vehicle and the obstacles within the first preset time length.

[0079] Step 601 can be performed with reference to step 101 and will not be described in detail here.

[0080] 602. Perform trajectory filtering on the initial main vehicle trajectory set and the initial obstacle trajectory set corresponding to each obstacle based on the kinematic constraint rule to obtain a target main vehicle trajectory set and a target obstacle trajectory set corresponding to each obstacle.

[0081] Specifically, the first displacement is determined according to the orientation, speed, and acceleration of any two target trajectory points on each predicted trajectory, and the second displacement is determined according to the coordinate values ​​corresponding to the two target trajectory points; if the difference between the first displacement and the second displacement exceeds a preset threshold, the predicted trajectory is deleted.

[0082] 603. Select one trajectory from each of the target main vehicle trajectory set and the target obstacle trajectory set corresponding to each obstacle, and combine them to obtain multiple pairs of trajectory combinations.

[0083] 604. Determine the initial risk score of each pair of trajectory combinations within a second preset time period in the future.

[0084] 605. Determine the percentile according to the relative positions of the host vehicle and the obstacle in the current frame.

[0085] 606. Among the multiple initial danger scores sorted from small to large, the initial danger score at the percentile is taken as the candidate danger score corresponding to each obstacle in the current frame.

[0086] Steps 603-606 may be performed with reference to steps 203-206 and will not be described in detail here.

[0087] 607. Select a maximum value among the candidate hazard scores corresponding to the obstacles in the current frame as the candidate hazard score corresponding to the current frame.

[0088] In this embodiment, multiple s in the current frame are taken obs The maximum value of the value is used as the dangerousness score of this frame. It can comprehensively consider various possible dangerous situations, select the obstacle with the greatest possibility of danger as the evaluation benchmark, enhance the system's sensitivity to risks, and thus better guide the vehicle's decision-making, avoid possible safety accidents in the future, and help prevent accidents more accurately.

[0089] For example, there are three obstacles in the current frame, and s can be determined according to the above method. obs_2 obs_1 obs_3 , then s obs_3 As the score s of the frame risk of this frame frame .

[0090] ​​608. Select a maximum value among the candidate risk scores corresponding to the frames within the first preset duration to obtain a target risk score corresponding to the first preset duration.

[0091] Take the s of all frames within the first preset duration frame The maximum value of is used as the target hazard score s corresponding to the first preset duration, which can comprehensively consider various possible dangerous situations, help to identify the most likely dangerous scenes in advance, enhance the system's sensitivity to potential risks, and thus better provide guidance for vehicle decision-making and avoid possible safety accidents in the future. The candidate hazard scores of multiple frames within the first preset duration can enhance the early warning capability of potential dangers in the future time period, which helps to prevent accidents more accurately.

[0092] For example, there are 3 frames in the first preset time length, and s can be determined according to the above method. frame_1 frame_3 frame_2 , then s frame_2 The score s is used as the frame risk score of this frame.

[0093] In this embodiment, a method of multimodal trajectory prediction and multiple trajectory combinations is adopted to solve the problem that the traditional driving system performs safety assessment based on static driving behavior and cannot adapt to the dynamic driving behavior changes that may occur in actual driving scenarios. By combining multiple trajectory combinations to assess driving safety, the safety of the driving system in a complex traffic environment can be more accurately and comprehensively assessed. The initial predicted trajectory set is screened by preset constraint rules to improve the rationality of the target trajectory set, and the calculation amount of subsequent safety assessment is reduced by reducing unreasonable trajectories. The percentile is determined by the relative position of the main vehicle and the obstacle, and the possibility of the existence of the avoidance intention can be assessed according to the relative position by selecting the candidate hazard score from multiple initial hazard scores sorted by the score size. The most appropriate target trajectory combination can be selected for safety assessment, effectively avoiding over-assessment of hazard, reducing the system's false alarms and avoiding omissions, and improving the accuracy of the safety assessment. The maximum value is selected from the candidate hazard scores corresponding to each frame within the first preset time length to obtain the target hazard score corresponding to the first preset time length, which can comprehensively consider various possible dangerous situations, help to identify the most likely dangerous scenes in advance, enhance the system's sensitivity to potential risks, and better provide guidance for vehicle decision-making to avoid possible safety accidents in the future. The candidate hazard scores of multiple frames within the first preset time period can enhance the early warning capability of potential hazards in the future time period, which helps to prevent accidents more accurately. Through multi-modal trajectory prediction, combined with the relative position of the main vehicle and obstacles, the safety of the driving system in various scenarios can be accurately measured, providing a reliable decision-making data basis for subsequent decision-making processes. ​​

[0094] The above describes the driving system safety assessment method in this application. The following describes the driving system safety assessment device in this application. Figure 7 , an embodiment of the driving system safety assessment device in the present application includes:

[0095] Prediction module 701, used to perform multi-modal trajectory prediction on the trajectories of the host vehicle and the obstacles within a first preset time period in the future, and obtain a target host vehicle trajectory set and a target obstacle trajectory set;

[0096] A combination module 702 is used to select one track from each of the target vehicle track set and the target obstacle track set to combine, so as to obtain multiple track combinations;

[0097] The first evaluation module 703 is used to determine the initial risk score of each pair of trajectory combinations within the second preset time period in the future, and determine the candidate risk score corresponding to the current frame from the multiple initial risk scores according to the relative positions of the host vehicle and the obstacle in the current frame;

[0098] The second evaluation module 704 is used to select the maximum value among the candidate risk scores corresponding to the frames within the first preset duration to obtain the target risk score corresponding to the first preset duration.

[0099] In this embodiment, the method of multimodal trajectory prediction and multiple pairs of trajectory combinations is adopted to solve the problem that the traditional driving system performs safety assessment based on static driving behavior and cannot adapt to the dynamic driving behavior changes that may occur in actual driving scenarios. By combining multiple trajectory combinations to evaluate driving safety, the safety of the driving system in complex traffic environments can be more accurately and comprehensively evaluated; by selecting a target trajectory combination from the predicted trajectory set of the main vehicle and the obstacle according to the relative position of the main vehicle and the obstacle to determine the candidate hazard score corresponding to the current frame, a more appropriate trajectory combination can be selected to better adapt to the behavior mode of the main vehicle and the obstacle to improve the accuracy of safety assessment; by selecting the maximum value from the candidate hazard scores corresponding to each frame within the first preset time length, the target hazard score corresponding to the first preset time length can be obtained, which can comprehensively consider various possible dangerous situations, help to identify the most likely dangerous scenes in advance, enhance the system's sensitivity to potential risks, and thus better provide guidance for vehicle decision-making and avoid possible safety accidents in the future. The candidate hazard scores of multiple frames within the first preset time length can enhance the early warning capability of potential hazards in the future time period, which helps to prevent accidents more accurately. Through multimodal trajectory prediction, combined with the relative positions of the main vehicle and obstacles, the safety of the driving system in various scenarios can be accurately measured, providing a reliable decision-making data basis for subsequent decision-making processes.

[0100] See also Figure 8Another embodiment of the driving system safety assessment device in the present application includes:

[0101] Prediction module 701, used to perform multi-modal trajectory prediction on the trajectories of the host vehicle and the obstacles within a first preset time period in the future, and obtain a target host vehicle trajectory set and a target obstacle trajectory set;

[0102] A combination module 702 is used to select one track from each of the target vehicle track set and the target obstacle track set to combine, so as to obtain multiple track combinations;

[0103] The first evaluation module 703 is used to determine the initial risk score of each pair of trajectory combinations within the second preset time period in the future, and determine the candidate risk score corresponding to the current frame from the multiple initial risk scores according to the relative positions of the host vehicle and the obstacle in the current frame;

[0104] The second evaluation module 704 is used to select the maximum value among the candidate risk scores corresponding to the frames within the first preset duration to obtain the target risk score corresponding to the first preset duration.

[0105] Optionally, the prediction module 701 includes:

[0106] The prediction unit 7011 is used to perform multimodal trajectory prediction on the host vehicle and the obstacle to obtain an initial host vehicle trajectory set and an initial obstacle trajectory set within a first preset time period in the future;

[0107] The filtering unit 7012 is used to filter the initial host vehicle trajectory set and the initial obstacle trajectory set based on a preset constraint rule to obtain a target host vehicle trajectory set and a target obstacle trajectory set.

[0108] Optionally, the filtering unit 7012 is specifically used to: determine the first displacement according to the orientation, speed, and acceleration of any two target trajectory points on each predicted trajectory, and determine the second displacement according to the coordinate values ​​corresponding to the two target trajectory points;

[0109] If the difference between the first displacement and the second displacement exceeds a preset threshold, the predicted trajectory is deleted.

[0110] Optionally, the first evaluation module 703 includes:

[0111] The distance determination unit 7031 is used to determine the change of the outline distance between the host vehicle and the obstacle within a preset second preset time period according to each pair of trajectory combinations;

[0112] The first determination unit 7032 is used to determine the initial risk scores of the host vehicle and the obstacle within a second preset time period according to the change of the contour distance.

[0113] A position evaluation unit 7033, for determining a percentile according to the relative position of the host vehicle and the obstacle in the current frame;

[0114] The second determining unit 7034 is used to determine, according to the percentile, a candidate hazard score corresponding to the obstacle from a plurality of initial hazard scores sorted by score size.

[0115] Optionally, the first evaluation unit 7032 is specifically used to: determine the minimum contour distance between the main vehicle and the obstacle every third preset time and convert it into a danger score, to obtain a danger score corresponding to every third preset time, and the minimum contour distance is negatively correlated with the danger score;

[0116] The maximum value of the product of the danger score corresponding to every third preset time length and the preset time weight is determined as the initial danger score of the main vehicle and the obstacle for the second preset time length.

[0117] Optionally, the number of obstacles in the current frame is greater than one; the first evaluation module 703 also includes: a third determination unit 7035, which is used to take the initial hazard score at the percentile among multiple initial hazard scores sorted by score size as the candidate hazard score corresponding to the obstacle.

[0118] The maximum value among the candidate hazard scores corresponding to the obstacles in the current frame is selected as the candidate hazard score corresponding to the current frame.

[0119] In this embodiment, a method of multimodal trajectory prediction and multiple trajectory combinations is adopted to solve the problem that the traditional driving system performs safety assessment based on static driving behavior and cannot adapt to the dynamic driving behavior changes that may occur in actual driving scenarios. By combining multiple trajectory combinations to assess driving safety, the safety of the driving system in a complex traffic environment can be more accurately and comprehensively assessed. The initial predicted trajectory set is screened by preset constraint rules to improve the rationality of the target trajectory set, and the calculation amount of subsequent safety assessment is reduced by reducing unreasonable trajectories. The percentile is determined by the relative position of the main vehicle and the obstacle, and the possibility of the existence of the avoidance intention can be assessed according to the relative position by selecting the candidate hazard score from multiple initial hazard scores sorted by the score size. The most appropriate target trajectory combination can be selected for safety assessment, effectively avoiding over-assessment of hazard, reducing the system's false alarms and avoiding omissions, and improving the accuracy of the safety assessment. The maximum value is selected from the candidate hazard scores corresponding to each frame within the first preset time length to obtain the target hazard score corresponding to the first preset time length, which can comprehensively consider various possible dangerous situations, help to identify the most likely dangerous scenes in advance, enhance the system's sensitivity to potential risks, and better provide guidance for vehicle decision-making to avoid possible safety accidents in the future. The candidate hazard scores of multiple frames within the first preset time period can enhance the early warning capability of potential hazards in the future time period, which helps to prevent accidents more accurately. Through multi-modal trajectory prediction, combined with the relative position of the main vehicle and obstacles, the safety of the driving system in various scenarios can be accurately measured, providing a reliable decision-making data basis for subsequent decision-making processes.

[0120] above Figure 7 and Figure 8 The driving system safety assessment device in the present application is described in detail from the perspective of modular functional entities, and the driving system in the present application is described in detail from the perspective of hardware processing.

[0121] See also Fig. 9 As shown, the driving system includes a processor 900 and a memory 901 , wherein the memory 901 stores machine executable instructions that can be executed by the processor 900 , and the processor 900 executes the machine executable instructions to implement the above-mentioned driving system safety assessment method.

[0122] further, Fig. 9 The driving system shown also includes a bus 902 and a communication interface 903 , and the processor 900 , the communication interface 903 and the memory 901 are connected via the bus 902 .

[0123] The memory 901 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), for example, at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 903 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 902 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0124] The processor 900 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 900. The above processor 900 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present disclosure can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present disclosure can be directly embodied as a hardware decoding processor to be executed, or a combination of hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 901 , and the processor 900 reads the information in the memory 901 and completes the method steps of the above-mentioned embodiment in combination with its hardware.

[0125] The present application also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the driving system safety assessment method.

[0126] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0127] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0128] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A driving system safety assessment method, characterized in that: The driving system safety assessment method comprises: Perform multimodal trajectory prediction on the trajectories of the main vehicle and the obstacles within a first preset time period in the future to obtain a target main vehicle trajectory set and a target obstacle trajectory set; Selecting one track from each of the target vehicle track set and the target obstacle track set to combine them, to obtain multiple track combinations; Determine the initial risk score of each pair of trajectory combinations within a second preset time period in the future, and determine a candidate risk score corresponding to the current frame from a plurality of initial risk scores according to the relative positions of the host vehicle and the obstacle in the current frame; A maximum value is selected from the candidate risk scores corresponding to each frame within the first preset duration to obtain a target risk score corresponding to the first preset duration.

2. The driving system safety assessment method according to claim 1, characterized in that: The step of determining a candidate risk score corresponding to the current frame from a plurality of initial risk scores according to the relative positions of the host vehicle and the obstacle in the current frame includes: Determining a percentile according to the relative positions of the host vehicle and the obstacle in the current frame; A candidate hazard score corresponding to the obstacle is determined according to the percentile from a plurality of initial hazard scores sorted according to the score size.

3. The driving system safety assessment method according to claim 2, characterized in that: The number of obstacles in the current frame is greater than one; After determining the candidate hazard score corresponding to the obstacle from a plurality of initial hazard scores sorted according to the score size according to the percentile, the method further includes: The maximum value among the candidate hazard scores corresponding to the obstacles in the current frame is selected as the candidate hazard score corresponding to the current frame.

4. The driving system safety assessment method according to claim 1, characterized in that: The step of determining the initial risk score of each pair of trajectory combinations within a preset second preset time period includes: Determine, according to each pair of trajectory combinations, a change in the contour distance between the host vehicle and the obstacle within a preset second preset time period; Initial risk scores of the host vehicle and the obstacle within the second preset time period are determined according to the change in the contour distance.

5. The driving system safety assessment method according to claim 4, characterized in that: The determining, according to the change in the contour distance, the initial risk scores of the host vehicle and the obstacle within the second preset time period includes: Determine the minimum contour distance between the main vehicle and the obstacle every third preset time and convert it into a danger score, so as to obtain a danger score corresponding to every third preset time, wherein the minimum contour distance is negatively correlated with the danger score; The maximum value of the product of the danger score corresponding to every third preset time length and the preset time weight is determined as the initial danger score of the host vehicle and the obstacle in the second preset time length.

6. The driving system safety assessment method according to any one of claims 1 to 5, characterized in that: The multimodal trajectory prediction of the trajectories of the main vehicle and the obstacles within a first preset time period in the future is performed to obtain a target main vehicle trajectory set and a target obstacle trajectory set, including: Perform multimodal trajectory prediction on the main vehicle and the obstacle to obtain an initial main vehicle trajectory set and an initial obstacle trajectory set within a first preset time period in the future; The initial main vehicle trajectory set and the initial obstacle trajectory set are subjected to trajectory filtering based on preset constraint rules to obtain a target main vehicle trajectory set and a target obstacle trajectory set.

7. The driving system safety assessment method according to claim 6, characterized in that: The filtering of the initial main vehicle trajectory set and the initial obstacle trajectory set based on the preset constraint rules to obtain a target main vehicle trajectory set and a target obstacle trajectory set includes: Determine a first displacement according to the orientation, velocity, and acceleration of any two target trajectory points on each predicted trajectory, and determine a second displacement according to the coordinate values ​​corresponding to the two target trajectory points; If the difference between the first displacement and the second displacement exceeds a preset threshold, the predicted trajectory is deleted.

8. A driving system safety assessment device, characterized in that: The driving system safety assessment device comprises: A prediction module is used to perform multimodal trajectory prediction on the trajectories of the main vehicle and the obstacles within a first preset time period in the future to obtain a target main vehicle trajectory set and a target obstacle trajectory set; A combination module, used to select one track from each of the target vehicle track set and the target obstacle track set for combination, to obtain multiple track combinations; A first evaluation module is used to determine the initial risk score of each pair of trajectory combinations within a second preset time period in the future, and determine a candidate risk score corresponding to the current frame from a plurality of initial risk scores according to the relative positions of the host vehicle and the obstacle in the current frame; The second evaluation module is used to select a maximum value among the candidate risk scores corresponding to each frame within the first preset time length to obtain a target risk score corresponding to the first preset time length.

9. A driving system, characterized in that: The driving system comprises: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the driving system to execute the driving system safety assessment method according to any one of claims 1 to 7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instruction is read and executed, the driving system safety assessment method as described in any one of claims 1-7 is executed.

Images