Driving system safety evaluation method, device, system and storage medium

Abstract

The application relates to the technical field of intelligent driving, and discloses a driving system safety evaluation method, device and system and a storage medium, which are used for accurately measuring driving system safety by combining the relative positions of a host vehicle and an obstacle through multi-modal trajectory prediction. The driving system safety evaluation method comprises the following steps: performing multi-modal trajectory prediction on the trajectories of the host vehicle and the obstacle within a first preset time period in the future to obtain a target host vehicle trajectory set and a target obstacle trajectory set; combining one trajectory from the target host vehicle trajectory set and one trajectory from the target obstacle trajectory set to obtain a plurality of pairs of trajectory combinations; determining the initial danger scores of the pairs of trajectory combinations within a second preset time period in the future, and determining the candidate danger scores corresponding to the current frame according to the relative position of the host vehicle and the obstacle in the current frame; and selecting the maximum value from the candidate danger scores corresponding to each frame within the first preset time period to obtain the target danger score corresponding to the first preset time period.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to a driving system safety assessment method, device, system and storage medium. Background Art

[0002] With the development of artificial intelligence technology, intelligent driving technology that applies artificial intelligence technology to vehicles has emerged. The intelligent driving system used in this scenario integrates control technology, environmental perception, and multi-level assisted driving functions, enabling vehicles to travel in a safer, more comfortable and convenient state. Among them, safety assessment is the most critical indicator of the vehicle's driving system, especially the autonomous driving system.

[0003] Evaluating system performance through the safety of each frame during driving helps respond to environmental changes promptly, take risk avoidance measures promptly, and ensure driving safety. The safety assessment of each frame can also be used to further optimize and iterate the system.

[0004] In related technologies, safety at a given frame is typically defined as the probability of a collision with another obstacle within a certain period of time. The most commonly used method is to measure safety using the Time to Collision (TTC). This is calculated by dividing the relative distance between the vehicle and the obstacle in the current frame by the relative speed. A shorter TTC indicates a greater likelihood of a future collision. This method is computationally simple and has clear physical meaning. However, its drawback is that it only considers the scenario where the speed and direction of the vehicle and obstacle remain unchanged in the future. This assumption often fails in real-world scenarios, as the behavior of vehicles and obstacles often changes dynamically. Traditional safety assessment methods cannot accurately assess safety in complex situations. Summary of the Invention

[0005] The present application provides a driving system safety assessment method, device, system and storage medium, which are used to solve the problem that traditional driving systems perform safety assessments based on static driving behaviors and cannot adapt to dynamic driving behavior changes that may occur in actual driving scenarios. Traditional safety assessment methods cannot accurately assess safety in complex situations.

[0006] A first aspect of the present application provides a driving system safety assessment method, comprising: performing multimodal trajectory prediction on the trajectories of a host vehicle and an obstacle within a first preset time period in the future to obtain a target host vehicle trajectory set and a target obstacle trajectory set;

[0007] Selecting one trajectory from each of the target vehicle trajectory set and the target obstacle trajectory set to combine them to obtain multiple pairs of trajectory combinations;

[0008] Determining an initial risk score for each pair of trajectory combinations within a second preset time period in the future, and determining a candidate risk score corresponding to the current frame from the multiple initial risk scores based on the relative positions of the host vehicle and the obstacle in the current frame;

[0009] A maximum value is selected from the candidate risk scores corresponding to each frame within the first preset duration to obtain a target risk score corresponding to the first preset duration.

[0010] A second aspect of the present application provides a driving system safety assessment device, comprising: a prediction module, configured to perform multimodal trajectory prediction on the trajectories of a host vehicle and an obstacle within a first preset time period in the future, to obtain a target host vehicle trajectory set and a target obstacle trajectory set;

[0011] a combining module, configured to select one trajectory from each of the target vehicle trajectory set and the target obstacle trajectory set, and combine them to obtain multiple pairs of trajectory combinations;

[0012] a first evaluation module, configured to determine an initial risk score for each pair of trajectory combinations within a second preset time period in the future, and determine a candidate risk score corresponding to the current frame from a plurality of initial risk scores based on the relative positions of the host vehicle and the obstacle in the current frame;

[0013] The second evaluation module is configured to select a maximum value among the candidate risk scores corresponding to the frames within the first preset duration to obtain a target risk score corresponding to the first preset duration.

[0014] The third aspect of the present application provides a driving system, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the driving system executes the above-mentioned driving system safety assessment method.

[0015] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, enable the computer to execute the above-mentioned driving system safety assessment method.

[0016] The technical solution provided by this application employs a multimodal trajectory prediction and multiple trajectory combination method to address the problem that traditional driving systems, which perform safety assessments based on static driving behavior, are unable to adapt to dynamic driving behavior changes that may occur in actual driving scenarios. By combining multiple trajectory combinations to assess driving safety, the safety of driving systems in complex traffic environments can be more accurately and comprehensively assessed. A target trajectory combination is selected from a set of predicted trajectories of the host vehicle and obstacle based on their relative positions to determine the candidate hazard score corresponding to the current frame. This allows for better adaptation to the behavior patterns of the host vehicle and obstacle, selecting a more appropriate trajectory combination to improve the accuracy of the safety assessment. The maximum value of the candidate hazard scores corresponding to each frame within a first preset duration is selected to obtain a target hazard score corresponding to the first preset duration. This comprehensively considers various possible hazardous situations, helps identify the most likely hazardous scenarios in advance, enhances the system's sensitivity to potential risks, and thus better guides vehicle decision-making and avoids future safety accidents. The candidate hazard scores of multiple frames within the first preset duration can enhance the early warning capability for potential hazards in the future time period, contributing to more accurate accident prevention. Through multimodal trajectory prediction, combined with the relative positions of the main vehicle and obstacles, the safety of the driving system in various scenarios can be accurately measured, providing a reliable decision-making data basis for subsequent decision-making processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a schematic diagram of the first embodiment of the driving system safety assessment method in this application;

[0018] Figure 2 This is a schematic diagram of a second embodiment of the driving system safety assessment method in this application;

[0019] Figure 3 This is a schematic diagram of an embodiment of the distance-score function in this application;

[0020] Figure 4 This is a schematic diagram of an embodiment of the time-weight function in this application;

[0021] Figure 5 This is a schematic diagram of an embodiment of the obstacle relative yaw angle-percentage function in this application;

[0022] Figure 6 This is a schematic diagram of a third embodiment of the driving system safety assessment method in this application;

[0023] Figure 7 This is a schematic diagram of an embodiment of a driving system safety assessment device in this application;

[0024] Figure 8 This is a schematic diagram of another embodiment of the driving system safety assessment device in this application;

[0025] Figure 9 This is a schematic diagram of an embodiment of the driving system in this application. DETAILED DESCRIPTION

[0026] The present application provides a driving system safety assessment method, device, system and storage medium for accurately measuring the driving system safety through multimodal trajectory prediction combined with the relative position of the main vehicle and obstacles.

[0027] The terms "first," "second," "third," "fourth," and the like (if any) in the specification and claims of this application and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 , an embodiment of the driving system safety assessment method in this application includes:

[0029] 101. Perform multimodal trajectory prediction on the trajectories of the host vehicle and the obstacles within a first preset time period in the future to obtain a target host vehicle trajectory set and a target obstacle trajectory set.

[0030] It is understood that the execution subject of this application can be a driving system safety assessment device, or a vehicle terminal, a driving system, an autonomous driving system, or a server, and the specific implementation is not limited here. The embodiment of this application is described by taking the autonomous driving system as the execution subject.

[0031] In this embodiment, multimodal trajectory prediction (MTP) is an advanced prediction method used to provide multiple predictions for a target object to cover all modes of future trajectories, that is, multiple possible motion trajectories in the future, wherein the target object is the main vehicle and obstacles indicated in this application. The core of multimodal trajectory prediction technology is to utilize multiple sensor data, such as laser radar (LIDAR), camera, millimeter wave radar, ultrasonic sensor and other multimodal information for fusion to improve the accuracy and robustness of the prediction. A variety of models such as noise-based MTP framework, anchor condition MTP framework, grid-based MTP framework, etc. can be used for prediction, and this embodiment does not impose specific restrictions.

[0032] It should be further explained that the number of obstacles in this application may be one, two or more, and there is no specific limitation. The obstacles may be intelligent agents such as pedestrians, other vehicle terminals or other road users in the same driving environment as the main vehicle. Multimodal trajectory prediction is usually used to generate a diverse, acceptable and explainable future prediction distribution for each intelligent agent. In special cases, the obstacles of this application may also be other non-intelligent agents such as roadblocks. The position of non-intelligent agents is usually fixed, that is, their trajectory is usually a point. Non-intelligent agents may move due to collisions with other intelligent agents. The multimodal trajectory prediction of this application predicts the trajectories of all obstacles in the current frame when appropriate to evaluate safety in various possible scenarios.

[0033] Specifically, the autonomous driving system outputs the multimodal predicted trajectories of the main vehicle and obstacles within a first preset time period in the future based on multimodal data such as the main vehicle information, obstacle information, map information, traffic light information, etc. in the current frame and the past period, and obtains the target main vehicle trajectory set and the target obstacle trajectory set.

[0034] In this embodiment, the target vehicle trajectory set includes multiple predicted trajectories of the vehicle within a first preset time period in the future, referred to as the vehicle trajectory for ease of distinction. The target obstacle trajectory set includes multiple predicted trajectories of the obstacle within a first preset time period in the future, referred to as the obstacle trajectory for ease of distinction. The output form of the predicted trajectory can be coordinates, direction, velocity, acceleration, etc., at intervals of a third preset time period in the future.

[0035] 102. Select one trajectory from each of the target vehicle trajectory set and the target obstacle trajectory set to combine them, thereby obtaining multiple pairs of trajectory combinations.

[0036] In this embodiment, for an obstacle (obstacle, obs), all the predicted trajectories of the main vehicle in the target main vehicle trajectory set and all the predicted obstacle trajectories corresponding to the obstacle in the target obstacle trajectory set are summed pairwise to obtain multiple pairs of trajectory combinations, each pair of which includes a main vehicle trajectory and an obstacle trajectory.

[0037] Specifically, suppose a frame includes a vehicle and an obstacle. Assume the vehicle has m predicted trajectories and an obstacle obs has n predicted trajectories. Combining these predicted trajectories in pairs yields m*n combinations. For example, suppose the vehicle has three predicted trajectories (V_1, V_2, V_3) and the obstacle obs has two predicted trajectories (O_1, O_2). There are six possible combinations: (V_1, O_1), (V_1, O_2), (V_2, O_1), (V_2, O_2), (V_3, O_1), and (V_3, O_2).

[0038] 103. Determine an initial risk score for each pair of trajectory combinations within a second preset time period in the future, and determine a candidate risk score corresponding to the current frame from the multiple initial risk scores based on the relative positions of the host vehicle and the obstacle in the current frame.

[0039] Specifically, the minimum distance between the host vehicle and the obstacle within the second preset time period in the future is determined based on each pair of trajectory combinations, and the minimum distance is converted into a danger score. All trajectory combinations are traversed to obtain the initial danger score corresponding to each pair of trajectory combinations; the target trajectory combination is selected according to the relative position of the host vehicle and the obstacle in the current frame, and the initial danger score corresponding to the target trajectory combination is determined as the candidate danger score corresponding to the current frame.

[0040] Among them, the danger score is used to indicate the possibility of a collision between the main vehicle and the obstacle within the second preset time period in the future. The higher the score, the greater the risk and the lower the safety of the main vehicle. The distance between the main vehicle and the obstacle is negatively correlated with the danger score, that is, the smaller the distance, the greater the danger score.

[0041] In this embodiment, the relative position of the main vehicle and the obstacle in the current frame is used to indicate the spatial position relationship between the main vehicle and the obstacle in the current frame, for example, the obstacle is located directly in front of the main vehicle, the obstacle is located directly behind the main vehicle, or the obstacle is located between directly behind and directly in front of the main vehicle, etc.

[0042] Understandably, multimodal trajectory prediction for the vehicle and obstacles generates a vast number of possible trajectory combinations, representing their future motion patterns at different points in time. Selecting the most representative trajectory combinations that best reflect actual risk is key to improving safety assessment accuracy.

[0043] This application selects the appropriate trajectory combination based on the relative position of the main vehicle and the obstacle, ensuring that during the risk assessment, the selected trajectory can accurately reflect the dynamic changes of the main vehicle and the obstacle under different positional relationships.

[0044] Taking the example of a scenario where both the main vehicle and the obstacle are intelligent entities, the main vehicle is the first vehicle and the obstacle is the second vehicle. During driving, the relative position of the two is strongly correlated with whether there is an avoidance intention. If the second vehicle is directly behind the first vehicle, since the first vehicle is within its field of view, the second vehicle will usually have a corresponding avoidance intention in response to the changes in the first vehicle's driving behavior. In this case, a trajectory combination with a smaller initial risk score can be selected, that is, the safest trajectory combination can better simulate the main vehicle's avoidance behavior of the obstacle. This method can effectively avoid over-evaluation of risk and reduce false alarms of the system. Similarly, when the second vehicle is directly in front of the first vehicle, the trajectory combination with the largest initial risk score, that is, the least safe trajectory combination, must be selected to ensure that the system provides a reasonable warning of potential dangers. If the relative position relationship is judged based on the main vehicle, the selected trajectory combination is exactly the opposite, and this embodiment does not impose specific restrictions.

[0045] 104. Select a maximum value among the candidate risk scores corresponding to the frames within the first preset duration to obtain a target risk score corresponding to the first preset duration.

[0046] It can be understood that the second preset duration and the first preset duration can be set according to actual conditions. The first preset duration is greater than the second preset duration. Through step 103, the candidate risk scores corresponding to each frame within the first preset duration can be determined to obtain multiple candidate risk scores; the maximum value among the multiple candidate risk scores is selected as the target risk score corresponding to the first preset duration.

[0047] In this embodiment, a multimodal trajectory prediction and multiple trajectory combinations approach address the problem that traditional driving systems, which assess safety based on static driving behavior, are unable to adapt to the dynamic driving behavior changes that may occur in actual driving scenarios. By combining multiple trajectory combinations to assess driving safety, the system can more accurately and comprehensively assess the safety of the driving system in complex traffic environments. A target trajectory combination is selected from the predicted trajectory set based on the relative positions of the host vehicle and the obstacle to determine the candidate hazard score for the current frame. This allows for better adaptation to the behavior patterns of the host vehicle and the obstacle, selecting a more appropriate trajectory combination and improving the accuracy of the safety assessment. The maximum value among the candidate hazard scores corresponding to each frame within a first preset duration is selected to obtain a target hazard score corresponding to the first preset duration. This comprehensively considers various possible hazardous situations, helps identify the most likely hazardous scenarios in advance, enhances the system's sensitivity to potential risks, and thus better guides vehicle decision-making and avoids possible safety accidents in the future. The candidate hazard scores for multiple frames within the first preset duration can enhance the early warning capability for potential hazards in the future time period, facilitating more accurate accident prevention. Through multimodal trajectory prediction, combined with the relative positions of the main vehicle and obstacles, the safety of the driving system in various scenarios can be accurately measured, providing a reliable decision-making data basis for subsequent decision-making processes.

[0048] In practical applications, the behaviors of the host vehicle and obstacles are usually dynamic, so they may have multiple possible trajectories in the future. Especially when there are multiple obstacles or complex road conditions, there are many possible trajectory combinations. How to select the most representative trajectory combination that best reflects the actual risk is the key to improving the accuracy of risk assessment. Figure 2 The second embodiment of the driving system safety assessment method in this application includes:

[0049] 201. Perform multimodal trajectory prediction on the host vehicle and obstacles to obtain an initial host vehicle trajectory set and an initial obstacle trajectory set within a first preset time period in the future.

[0050] Step 201 can be performed with reference to step 101 and will not be described again here.

[0051] 202. Perform trajectory filtering on the initial main vehicle trajectory set and the initial obstacle trajectory set based on preset constraint rules to obtain a target main vehicle trajectory set and a target obstacle trajectory set.

[0052] Specifically, unreasonable trajectories in the initial host vehicle trajectory set and the initial obstacle trajectory set are screened according to preset constraint rules to improve the rationality of the target host vehicle trajectory set and the target obstacle trajectory set. By reducing unreasonable trajectories, the computational complexity of subsequent safety assessment is reduced. While comprehensively evaluating multiple possible trajectories, the efficiency of safety assessment is improved.

[0053] In this embodiment, the constraint rule is used to indicate the rule for filtering the predicted trajectory set, which can be a combination of one or more rules of road constraint rules, dynamic constraint rules, traffic rule constraint rules, etc., without specific limitation.

[0054] It is understandable that the constraint rules for screening the initial main vehicle trajectory set may be the same as or different from the constraint rules for screening the initial obstacle trajectory set. Corresponding constraint rules may be selected according to different types of obstacles. That is, the constraint rules corresponding to different obstacles may be the same or different.

[0055] 203. Select one trajectory from each of the target vehicle trajectory set and the target obstacle trajectory set to combine them, thereby obtaining multiple pairs of trajectory combinations.

[0056] Step 203 can be performed with reference to step 102 and will not be described again here.

[0057] 204. Determine the initial risk score of each pair of trajectory combinations within a second preset time period in the future.

[0058] Specifically, the change in the outline distance between the host vehicle and the obstacle within a preset second preset time period is determined based on each pair of trajectory combinations; and the initial risk scores of the host vehicle and the obstacle within the second preset time period are determined based on the change in the outline distance.

[0059] The above-mentioned determination of the initial risk score of the main vehicle and the obstacle in the second preset time period based on the change in the contour distance includes: determining the minimum contour distance between the main vehicle and the obstacle every third preset time period and converting it into a risk score, obtaining a risk score corresponding to every third preset time period, and the minimum contour distance is negatively correlated with the risk score; determining the maximum value of the product of the risk score corresponding to every third preset time period and the preset time weight as the initial risk score of the main vehicle and the obstacle in the second preset time period.

[0060] For example, the risk score is obtained based on the minimum contour distance and a preset distance-score function. The distance-score function is used to indicate the functional relationship between the distance and the risk score, such as Figure 3 As shown, it can be understood that the function shown in the figure is only one possible example.

[0061] For the combination of the i-th predicted trajectory of the main vehicle and the j-th predicted trajectory of the obstacle, calculate the contour distance dt between the main vehicle and the obstacle every 0.1 seconds in the next 1 second, and convert dt into Figure 3 The relationship shown is converted into the danger level st, that is, the smaller the distance between the main vehicle and the obstacle outline, the more dangerous it is. The danger level reaches 1 when the distance is 0, and the danger level is 0 when the distance is greater than or equal to 1 meter.

[0062] according to Figure 4 The time-weight function calculates the time-related weight wt. The closer to the current moment, the larger the wt is, that is, the closer the prediction is, the more reliable it is. t *w t The maximum value of is taken as the risk s of the combination of the i-th main vehicle prediction trajectory and the j-th obstacle prediction trajectory (i,j) It can be understood that the function described in the figure is only one possible example.

[0063] 205. Determine the percentile based on the relative positions of the host vehicle and the obstacle in the current frame.

[0064] In this embodiment, the percentile is a position correlation factor indicated by the relative position between the obstacle and the main vehicle. The percentile can select the candidate hazard score of the corresponding position as the candidate hazard score from multiple initial hazard scores sorted by size, that is, the target trajectory combination pair can be determined according to the percentile.

[0065] The relative position of this embodiment can be represented by the relative yaw angle between the obstacle and the main vehicle. The percentile can be determined based on the relative yaw angle-percentage function. The obstacle relative yaw angle-percentage function can refer to Figure 5 As shown, it can be understood that the function described in the figure is only a possible example.

[0066] Optionally, when the position of the obstacle relative to the main vehicle is between the rear and the front, a percentile q is calculated based on the relative position and a preset relative yaw angle-percentage function, q∈[0,1]; when the obstacle is directly in front of the main vehicle, that is, yaw is 0, q is 1; when the obstacle is directly behind the main vehicle, that is, yaw is 180 degrees, q is 0.

[0067] It is understandable that when the obstacle is located directly behind the main vehicle, the obstacle will generally make corresponding avoidances to the various trajectories of the main vehicle. For example, the obstacle directly behind the main vehicle will respond to the deceleration trajectory of the main vehicle by slowing down and avoiding it. At this time, the danger to the main vehicle is relatively small, so the minimum value of all sij is selected, that is, when yaw is 180 degrees, the percentile q is 0. When the obstacle is located directly in front of the main vehicle, the obstacle will generally not respond to the various trajectories of the main vehicle. At this time, the danger to the main vehicle is greater than when the obstacle is behind it, so all s(i,j) The maximum value in .

[0068] It should be further explained that the relationship between the above relative yaw angle-percentage function is related to the subject as the comparison benchmark and the subsequent ranking method of multiple initial risk scores. Figure 5 The percentage shown is determined by the criterion of whether the obstacle will react to the various trajectories of the main vehicle. The danger score describes the danger caused by the obstacle in the current frame. In actual situations, it is also possible to set the judgment criteria based on whether the main vehicle will react to the various trajectories of the obstacle and adaptively adjust the corresponding steps.

[0069] 206. Determine, based on the percentile, a candidate hazard score corresponding to the obstacle from a plurality of initial hazard scores sorted by score size.

[0070] Optionally, multiple initial risk scores are sorted from small to large, and the initial risk score at the percentile is taken as the candidate risk score corresponding to the obstacle.

[0071] For example, there are m*n trajectory combinations, and m*n s can be obtained according to step 205. (i,j) , sort them from small to large, and take the qth percentile s (i,j) As the danger score s caused by the obstacle obs in this frame obs , assuming that there are 6 trajectory combinations, the obstacle is directly behind the main vehicle in the current frame, q = 0, and the 6 trajectory combinations are ranked from small to large according to their risk scores: s (V_3,O_1) (V_2,O_2) (V_ 1,O _1) (V_2,O_1) (V_1,O_2) (V_3,O_2) , then s (V_3,O_1) Determine the danger score s caused by the obstacle obs in this frame obs .

[0072] It is understandable that if multiple initial risk scores are sorted from large to small, the initial risk score at the percentile (1-q) can be used as the candidate risk score corresponding to the obstacle.

[0073] 207 : Select a maximum value among the candidate risk scores corresponding to the frames within the first preset duration to obtain a target risk score corresponding to the first preset duration.

[0074] Step 207 can be performed with reference to step 104 and will not be described again here.

[0075] ​​​​​In this embodiment, a multimodal trajectory prediction and multiple trajectory combination method is used to address the problem that traditional driving systems, which perform safety assessments based on static driving behavior, are unable to adapt to dynamic driving behavior changes that may occur in actual driving scenarios. By combining multiple trajectory combinations to assess driving safety, the safety of the driving system in complex traffic environments can be more accurately and comprehensively assessed. Preset constraint rules are used to filter the initial predicted trajectory set, thereby improving the rationality of the target trajectory set and reducing the computational complexity of subsequent safety assessments by reducing unreasonable trajectories. Percentiles are determined based on the relative positions of the host vehicle and obstacles. By selecting candidate hazard scores from multiple initial hazard scores sorted by score, the likelihood of an avoidance intent can be assessed based on the relative positions. The most appropriate target trajectory combination can be selected for safety assessment, effectively avoiding over-assessment of hazard, reducing false alarms and avoiding under-reporting, thereby improving the accuracy of the safety assessment. The maximum value of the candidate hazard scores corresponding to each frame within a first preset duration is selected to obtain the target hazard score corresponding to the first preset duration. This comprehensively considers various possible hazardous situations, helps to identify the most likely hazardous scenarios in advance, enhances the system's sensitivity to potential risks, and thus better guides vehicle decision-making and avoids possible safety accidents in the future. By using candidate hazard scores across multiple frames within a pre-set timeframe, we can enhance early warning capabilities for potential hazards in the future, helping to more accurately prevent accidents. Multimodal trajectory prediction, combined with the relative positions of the vehicle and obstacles, accurately measures the safety of the driving system in various scenarios, providing a reliable data foundation for subsequent decision-making.

[0076] In actual driving, there may be two or more obstacles in each frame. Please refer to Figure 3 A third embodiment of the driving system safety assessment method of the present application is provided:

[0077] 601. Perform multimodal trajectory prediction on the trajectories of the host vehicle and each obstacle within a first preset time period in the future to obtain an initial host vehicle trajectory set corresponding to the host vehicle and an initial obstacle trajectory set corresponding to each obstacle.

[0078] It is understandable that, due to the current presence of at least two obstacles, the motion trajectories of each obstacle and the host vehicle may affect each other, and the motion trajectories of the obstacles may also affect each other. For example, the original motion trajectories of the first obstacle and the host vehicle are independent of each other, but the first obstacle may change its own trajectory under the influence of the second obstacle's motion trajectory, thereby affecting the motion trajectory of the host vehicle. The multimodal trajectory prediction of this embodiment comprehensively considers various current situations to predict various driving trajectories of the host vehicle and the obstacles within a first preset time period.

[0079] Step 601 can be performed with reference to step 101 and will not be described again here.

[0080] 602. Perform trajectory filtering on the initial host vehicle trajectory set and the initial obstacle trajectory set corresponding to each obstacle based on kinematic constraint rules to obtain a target host vehicle trajectory set and a target obstacle trajectory set corresponding to each obstacle.

[0081] Specifically, a first displacement is determined based on the orientation, velocity, and acceleration of any two target trajectory points on each predicted trajectory, and a second displacement is determined based on the coordinate values ​​corresponding to the two target trajectory points; if the difference between the first displacement and the second displacement exceeds a preset threshold, the predicted trajectory is deleted.

[0082] 603. Select one trajectory from each of the target main vehicle trajectory set and the target obstacle trajectory set corresponding to each obstacle, and combine them to obtain multiple pairs of trajectory combinations.

[0083] 604. Determine the initial risk score of each pair of trajectory combinations within a second preset time period in the future.

[0084] 605. Determine a percentile based on the relative positions of the host vehicle and the obstacle in the current frame.

[0085] 606. Among the multiple initial risk scores sorted from small to large, the initial risk score at the percentile is taken as the candidate risk score corresponding to each obstacle in the current frame.

[0086] Steps 603-606 can be executed with reference to steps 203-206 and will not be repeated here.

[0087] 607. Select the maximum value among the candidate hazard scores corresponding to the obstacles in the current frame as the candidate hazard score corresponding to the current frame.

[0088] In this embodiment, multiple s in the current frame are taken obs The maximum value in is used as the dangerousness score of this frame, which can comprehensively consider various possible dangerous situations, select the obstacle with the greatest possibility of danger as the evaluation benchmark, enhance the system's sensitivity to risks, and thus better guide the vehicle's decision-making, avoid possible safety accidents in the future, and help prevent accidents more accurately.

[0089] For example, there are three obstacles in the current frame, and s can be determined according to the above method. obs_2 obs_1 obs_3 , then s obs_3 As the score s of the frame risk of this frame frame .

[0090] ​​608. Select a maximum value among the candidate risk scores corresponding to the frames within the first preset duration to obtain a target risk score corresponding to the first preset duration.

[0091] Take the s of all frames within the first preset duration frame The maximum value of the candidate hazard scores (s) is used as the target hazard score for the first preset duration. This comprehensively considers various possible hazardous situations, helping to identify the most likely dangerous scenarios in advance and enhancing the system's sensitivity to potential risks. This in turn provides better guidance for vehicle decision-making and helps avoid future safety accidents. Using candidate hazard scores from multiple frames within the first preset duration can enhance early warning capabilities for potential hazards in future time periods, helping to more accurately prevent accidents.

[0092] For example, there are 3 frames in the first preset time length, and s can be determined according to the above method. frame_1 frame_3 frame_2 , then s frame_2 The score s is used as the frame risk score of this frame.

[0093] In this embodiment, a multimodal trajectory prediction and multiple trajectory combination method is used to address the problem that traditional driving systems, which perform safety assessments based on static driving behavior, are unable to adapt to dynamic driving behavior changes that may occur in actual driving scenarios. By combining multiple trajectory combinations to assess driving safety, the safety of the driving system in complex traffic environments can be more accurately and comprehensively assessed. Preset constraint rules are used to filter the initial predicted trajectory set, thereby improving the rationality of the target trajectory set and reducing the computational complexity of subsequent safety assessments by reducing unreasonable trajectories. Percentiles are determined based on the relative positions of the host vehicle and obstacles. By selecting candidate hazard scores from multiple initial hazard scores sorted by score, the likelihood of an avoidance intent can be assessed based on the relative positions. The most appropriate target trajectory combination can be selected for safety assessment, effectively avoiding over-assessment of hazard, reducing false alarms and avoiding under-reporting, thereby improving the accuracy of the safety assessment. The maximum value of the candidate hazard scores corresponding to each frame within a first preset duration is selected to obtain the target hazard score corresponding to the first preset duration. This comprehensively considers various possible hazardous situations, helps to identify the most likely hazardous scenarios in advance, enhances the system's sensitivity to potential risks, and thus better guides vehicle decision-making and avoids possible safety accidents in the future. By using candidate hazard scores across multiple frames within a pre-set timeframe, we can enhance early warning capabilities for potential hazards in the future, helping to more accurately prevent accidents. Multimodal trajectory prediction, combined with the relative positions of the vehicle and obstacles, accurately measures the safety of the driving system in various scenarios, providing a reliable data foundation for subsequent decision-making. ​​

[0094] The above describes the driving system safety assessment method in this application. The following describes the driving system safety assessment device in this application. Figure 7 In this application, an embodiment of a driving system safety assessment device includes:

[0095] Prediction module 701, configured to perform multimodal trajectory prediction on the trajectories of the host vehicle and obstacles within a first preset time period in the future to obtain a target host vehicle trajectory set and a target obstacle trajectory set;

[0096] A combining module 702 is configured to select one trajectory from each of the target vehicle trajectory set and the target obstacle trajectory set to combine, thereby obtaining multiple trajectory combinations.

[0097] The first evaluation module 703 is configured to determine an initial risk score for each pair of trajectory combinations within a second predetermined time period in the future, and determine a candidate risk score corresponding to the current frame from among the multiple initial risk scores based on the relative positions of the host vehicle and the obstacle in the current frame;

[0098] The second evaluation module 704 is configured to select a maximum value from the candidate risk scores corresponding to the frames within the first preset duration to obtain a target risk score corresponding to the first preset duration.

[0099] In this embodiment, a multimodal trajectory prediction and multiple trajectory combinations approach address the problem that traditional driving systems, which assess safety based on static driving behavior, are unable to adapt to the dynamic driving behavior changes that may occur in actual driving scenarios. By combining multiple trajectory combinations to assess driving safety, the system can more accurately and comprehensively assess the safety of the driving system in complex traffic environments. A target trajectory combination is selected from the predicted trajectory set based on the relative positions of the host vehicle and the obstacle to determine the candidate hazard score for the current frame. This allows for better adaptation to the behavior patterns of the host vehicle and the obstacle, selecting a more appropriate trajectory combination and improving the accuracy of the safety assessment. The maximum value among the candidate hazard scores corresponding to each frame within a first preset duration is selected to obtain a target hazard score corresponding to the first preset duration. This comprehensively considers various possible hazardous situations, helps identify the most likely hazardous scenarios in advance, enhances the system's sensitivity to potential risks, and thus better guides vehicle decision-making and avoids possible safety accidents in the future. The candidate hazard scores for multiple frames within the first preset duration can enhance the early warning capability for potential hazards in the future time period, facilitating more accurate accident prevention. Through multimodal trajectory prediction, combined with the relative positions of the main vehicle and obstacles, the safety of the driving system in various scenarios can be accurately measured, providing a reliable decision-making data basis for subsequent decision-making processes.

[0100] See also Figure 8Another embodiment of the driving system safety assessment device in the present application includes:

[0101] Prediction module 701, configured to perform multimodal trajectory prediction on the trajectories of the host vehicle and obstacles within a first preset time period in the future to obtain a target host vehicle trajectory set and a target obstacle trajectory set;

[0102] A combining module 702 is configured to select one trajectory from each of the target vehicle trajectory set and the target obstacle trajectory set to combine, thereby obtaining multiple trajectory combinations.

[0103] The first evaluation module 703 is configured to determine an initial risk score for each pair of trajectory combinations within a second predetermined time period in the future, and determine a candidate risk score corresponding to the current frame from among the multiple initial risk scores based on the relative positions of the host vehicle and the obstacle in the current frame;

[0104] The second evaluation module 704 is configured to select a maximum value from the candidate risk scores corresponding to the frames within the first preset duration to obtain a target risk score corresponding to the first preset duration.

[0105] Optionally, the prediction module 701 includes:

[0106] The prediction unit 7011 is used to perform multimodal trajectory prediction on the host vehicle and obstacles to obtain an initial host vehicle trajectory set and an initial obstacle trajectory set within a first preset time period in the future;

[0107] The filtering unit 7012 is configured to filter the initial host vehicle trajectory set and the initial obstacle trajectory set based on a preset constraint rule to obtain a target host vehicle trajectory set and a target obstacle trajectory set.

[0108] Optionally, the filtering unit 7012 is specifically configured to: determine a first displacement according to the orientation, velocity, and acceleration of any two target trajectory points on each predicted trajectory, and determine a second displacement according to the coordinate values ​​corresponding to the two target trajectory points;

[0109] If the difference between the first displacement and the second displacement exceeds a preset threshold, the predicted trajectory is deleted.

[0110] Optionally, the first evaluation module 703 includes:

[0111] The distance determination unit 7031 is used to determine the change in the outline distance between the host vehicle and the obstacle within a preset second preset time period based on each pair of trajectory combinations;

[0112] The first determining unit 7032 is configured to determine initial risk scores of the host vehicle and the obstacle within a second preset time period according to the change in the outline distance.

[0113] a position evaluation unit 7033 for determining a percentile based on the relative position of the host vehicle and the obstacle in the current frame;

[0114] The second determining unit 7034 is configured to determine, according to the percentile, a candidate risk score corresponding to the obstacle from among a plurality of initial risk scores sorted by score size.

[0115] Optionally, the first evaluation unit 7032 is specifically configured to: determine a minimum contour distance between the host vehicle and the obstacle every third preset time period and convert the minimum contour distance into a risk score, thereby obtaining a risk score corresponding to every third preset time period, wherein the minimum contour distance is negatively correlated with the risk score;

[0116] The maximum value of the product of the danger score corresponding to every third preset time length and the preset time weight is determined as the initial danger score of the main vehicle and the obstacle for the second preset time length.

[0117] Optionally, the number of obstacles in the current frame is greater than one; the first evaluation module 703 also includes: a third determination unit 7035, which is used to take the initial hazard score at the percentile among multiple initial hazard scores sorted by score size as the candidate hazard score corresponding to the obstacle.

[0118] The maximum value among the candidate hazard scores corresponding to the obstacles in the current frame is selected as the candidate hazard score corresponding to the current frame.

[0119] In this embodiment, a multimodal trajectory prediction and multiple trajectory combination method is used to address the problem that traditional driving systems, which perform safety assessments based on static driving behavior, are unable to adapt to dynamic driving behavior changes that may occur in actual driving scenarios. By combining multiple trajectory combinations to assess driving safety, the safety of the driving system in complex traffic environments can be more accurately and comprehensively assessed. Preset constraint rules are used to filter the initial predicted trajectory set, thereby improving the rationality of the target trajectory set and reducing the computational complexity of subsequent safety assessments by reducing unreasonable trajectories. Percentiles are determined based on the relative positions of the host vehicle and obstacles. By selecting candidate hazard scores from multiple initial hazard scores sorted by score, the likelihood of an avoidance intent can be assessed based on the relative positions. The most appropriate target trajectory combination can be selected for safety assessment, effectively avoiding over-assessment of hazard, reducing false alarms and avoiding under-reporting, thereby improving the accuracy of the safety assessment. The maximum value of the candidate hazard scores corresponding to each frame within a first preset duration is selected to obtain the target hazard score corresponding to the first preset duration. This comprehensively considers various possible hazardous situations, helps to identify the most likely hazardous scenarios in advance, enhances the system's sensitivity to potential risks, and thus better guides vehicle decision-making and avoids possible safety accidents in the future. By using candidate hazard scores across multiple frames within a pre-set timeframe, we can enhance early warning capabilities for potential hazards in the future, helping to more accurately prevent accidents. Multimodal trajectory prediction, combined with the relative positions of the vehicle and obstacles, accurately measures the safety of the driving system in various scenarios, providing a reliable data foundation for subsequent decision-making.

[0120] above Figure 7 and Figure 8 The driving system safety assessment device in this application is described in detail from the perspective of modular functional entities, and the driving system in this application is described in detail from the perspective of hardware processing.

[0121] See also Figure 9 As shown, the driving system includes a processor 900 and a memory 901 . The memory 901 stores machine executable instructions that can be executed by the processor 900 . The processor 900 executes the machine executable instructions to implement the above-mentioned driving system safety assessment method.

[0122] further, Figure 9 The driving system shown further includes a bus 902 and a communication interface 903 , and the processor 900 , the communication interface 903 and the memory 901 are connected via the bus 902 .

[0123] Among them, the memory 901 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), for example, at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 903 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used. The bus 902 can be an ISA bus, a PCI bus, or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 9 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0124] The processor 900 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the processor 900. The above processor 900 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present disclosure. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present disclosure can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 901 , and the processor 900 reads the information in the memory 901 and completes the method steps of the aforementioned embodiment in combination with its hardware.

[0125] The present application also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions, which, when executed on a computer, cause the computer to execute the steps of the driving system safety assessment method.

[0126] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0127] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0128] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A driving system safety assessment method, characterized in that: The driving system safety assessment method includes: Perform multimodal trajectory prediction on the trajectories of the main vehicle and obstacles within a first preset time period in the future to obtain a target main vehicle trajectory set and a target obstacle trajectory set; Selecting one trajectory from each of the target vehicle trajectory set and the target obstacle trajectory set to combine them to obtain multiple pairs of trajectory combinations; Determining an initial risk score for each pair of trajectory combinations within a second preset time period in the future, and determining a candidate risk score corresponding to the current frame from the multiple initial risk scores based on the relative positions of the host vehicle and the obstacle in the current frame; A maximum value is selected from the candidate risk scores corresponding to each frame within the first preset duration to obtain a target risk score corresponding to the first preset duration.

2. The driving system safety assessment method according to claim 1, characterized in that: Determining a candidate risk score corresponding to the current frame from a plurality of initial risk scores based on the relative positions of the host vehicle and the obstacle in the current frame includes: Determining a percentile based on the relative positions of the host vehicle and the obstacle in the current frame; A candidate risk score corresponding to the obstacle is determined according to the percentile from a plurality of initial risk scores sorted according to the score size.

3. The driving system safety assessment method according to claim 2, characterized in that: The number of obstacles in the current frame is greater than one; After determining the candidate risk score corresponding to the obstacle from the plurality of initial risk scores sorted by score size according to the percentile, the method further includes: The maximum value among the candidate hazard scores corresponding to the obstacles in the current frame is selected as the candidate hazard score corresponding to the current frame.

4. The driving system safety assessment method according to claim 1, characterized in that: The determining of the initial risk score of each pair of trajectory combinations within the preset second preset time period includes: Determining, based on each pair of trajectory combinations, a change in the outline distance between the host vehicle and the obstacle within a preset second preset time period; Initial risk scores of the host vehicle and the obstacle within the second preset time period are determined according to the change in the contour distance.

5. The driving system safety assessment method according to claim 4, characterized in that: The determining, based on the change in the contour distance, the initial risk scores of the host vehicle and the obstacle within the second preset time period includes: Determining a minimum outline distance between the host vehicle and the obstacle every third preset time period and converting the minimum outline distance into a risk score, thereby obtaining a risk score corresponding to every third preset time period, wherein the minimum outline distance is negatively correlated with the risk score; The maximum value of the product of the danger score corresponding to every third preset time length and the preset time weight is determined as the initial danger score of the host vehicle and the obstacle in the second preset time length.

6. The driving system safety assessment method according to any one of claims 1 to 5, characterized in that: The multimodal trajectory prediction of the trajectories of the host vehicle and the obstacles within a first preset time period in the future to obtain a target host vehicle trajectory set and a target obstacle trajectory set includes: Perform multimodal trajectory prediction on the main vehicle and obstacles to obtain an initial main vehicle trajectory set and an initial obstacle trajectory set within a first preset time period in the future; The initial main vehicle trajectory set and the initial obstacle trajectory set are subjected to trajectory filtering based on preset constraint rules to obtain a target main vehicle trajectory set and a target obstacle trajectory set.

7. The driving system safety assessment method according to claim 6, characterized in that: The filtering of the initial main vehicle trajectory set and the initial obstacle trajectory set based on the preset constraint rules to obtain a target main vehicle trajectory set and a target obstacle trajectory set includes: Determine a first displacement based on the orientation, velocity, and acceleration of any two target trajectory points on each predicted trajectory, and determine a second displacement based on the coordinate values ​​corresponding to the two target trajectory points; If the difference between the first displacement and the second displacement exceeds a preset threshold, the predicted trajectory is deleted.

8. A driving system safety assessment device, characterized in that: The driving system safety assessment device comprises: A prediction module is used to perform multimodal trajectory prediction on the trajectories of the host vehicle and obstacles within a first preset time period in the future to obtain a target host vehicle trajectory set and a target obstacle trajectory set; a combining module, configured to select one trajectory from each of the target vehicle trajectory set and the target obstacle trajectory set, and combine them to obtain multiple pairs of trajectory combinations; a first evaluation module, configured to determine an initial risk score for each pair of trajectory combinations within a second preset time period in the future, and determine a candidate risk score corresponding to the current frame from a plurality of initial risk scores based on the relative positions of the host vehicle and the obstacle in the current frame; The second evaluation module is configured to select a maximum value among the candidate risk scores corresponding to the frames within the first preset duration to obtain a target risk score corresponding to the first preset duration.

9. A driving system, characterized in that: The driving system includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor calls the instructions in the memory to enable the driving system to execute the driving system safety assessment method according to any one of claims 1 to 7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instruction is read and executed, the driving system safety assessment method according to any one of claims 1 to 7 is executed.

Images