Automatic driving system abnormal behavior detection method, automatic driving system degradation method and related equipment

Abstract

The invention discloses an automatic driving system abnormal behavior detection method, an automatic driving system degradation method and related equipment, and the automatic driving system abnormal behavior detection method comprises the steps: detecting scene information, and outputting a scene classification label based on the scene information, the scene information comprises road information, traffic participant information, infrastructure information and / or temporary facility information, and the scene classification label is related to the own vehicle speed, the road information and the traffic participant information; a scene association threshold is determined based on the scene classification label, and the scene association threshold is in negative correlation with the severity level of a scene corresponding to the scene classification label after an accident occurs; and determining whether the vehicle has an abnormal behavior based on the scene information and the scene association threshold.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of automatic driving, in particular to an automatic driving system abnormal behavior detection method, an automatic driving system degradation method, and related equipment. BACKGROUND

[0002] With the rapid development of driving automation technology, the stability and safety of the driving automation system mounted on real vehicles have been significantly improved. However, the real vehicle operating environment is extremely complex and full of uncertainties. When the driving automation system runs in a long distance and complex environment, various faults and performance limitations will still be exposed, covering functional safety issues and expected functional safety issues. In order to effectively ensure the safety of the system when encountering faults and performance bottlenecks, it is particularly necessary to develop real-time detection and response technology applicable to real vehicles.

[0003] The current detection mechanism is mostly designed to detect a single failure or ego vehicle behavior, and to degrade after detecting system failure or abnormality. This method has the following disadvantages: excessive degradation: the existing mechanism does not fully consider the fault tolerance capability of the system redundant architecture. When a single sensor or computing unit fails, even if the system can maintain safe operation through multi-source heterogeneous fusion (such as visual / laser radar data complementation) or functional safety mechanism (such as separation design of monitoring layer and execution layer), unnecessary degradation process will still be triggered. Taking a multi-sensor perception system as an example, 200ms data loss of millimeter wave radar caused by bad weather can be compensated by laser radar point cloud data to realize environmental perception continuity. At this time, forced degradation will significantly reduce the system availability metric, which violates the "minimum intervention" design principle of the automatic driving system. Missing detection of unexpected behavior: the existing detection of unexpected behavior is usually to detect whether the control parameter of the system output exceeds the limit value. In order to avoid false positives, the limit value is usually large, which cannot identify unexpected behavior in many safety-critical scenarios, and thus cannot avoid harm. For example, in order to avoid unexpected acceleration behavior, the acceleration is usually detected to exceed the limit value, which is 4m / s^2. However, in a following vehicle scenario, a sudden 3m / s^2 acceleration is enough to hit the front vehicle. Insufficient time for driver takeover: during mass production, the detection threshold needs to be converged for a long time before it can be put into operation. However, the threshold after convergence is generally large enough to prevent false positives. Even if it is put into operation, it cannot fully cover the safety problems caused by failures in high-risk scenarios. For example, if the trajectory topic is lost for more than 2s, a fault code is reported. The threshold is fixed, and the time left for the driver to take over in high-risk scenarios such as high-speed scenarios and curved road scenarios is very small. After the fault is reported, an accident may have already occurred. SUMMARY

[0004] Embodiments of the present application provide an automatic driving system abnormal behavior detection method and an automatic driving system degradation method and related equipment, which are used to at least solve one of the above technical problems.

[0005] In a first aspect, embodiments of the present application provide an automatic driving system abnormal behavior detection method, which includes: detecting scene information, and outputting a scene classification label based on the scene information, wherein the scene information includes road information, traffic participant information, infrastructure information, and / or temporary facility information, and the scene classification label is related to the speed of the ego vehicle, road information, and traffic participant information; determining a scene correlation threshold based on the scene classification label, wherein the scene correlation threshold is negatively correlated with the severity level of the scene after an accident occurs; and determining whether the ego vehicle has abnormal behavior based on the scene information and the scene correlation threshold.

[0006] In a second aspect, embodiments of the present application provide an automatic driving system degradation method, which includes: detecting abnormal behavior of the ego vehicle and / or detecting a fault of the ego vehicle, and outputting an abnormal duration, wherein the abnormal duration includes the duration of the abnormal behavior and / or the duration of the fault of the ego vehicle; predicting a collision time of the ego vehicle and a target object based on at least the scene information; determining a driver takeover time based on the collision time and the abnormal duration; and determining whether to trigger degradation based on the driver takeover time and the abnormal duration.

[0007] In a third aspect, embodiments of the present application provide an electronic device, which includes at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any one of the above automatic driving system abnormal behavior detection methods and / or degradation methods.

[0008] In a fourth aspect, embodiments of the present application provide a storage medium, which stores one or more programs including execution instructions, and the execution instructions can be read and executed by an electronic device (including but not limited to a computer, a server, or a network device, etc.) to perform any one of the above automatic driving system abnormal behavior detection methods and / or automatic driving system degradation methods.

[0009] In a fifth aspect, embodiments of the present application further provide a computer program product, which includes a computer program stored on a storage medium, and the computer program includes program instructions, which, when executed by a computer, enable the computer to perform any one of the above abnormal behavior detection methods and / or system degradation methods.

[0010] In a sixth aspect, the embodiments of the present application further provide a movable platform comprising the electronic device of the third aspect.

[0011] The method of the present application can determine a scene correlation threshold based on a scene classification label, and then determine whether the ego vehicle has abnormal behavior based on the scene information and the scene correlation threshold, so that the safety boundary can be more accurately covered according to the scene, and the accident risk can be reduced. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0013] Figure 1 A flowchart of an automatic driving system abnormal behavior detection method provided by an embodiment of the present application; Figure 2 A flowchart of an automatic driving system degradation method provided by an embodiment of the present application; Figure 3 An abnormality detection and degradation flowchart of a specific example of the automatic driving system abnormal behavior detection method and the automatic driving system degradation method provided by an embodiment of the present application; Figure 4 A structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0014] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0015] The inventors found in the process of implementing the present application that, whether it is a functional safety standard or an expected functional safety standard, it is necessary to identify and evaluate hazard events in combination with scenarios to develop safety measures. Among them, for functional safety problems, it is necessary to identify the failure that leads to the occurrence of hazards through hazard analysis, and to form a hazard event in combination with the corresponding scenario. After determining the severity, controllability and exposure rate of the hazard event in the subsequent safety evaluation, corresponding safety measures are developed. For example: loss of track

failure

hazard

curve non-deceleration collision with guardrail

severity, controllability, exposure rate

trigger condition

hazard

curve acceleration collision with guardrail

severity, controllability

[0016] Please refer to Figure 1 , which shows a flowchart of a method for detecting abnormal behavior of an automatic driving system according to an embodiment of the present application. The method of the present application can be applied to a movable platform equipped with a driving automation system, for example, a vehicle, a movable robot, an aircraft, etc., which is not limited in the present application. The execution subject of the method of the present application can be an automatic driving system abnormal behavior detection device, which can be a software or a combination of software and hardware device (such as a controller in the movable platform, etc.).

[0017] As shown in Figure 1 , in step 101, scene information is detected, and a scene classification label is output based on the scene information; in step 102, a scene correlation threshold is determined based on the scene classification label; in step 103, it is determined whether the ego vehicle has an abnormal behavior based on the scene information and the scene correlation threshold.

[0018] In this embodiment, for step 101, the autonomous driving system abnormal behavior detection device detects scene information, and outputs a scene classification label based on the scene information, wherein the scene information includes road information, traffic participant information, infrastructure information, and / or temporary facility information, and the scene classification label is related to the speed of the ego vehicle, road information, and traffic participant information. For example, the scene information can include ego vehicle state information and perception result information of the perception system, wherein the ego vehicle state information can include the position and pose of the ego vehicle, motion state, dynamics and control state, vehicle configuration and mode, and environment perception related state, etc. The perception system is a subsystem of existing autonomous driving, which is used to perceive the environment and output the perception result information to the downstream for decision planning. The perception result information can include the dynamic and static target object category information around the ego vehicle, kinematics information (such as pose and trajectory), etc. The road information can include road curvature (real-time calculation), slope gradient (determined based on IMU and map data), etc. The infrastructure can include speed limit signs (identified by vision), guardrail types (matched by map), safety island positions, etc. The temporary facilities can include cones, water barriers, construction signs (which can be detected by vision and / or laser radar), etc. The traffic participants can include vehicles (including speed, position, heading angle) in the ego lane and adjacent lane within the perception range (such as 200m) in front, weak traffic participants (such as real-time motion trajectory of pedestrians and two-wheeled vehicles), etc. Further, the scene classification label can be the 12 scene classification labels defined in this application, or other scene classification labels, which are not limited herein. The 12 scene classification labels of this application include: high-speed scene, medium-speed scene, forward dangerous scene_vehicles, forward dangerous scene_obstacles, forward dangerous scene_weak traffic participants, lateral dangerous scene_vehicles, lateral dangerous scene_weak traffic participants, curve scene, slope scene, rearward oncoming vehicle scene, adjacent lane with conflict vehicle scene, and other regular scenes.

[0019] The autonomous driving system abnormal behavior detection device can output the “high-speed scene” or “medium-speed scene” label according to the speed of the ego vehicle, and output other scene labels according to the perception result information and the ego vehicle state information. Taking the “forward dangerous scene_vehicles” as an example, the perception result information input by the perception system includes the kinematics information and category information of the surrounding dynamic and static targets. When determining the scene label, all vehicles in the same lane as the ego vehicle are filtered out from the dynamic and static target information (assuming the result is two vehicles A and B), and further, based on the position information of A and B, the longitudinal position distance between A, B and the ego vehicle is calculated. If there is a vehicle in front of the ego vehicle and the longitudinal position distance between the vehicle and the ego vehicle is less than 200m, the “forward dangerous scene_vehicles” label is output. The judgment method of other scene classification labels is similar, which is not described herein.

[0020] Then, for step 102, the abnormal behavior detection device of the autonomous driving system determines a scene association threshold based on scene classification labels. The scene association threshold is negatively correlated with the severity level of an accident occurring in the scene corresponding to the scene classification label. In one specific embodiment, the mapping relationship between scene classification labels and scene association thresholds is pre-stored. Different scene classification labels result in different detection thresholds, which are scene association thresholds and are negatively correlated with the severity level of an accident. The severity level of an accident can be characterized, for example, by the scene risk level. For example, in a "forward hazard scenario," the severity of an accident involving a vulnerable road user (VRU) is higher than that involving a vehicle. Therefore, the threshold for unintended acceleration in "forward hazard scenario_vehicle" is larger than that in "forward hazard scenario_vulnerable road user." In other similar scenarios, the scene association threshold also varies depending on the road user and the scene risk level; this application does not impose any limitations on this.

[0021] Finally, for step 103, the abnormal behavior detection device of the autonomous driving system determines whether the vehicle exhibits abnormal behavior based on scene information and scene association thresholds. In a specific example, taking "Forward Dangerous Scene_Vehicle" as an example, if the speed of the vehicle in front is less than that of the vehicle itself, and the planned acceleration of the vehicle itself is too large (such as exceeding the corresponding scene association threshold), then it is considered unexpected acceleration, thus determining that abnormal behavior exists. The judgment methods for other scenes are similar and will not be elaborated here.

[0022] The method in this embodiment determines the scene association threshold based on scene classification labels, and then determines whether the vehicle has abnormal behavior based on scene information and scene association threshold. This allows for more accurate coverage of safety boundaries based on the scene, thereby reducing the risk of accidents.

[0023] In some optional embodiments, the scene classification labels include forward hazard scene_vehicle, forward hazard scene_obstacle, forward hazard scene_vulnerable road user, lateral hazard scene_vehicle, or lateral hazard scene_vulnerable road user. Abnormal behavior includes unexpected acceleration. Determining whether the vehicle exhibits abnormal behavior based on scene information and scene association thresholds includes any of the following: In the forward hazard scene_vehicle, if the speed of the vehicle in front is less than the vehicle's speed, and the planned acceleration of the vehicle is greater than a first acceleration threshold associated with the forward hazard scene_vehicle, then it is determined that the vehicle exhibits unexpected acceleration; or, in the forward hazard scene_obstacle, if the planned acceleration of the vehicle is greater than a second acceleration threshold associated with the forward hazard scene_obstacle. If a threshold is set, then the vehicle is determined to have unexpected acceleration; or, in a forward hazard scenario (vulnerable road user), if the vehicle's planned acceleration is greater than the third acceleration threshold associated with the forward hazard scenario (vulnerable road user), then the vehicle is determined to have unexpected acceleration; or, in a lateral hazard scenario (vehicle), if the vehicle in front has a lower speed than the vehicle and cuts into the vehicle's lane, and the vehicle's planned acceleration is greater than the fourth acceleration threshold associated with the lateral hazard scenario (vehicle), then the vehicle is determined to have unexpected acceleration; or, in a lateral hazard scenario (vulnerable road user), if the vehicle's planned acceleration is greater than the fifth acceleration threshold associated with the lateral hazard scenario (vulnerable road user), then the vehicle is determined to have unexpected acceleration. Therefore, different scenario-related thresholds can be used to determine whether the vehicle has unexpected acceleration for different forward and lateral hazard scenarios. Besides using the above acceleration thresholds, other thresholds that can determine unexpected acceleration can also be set according to actual usage, which will not be elaborated here. Further optionally, the setting of each acceleration threshold is negatively correlated with the scenario risk level. The higher the risk level of a scenario, the lower its corresponding scenario association threshold.

[0024] Optionally, the scene classification labels include rear-approaching vehicle scenarios, and abnormal behavior includes unexpected deceleration. Determining whether the vehicle exhibits abnormal behavior based on scene information and scene-related thresholds includes: in a rear-approaching vehicle scenario but not a forward hazard scenario, if the vehicle's planned deceleration is greater than the deceleration threshold associated with the rear-approaching vehicle scenario, then the vehicle is determined to have engaged in unexpected deceleration. Forward hazard scenarios include forward hazard scenario_vehicle, forward hazard scenario_obstacle, and / or forward hazard scenario_vulnerable road users. Therefore, for rear-approaching vehicle scenarios, the vehicle's unintended deceleration can be determined based on the deceleration threshold associated with that scenario. Besides using the aforementioned deceleration threshold, other thresholds that can determine unintended deceleration can also be set according to actual usage, which will not be elaborated here.

[0025] Optionally, the scenario classification labels include scenarios with conflicting vehicles in adjacent lanes, and abnormal behaviors include unexpected lane changes. Determining whether a vehicle exhibits abnormal behavior based on scenario information and scenario-related thresholds includes: in scenarios with conflicting vehicles in adjacent lanes, if the vehicle plans a lane change and the planned lane change preparation time is less than a preset lane change time threshold, then the vehicle is determined to have engaged in an unexpected lane change. Therefore, for scenarios with conflicting vehicles in adjacent lanes, the preset lane change time threshold associated with that scenario can be better utilized to determine whether the vehicle has engaged in an unexpected lane change. Besides using the preset lane change time threshold, other scenario-related thresholds can also be set according to actual circumstances to determine unexpected lane change behavior, which will not be elaborated upon here.

[0026] In some optional embodiments, the scene classification labels include slope scenes, curve scenes, medium-speed scenes, and / or high-speed scenes, and abnormal behavior includes unexpected vehicle control failure, where unexpected vehicle control failure refers to the vehicle failing to issue a control request when it should have. Determining whether the vehicle exhibits abnormal behavior based on scene information and scene-related thresholds includes: in slope scenes, curve scenes, medium-speed scenes, and / or high-speed scenes, if the vehicle does not issue a control request, then it is determined that the vehicle exhibits unexpected vehicle control failure. Therefore, for slope scenes, curve scenes, medium-speed scenes, and / or high-speed scenes, scene-related thresholds can be set separately to determine whether the vehicle exhibits unexpected vehicle control failure.

[0027] In some alternative embodiments, the scene classification label includes a curve scene, and the abnormal behavior includes unexpected lateral deviation. Determining whether the vehicle exhibits abnormal behavior based on scene information and scene association thresholds includes: in a curve scene, if the difference between the curvature of the vehicle's planned trajectory and the curvature of the road is greater than a preset curvature difference threshold, then it is determined that the vehicle has an unexpected lateral deviation. Therefore, for curve scenes, a preset curvature difference threshold can be used to determine unexpected lateral deviation. Besides using the preset curvature difference threshold, other scene association thresholds can also be selectively set according to actual conditions to determine unexpected lateral deviation behavior, which will not be elaborated here.

[0028] In this embodiment, the specific value of the scene association threshold is related to the scene classification label. In a specific example, in the forward hazard scene "vehicle", the vehicle's acceleration is greater than a first acceleration threshold, for example, greater than 5 m / s². 2 (First acceleration threshold). The second acceleration threshold could be, for example, 4 m / s². 2 The third acceleration threshold can be, for example, 3 m / s². 2 The fourth acceleration threshold could be, for example, 4 m / s². 2 The fifth acceleration threshold can be, for example, 5 m / s². 2 This allows for the use of different thresholds in different scenarios, and these thresholds are negatively correlated with the risk level of the scenario, thus enabling better customization of different thresholds for different scenarios.

[0029] In this embodiment, different scene classification labels correspond to different detection thresholds, unlike the design of using the same threshold for all scenes (e.g., all forward hazard scenes). Scene risk level is an inherent attribute of the scene. For example, in a "forward hazard scene," the severity of an accident involving a VRU is higher than that involving a vehicle because the VRU is more vulnerable. Therefore, the threshold for unintended acceleration of the vehicle in a forward hazard scene is higher. This embodiment uses scene-related thresholds instead of static thresholds to accurately identify unintended behaviors under different operating conditions. Because the thresholds for different operating conditions are related to the scene, it is possible to more accurately determine whether unintended behavior has occurred. Furthermore, the threshold settings can be dynamically adjusted according to different scenes.

[0030] In some optional embodiments, scene information includes vehicle state information, road information, and / or perception result information. The perception result information includes information on dynamic and static targets around the vehicle. This dynamic and static target information may include category information and kinematic information of targets such as other vehicles, VRUs, road boundaries, and temporary facilities. In a specific example, scene classification labels may include, for example, the following: high-speed scene, medium-speed scene, forward hazard scene (vehicle), forward hazard scene (obstacle), forward hazard scene (vulnerable road user), lateral hazard scene (vehicle), lateral hazard scene (vulnerable road user), curve scene, slope scene (sharp slope), rear-approaching vehicle scene, adjacent lane conflict vehicle scene, and other conventional scenes. The above is only one specific example of scene classification labels. Those skilled in the art will understand that scene classification labels can have other combinations, and this application does not limit this.

[0031] The scene category labels output based on scene information can include any one or a combination of the following (multiple scene category labels can be output simultaneously): Based on the vehicle's status information, determine whether the vehicle's speed is within a preset speed range, which includes medium speed and high speed. If the vehicle's speed is within the preset speed range, output the corresponding scene classification label based on the speed range in which the vehicle is located. For example, a vehicle speed > 80 kph (kilometers per hour) can be set as a high-speed scene, and a vehicle speed > 10 kph and ≤ 80 kph can be set as a medium-speed scene.

[0032] Based on road information, the system determines whether to output scene classification labels related to the road information. Specifically, when the slope is greater than a preset slope threshold (e.g., a road slope > 5% (a 5% slope means an increase of 5 meters in elevation for every 100 meters traveled), a scene classification label related to the slope is output (e.g., "Slope Scene"). And / or, when the vehicle is traveling at a constant speed along the tangent of the current lane centerline, and the time it takes to cross the road boundary is less than a preset time threshold (e.g., when the vehicle is on a curve, the direction of a tangent to the lane centerline at its current location indicates the direction the vehicle is pointing), the system assumes the driver makes no steering corrections and does not change speed, and the vehicle will exit the curve in a straight line. Assuming the preset time threshold is 1.5 seconds, if the time it takes for the vehicle to cross the road boundary is less than 1.5 seconds, a scene classification label related to the road curvature is output (e.g., "Curve Scene").

[0033] The system iterates through information on both static and dynamic objects surrounding the vehicle, and selects the first target object located in front of the vehicle in the same lane. The first target object can be categorized as a vehicle, obstacle, or vulnerable road user. When a first target object exists and the distance between the vehicle and the first target object is less than a first preset distance threshold, a scene classification label related to the forward hazard scenario and the category of the first target object is output. For example, if there is a vehicle within 200 meters ahead of the vehicle in the lane, the output is "Forward Hazard Scenario_Vehicle"; if there is a static obstacle (such as a manhole cover or rock) within 200 meters ahead of the vehicle in the lane, the output is "Forward Hazard Scenario_Obstacle"; if there is a pedestrian / two-wheeled vehicle within 200 meters ahead of the vehicle in the lane, the output is "Forward Hazard Scenario_Vulnerable Road User".

[0034] The system iterates through dynamic and static object information around the vehicle, and filters out second objects that are in adjacent lanes and cutting into the vehicle's lane. These second objects are categorized as vehicles and vulnerable road users. When a second object exists and the distance between the vehicle and the second object is less than a second preset distance threshold, a scene classification label related to the lateral hazard scenario and the second object's category is output. For example, if a vehicle cuts into the vehicle's lane within 200m of the adjacent lane, the system outputs "lateral hazard scenario_vehicle"; if a pedestrian / two-wheeled vehicle intrudes into the vehicle's lane within 200m of the adjacent lane, the system outputs "lateral hazard scenario_vulnerable road user".

[0035] The system iterates through the static and dynamic objects surrounding the vehicle, filtering out vehicles behind it. If a target vehicle is behind the vehicle and its speed is greater than or equal to the vehicle's speed, it outputs a scene classification label related to the following vehicle. For example, if a vehicle is following the vehicle and its speed is greater than or equal to the vehicle's speed, it outputs a scene of following vehicles.

[0036] The system iterates through static and dynamic objects around the vehicle and filters out vehicles in adjacent lanes based on this information. When the speed of a vehicle behind the vehicle in an adjacent lane exceeds a first preset percentage of the vehicle's speed, or the speed of a vehicle in front of the vehicle in an adjacent lane is lower than a second preset percentage of the vehicle's speed, a scenario classification label related to a conflicting vehicle in the adjacent lane is output. For example, if there is a vehicle adjacent to the vehicle in the adjacent lane, or a slow vehicle (speed < 20% of the vehicle's speed) in front of the vehicle in the adjacent lane, or a fast vehicle (speed > 20% of the vehicle's speed) behind the vehicle in the adjacent lane, a scenario of a conflicting vehicle in the adjacent lane is output.

[0037] If none of the above scenarios exist, output the scenario category label related to the regular scenario, such as "other regular scenarios".

[0038] In a specific example, taking "Forward Hazardous Scene_Vehicle" as an example, the perception result information input by the perception system includes the state information and category information of surrounding dynamic and static targets. During calculation, the dynamic and static target information is traversed, and all vehicles in the same lane as the vehicle are selected (assuming the result is two vehicles, A1 and A2). Furthermore, based on the position information of A1 and A2, the longitudinal distance between A1, A2 and the vehicle is calculated. If there is a vehicle in front of the vehicle and the longitudinal distance between them is less than 200m, the label "Forward Hazardous Scene_Vehicle" is output; otherwise, other labels are output, depending on the calculation results of other labels.

[0039] Therefore, the embodiments of this application can determine the corresponding scene classification labels through the above methods. Then, based on the scene classification labels, the corresponding scene association thresholds can be obtained to determine whether the vehicle exhibits unexpected behavior.

[0040] Furthermore, after the autonomous driving system determines that the vehicle is exhibiting unexpected behavior (abnormal behavior), in order to avoid harmful events, the autonomous driving system can make a degrade decision based on a comprehensive analysis of factors such as the duration of the abnormal behavior, the time of driver takeover, and vehicle malfunctions, and control the vehicle based on the decision results. This can improve the accuracy of the degrade and effectively enhance system availability and user experience.

[0041] Please refer to Figure 2 The document illustrates a flowchart of an autonomous driving system downgrade method according to an embodiment of this application. The method can be applied to a mobile platform equipped with a driving automation system for downgrading the autonomous driving system. This mobile platform can be, for example, a vehicle, a mobile robot, an aircraft, etc., and this application does not limit the scope of the application. The executing entity of this method can be an autonomous driving system downgrade device, which can be a software or a combination of software and hardware (such as a controller in a mobile platform).

[0042] like Figure 2As shown, in step 201, abnormal behavior of the vehicle is detected, and / or a vehicle malfunction is detected, and the duration of the abnormality is output; in step 202, the collision time between the vehicle and the target object is predicted based at least on the scene information; in step 203, the driver takeover time is determined based on the collision time and the duration of the abnormality; in step 204, whether to trigger a downgrade is determined based on the driver takeover time and the duration of the abnormality.

[0043] In this embodiment, for step 201, the autonomous driving system downgrade device detects abnormal behavior of the vehicle and / or detects a vehicle malfunction, and outputs the duration of the abnormality. The vehicle malfunction may include at least one of the following: message loss, message data exceeding limits, message data jump, and message data stagnation. Message loss may include the loss of key data such as perception messages, decision-making and planning messages, and control execution messages; exceeding limits may include exceeding limits for key parameters in perception, decision-making and planning, and control execution messages, such as exceeding speed limits, acceleration limits, or jerk limits; message data jump may include jumps in key parameters in perception, decision-making and planning, and control execution messages, such as jumps in position or speed messages; and message stagnation may include stagnation in parameters in perception, decision-making and planning, and control execution messages, such as stagnation in trajectory data. The duration of the abnormality includes the duration of the abnormal behavior and / or the duration of the vehicle malfunction. For example, the duration of the abnormality may be determined based on the timestamp of the abnormal behavior and / or vehicle malfunction detection signal and the current time, or it may be statistically analyzed using a system timer (e.g., a timer is activated when abnormal behavior and / or vehicle malfunction are detected).

[0044] Then, for step 202, the autonomous driving system degradation device predicts the collision time between the vehicle and the target object based at least on scene information. Taking "Forward Hazardous Scenario_Vehicle" as an example, the autonomous driving system predicts the trajectory of the vehicle and the trajectory of the vehicle in front based on the perception system (such as the perception module in the end-to-end model). Based on the trajectory of the vehicle and the predicted trajectory of the vehicle in front, the system calculates the collision time by first aligning the timestamps of the starting point of the planned trajectory of the vehicle and the predicted trajectory point of the vehicle in front. Then, it calculates the lateral and longitudinal distances between the trajectory points of the vehicle and the predicted trajectory points of the vehicle in front at the same time. If the lateral distance between any pair of trajectory points is less than 2m or the longitudinal distance is less than 5m, it is determined to be a collision trajectory point. The timestamp of the collision trajectory point minus the time of the trajectory starting point is the collision time. If there is no collision trajectory point, the default collision time is 100s. The collision time calculation for other scenarios is similar to the above example and will not be repeated here.

[0045] Next, in step 203, the autonomous driving system downgrade device determines the driver takeover time based on the collision time and the duration of the anomaly. For example, when the vehicle is traveling on a highway, the system suddenly detects a construction area ahead, and the current vehicle path is not safe to pass. If the vehicle does not change lanes, it will collide with the obstacle after 10 seconds (collision time). At this time, if a takeover request is issued, the total time window (driver takeover time) for the driver is 10 seconds. If the anomaly has lasted for 2 seconds, and the vehicle continues to move forward and gets closer to the obstacle within these 2 seconds, then the total time window for the driver is 8 seconds.

[0046] Finally, for step 204, the autonomous driving system degradation device determines whether to trigger degradation based on the driver takeover time and the duration of the anomaly. For example, if the driver takeover time threshold is 1.5s, taking "trajectory message data loss in a following scenario" as an example, if the collision time is 2.0s and the trajectory message data loss duration (anomaly duration) is 0.5s, then the driver's takeover time is 1.5s, which is equal to the driver takeover time threshold, and degradation needs to be triggered; if the collision time is 5s and the trajectory message data loss duration is 0.5s, then the driver's takeover time is 4.5s, which is greater than the driver takeover time threshold, and degradation does not need to be triggered; if the trajectory message data loss duration threshold is 2.0s, the collision time is 5.0s (less than the threshold of 10s), and the trajectory message data loss duration is 2.0s, then the driver's takeover time is 3.0s, which is greater than the driver takeover time threshold, but because the trajectory message data loss duration is too long, degradation needs to be triggered.

[0047] The method in this application calculates the driver takeover time by measuring the collision time and the duration of the anomaly, and then determines whether to trigger a downgrade based on the driver takeover time and the duration of the anomaly. This allows for a more accurate downgrade determination based on whether the driver takeover time is sufficient.

[0048] In some optional embodiments, vehicle malfunctions include message loss, abnormal message data exceeding limits, message lag, and / or message data jumps. Detecting a vehicle malfunction includes: if no upstream message is received within a first time threshold, then it is determined that the vehicle has experienced message loss, wherein the upstream message includes perception messages, decision planning messages, and / or control execution messages, and the first time can be determined according to the system's detection accuracy requirements, for example, 0.5s for high accuracy requirements or 1s for low accuracy requirements; and / or, if the data parameter values ​​in the upstream message exceed the corresponding preset value threshold, then it is determined that the vehicle has experienced message loss. Data anomalies exceeding limits, such as speed, acceleration, and jerk exceeding limits, are identified. Different parameter types have different preset threshold values, which can be set according to actual needs. Alternatively, if the upstream message remains unchanged except for the timestamp, it indicates a message lag, such as trajectory data lag. And / or, if the difference between two consecutive frames of data parameters in the upstream message exceeds the corresponding physical limit, it indicates a message data jump, such as position or speed jumps. The physical limit is an empirical value derived from historical vehicle information or common sense physics. Therefore, the existence and specific type of vehicle malfunction can be determined using these methods.

[0049] In some optional embodiments, after determining whether a downgrade has been triggered, the method further includes: if a downgrade has been triggered, alerting the driver to take over within a first time frame. For example, 0-500ms after receiving the downgrade command, a takeover request is sent to the driver via an HMI interface (such as a red warning icon on the dashboard) + audible and visual alerts (such as continuous beeping + steering wheel vibration). If the driver does not take over within the first time frame, the system actively decelerates and activates the hazard warning lights within a second time frame. For example, 500ms-T (T can be preset to 2 seconds or 3 seconds, which is not limited in this application) after receiving the downgrade command, if the driver still does not take over, the system performs active deceleration (deceleration -1m / s²). 2 The system will activate hazard warning lights. If the driver still does not take over within the second timeframe, the vehicle speed will be gradually reduced to 0, and the system will exit autonomous driving mode and switch to manual driving. Furthermore, in high-speed scenarios, to improve safety, if the driver still does not take over within the second timeframe, the system can prioritize controlling the vehicle to stop in the emergency lane. This forms a tiered degrading execution mechanism, dynamically selecting step-by-step response strategies such as warnings, deceleration, and exiting intelligent driving mode based on the sufficiency of takeover time or the duration of abnormal situations.

[0050] In some optional embodiments, the method further includes: during active deceleration, maintaining the vehicle's longitudinal deceleration below a preset deceleration threshold and the vehicle's lateral offset below a preset offset threshold, thereby avoiding passenger discomfort and ensuring lane safety. The preset deceleration threshold and preset offset threshold are empirical values ​​determined based on extensive historical data to improve comfort and safety. For example, during active deceleration, ensuring longitudinal deceleration < 2 m / s² 2 This can prevent passenger discomfort and ensure lateral offset is less than 0.5m, thus ensuring lane safety. However, the thresholds may vary depending on the historical user driving data used by different vehicles, allowing for a more tailored approach to the comfort and safety needs of different drivers.

[0051] In some optional embodiments, determining whether to trigger degradation based on driver takeover time and the duration of the anomaly includes: if the driver takeover time is less than the takeover time threshold, or the duration of the anomaly is greater than the maximum duration threshold, then degradation is triggered; if the driver takeover time is greater than or equal to the takeover time threshold, and the duration of the anomaly is less than or equal to the maximum duration threshold, then degradation is not triggered. For example, taking the loss of trajectory message data in a following scenario as an example, if the takeover time threshold is 1.5s, the collision time is 2.0s, and the duration of trajectory message data loss is 0.5s, then the driver takeover time is 1.5s, which is equal to the driver takeover time threshold, prompting a degradation suggestion; if the duration of trajectory message data loss is 2.0s, the collision time is 5.0s (less than the threshold of 10s), and the duration of trajectory message data loss is 2.0s, then the driver takeover time is 3.0s. Although the driver takeover time is greater than the driver takeover time threshold, the duration is too long, requiring direct degradation. Thus, when the time left for driver takeover is insufficient to reach the takeover time threshold or the duration of the anomaly exceeds the duration threshold, degradation is triggered immediately, improving system security.

[0052] In some optional embodiments, the takeover time threshold and / or maximum duration threshold are related to the scene classification label and negatively correlated with the severity level of an accident in the scene corresponding to the scene classification label. For example, the severity level of an accident in a high-speed curve scene is higher than that in other conventional scenes. Therefore, the trajectory loss threshold in a high-speed curve scene can be shortened from 2s (the threshold corresponding to other conventional scenes) to 800ms, and the maximum duration threshold can be shortened from 2s (the threshold corresponding to other conventional scenes) to 1s. In high-speed scenes, the scene association threshold, takeover time threshold, and maximum duration threshold can also be shortened accordingly to ensure that the takeover window left for the driver in high-risk scenarios (such as triggering a warning when TTC < 1.0s) meets the safety redundancy requirements and avoids the accident risk caused by fixed thresholds.

[0053] In some optional embodiments, predicting the collision time between the vehicle and the target object based on scene information includes: predicting the future trajectory information of the target object using a kinematic model or the prediction module of the autonomous driving system based on scene information; calculating the collision time between the vehicle and the target object based on the future trajectory information of the target object and the trajectory information of the vehicle, wherein the trajectory information of the vehicle uses the planned trajectory of the vehicle, and when the planned trajectory of the vehicle is not updated in time, the trajectory information predicted by the kinematic model is used. The kinematic model is the basis and constraint for the planned trajectory, and the planned trajectory is the expected realization of the kinematic model. The planned trajectory of the vehicle refers to a specific and executable path planned by the decision-making layer of the intelligent driving system (such as using an artificial intelligence model) for the vehicle (i.e., the vehicle itself) over a future period. The kinematic model describes the mathematical relationship between the vehicle's position, speed, orientation, and other state variables and the control input, without considering the forces that cause the vehicle's movement (such as tire friction, air resistance, etc., which fall under the category of dynamics models). It is essentially a mathematical abstraction of the vehicle's geometry and motion.

[0054] In a specific example, based on predicted information about moving and static objects (including future trajectories of other vehicles and VRUs, and location information of road boundaries and temporary facilities), combined with the vehicle's trajectory information, the collision time between the vehicle and the moving and static objects is calculated. The vehicle's trajectory information includes both planned trajectory information and trajectory information predicted by the kinematic model. When the planned trajectory information is not updated in a timely manner, the trajectory information predicted by the kinematic model is used for collision calculation; otherwise, it is not used. Therefore, when the trajectory information is not updated in a timely manner, its accuracy is lower than that of the kinematic model; using the trajectory information predicted by the kinematic model yields more accurate results.

[0055] In some optional embodiments, calculating the collision time between the vehicle and the target object based on the target object's future trajectory information and the vehicle's trajectory information includes: aligning the timestamps of the starting point of the vehicle's trajectory information with the starting point of the target object's future trajectory information; calculating the lateral and longitudinal distances between trajectory points in the vehicle's trajectory information and trajectory points in the target object's future trajectory information at the same moment; if at any moment the lateral distance between two trajectory points is less than a lateral threshold or the longitudinal distance between two trajectory points is less than a longitudinal threshold, then it is determined that the vehicle and the target object have collided, and the collision time is obtained by subtracting the timestamp of the starting point from the timestamp of the trajectory point at that moment; and if at any moment the lateral distance between two trajectory points is greater than or equal to the lateral threshold and the longitudinal distance between two trajectory points is greater than or equal to the longitudinal threshold, then the default time is determined as the collision time.

[0056] In a specific example, taking "Forward Dangerous Scenario_Vehicle" as an example, calculations are performed based on the vehicle's trajectory and the predicted trajectory of the vehicle in front. First, the timestamps of the starting point of the vehicle's planned trajectory and the predicted trajectory points of the vehicle in front are aligned. Then, the horizontal and vertical distances between the trajectory points of the vehicle and the predicted trajectory points of the vehicle in front are calculated at the same moment, resulting in the horizontal and vertical distances of multiple pairs of trajectory points. After that, if the horizontal distance of any pair of trajectory points is less than 2m or the vertical distance is less than 5m, a collision is determined. The collision time is the timestamp of the collision trajectory point minus the time of the trajectory starting point. If there is no collision, the default collision time is 100s.

[0057] Please refer to Figure 3 This document illustrates a flowchart of an anomaly detection and degradation process, representing a specific example of an embodiment of this application. The purpose of this application is to design an anomaly detection and degradation method and system for a driving automation system, aiming to address the problems of excessive degradation, missed detection of unexpected behaviors, and insufficient driver takeover time in existing detection mechanisms. This will help improve the usability of autonomous driving systems while ensuring safety, preventing users from experiencing negative impacts from frequent erroneous degradations, and ensuring adequate safety boundary coverage and sufficient human-machine takeover buffer windows in complex scenarios.

[0058] Regarding the issue of excessive degradation: Using this application, the system can comprehensively assess the actual risk of a single fault by combining scenario information (such as highway / curve scenarios, traffic participant status). For example, when millimeter-wave radar experiences a brief data loss, it can determine that no degradation is necessary through lidar data compensation, avoiding unnecessary degradation operations triggered by minor anomalies that can be covered by redundant systems. Regarding the issue of missed detection of unexpected behavior: Using this application, the system can use scenario-related thresholds (such as 3m / s in a following vehicle scenario) to detect unexpected behavior. 2 Acceleration triggers warnings, replacing static thresholds and solving the problem of traditional fixed thresholds (such as 4 m / s²). 2 This invention addresses the safety boundary failure issue in low-speed, close-range scenarios, accurately identifying unexpected behaviors under different operating conditions (such as slight overspeeding during low-speed following). Regarding insufficient driver takeover time: using this application, the system can dynamically calculate the takeover time based on the scenario's risk level (e.g., reducing the trajectory loss threshold from 2s to 800ms in high-speed curve scenarios), ensuring that the takeover window for the driver in high-risk scenarios (e.g., triggering a warning when TTC < 1.0s) meets safety redundancy requirements, avoiding accident risks caused by fixed thresholds.

[0059] like Figure 3 As shown, this application proposes a method and system for anomaly detection and degradation in a driving automation system. Through the execution of a detection module (for dynamic scene perception), an evaluation module (for deep collaboration between multi-dimensional anomaly identification and risk controllability assessment), and a degradation module, it achieves accurate safety degradation decisions and scenario adaptability. The method of this application may include the following steps: Step 1: Detection module execution. The detection module is used to detect scene information ( Figure 3 External scene detection), system anomaly detection Figure 3 In the context of vehicle anomaly detection, system anomalies include vehicle malfunctions and unexpected behaviors. Figure 3 The scene detection module collects and outputs the following information in real time through sensor fusion data: input information and scene classification labels (in the vehicle abnormality information). Figure 3 (Key scene information in the text).

[0060] The input information may include: road information: road curvature (which can be calculated in real time), slope (which can be obtained based on IMU and map data); infrastructure: speed limit signs (which can be visually recognized), guardrail type (which can be obtained through map matching), and safety island location; temporary facilities: cones, water-filled barriers, and construction signs (which can be detected using visual and lidar methods); traffic participants: vehicles (speed, position, heading angle) and vulnerable traffic participants (such as pedestrians and two-wheeled vehicles with real-time movement trajectories) within a preset range (e.g., 200m) in front of the lane and adjacent lanes.

[0061] Scene classification output (12 categories in total): High-speed scene: Vehicle speed > 80kph; Medium-speed scene: Vehicle speed > 10kph and ≤ 80kph; Forward hazard scene - Vehicle: There is a vehicle in front within 200m of the vehicle's lane; Forward hazard scene - Obstacle: There is a static obstacle (such as a manhole cover or stone) within 200m of the vehicle's lane; Forward hazard scene - Vulnerable road user: There is a pedestrian / two-wheeled vehicle within 200m of the vehicle's lane; Lateral hazard scene - Vehicle: A vehicle cuts into the vehicle's lane within 200m of the adjacent lane; Lateral hazard scene - Vulnerable road user: A pedestrian / two-wheeled vehicle intrudes into the vehicle's lane within 200m of the adjacent lane; Curve scene (high curvature): When the vehicle is traveling at a constant speed along the tangent of the current lane center, the time to cross the road boundary is < t_curve (e.g., 1.5s); Slope scene (high gradient): Road gradient > s_slope (e.g., 5%); Rear approach vehicle scenario: There is a vehicle following behind the vehicle and its speed is greater than or equal to the vehicle's speed; Conflicting vehicle scenario in adjacent lane: There is a vehicle in the adjacent position of the adjacent lane, or there is a slow vehicle in front of the adjacent lane (speed < 20% of the vehicle's speed), or there is a fast vehicle behind the adjacent lane (speed > 20% of the vehicle's speed); and other common scenarios: Low-risk conditions other than those mentioned above.

[0062] The anomaly detection module detects vehicle malfunctions (including message loss, abnormal message data exceeding limits, message data jumps, etc.) and abnormal vehicle behaviors (including unexpected acceleration, unexpected deceleration, unexpected lane changes, unexpected loss of vehicle control, unexpected lateral deviation, etc.), and outputs the duration of the anomaly t_abnormal. Figure 3 The abnormal information of the vehicle may also include the duration of the abnormality.

[0063] The aforementioned vehicle malfunctions encompass faults covering the entire perception / decision / control link. Faults include message loss, abnormal message data exceeding limits, message data jumps, and message data stagnation. Specifically, message loss refers to the downstream system not receiving messages from the upstream system for consecutive time intervals (ts), including the loss of critical data such as perception messages, decision-making and planning messages, and control execution messages; abnormal data exceeding limits refers to data parameter values ​​exceeding thresholds, including exceeding limits for critical parameters in perception, decision-making and planning, and control execution messages, such as exceeding speed, acceleration, and jerk limits; abnormal message data exceeding limits refers to the difference between values ​​in two consecutive frames exceeding physical limits, including jumps in critical parameters in perception, decision-making and planning, and control execution messages, such as jumps in position and speed messages; and message stagnation refers to message data remaining unchanged except for timestamp changes, including parameter stagnation in perception, decision-making and planning, and control execution messages, such as trajectory data stagnation.

[0064] The abnormal behaviors of the autonomous vehicle include unexpected acceleration, unexpected deceleration, unexpected lane change, unexpected loss of vehicle control, and unexpected lateral deviation. Among them, unexpected acceleration refers to the autonomous vehicle planning excessive acceleration in forward-facing or lateral-facing hazardous scenarios. The specific judgment logic depends on the scenario. In a forward hazard scenario (vehicle), if the speed of the vehicle in front is less than that of the vehicle, and the vehicle's planned acceleration is excessively large (> acc_veh_thres), it is considered unintended acceleration. In a forward hazard scenario (obstacle), if the vehicle's planned acceleration is excessively large (> acc_obs_thres), it is considered unintended acceleration. In a forward hazard scenario (vulnerable road user), if the vehicle's planned acceleration is excessively large (> acc_vru_thres), it is considered unintended acceleration. In a lateral hazard scenario (vehicle), if the speed of the vehicle in front is less than that of the vehicle and it cuts into the vehicle's lane, and the vehicle's planned acceleration is excessively large (> acc_cutin_thres), it is considered unintended acceleration. In a lateral hazard scenario (vulnerable road user), if the vehicle's planned acceleration is excessively large (> acc_latvru_thres), it is considered unintended acceleration. Unintended deceleration refers to an excessively large deceleration planned by the vehicle in a backward hazard scenario. The specific judgment logic is scenario-dependent; in a scenario with oncoming traffic but not a forward hazard scenario, if the planned deceleration of the vehicular vehicle is too large (> dec_rear_thres, a scenario-related deceleration threshold, for example, 5 m / s²), it indicates a problem. 2 The unintended lane change behavior refers to a scenario where there are conflicting vehicles in the adjacent lane, and the vehicle plans a lane change. The specific judgment logic is scenario-dependent. In a scenario where there are conflicting vehicles in the adjacent lane, the vehicle plans a lane change, and the planned lane change preparation time is very short (< t_lanechange, a scenario-related lane change preparation time threshold, for example, 5 m / s). 2The aforementioned "unexpected lack of vehicle control" refers to situations where, in scenarios such as slopes, curves, medium speeds, and high speeds, the vehicle should have taken control, but instead, it issued a control request. The aforementioned "unexpected lateral deviation behavior" refers to situations where, in curve scenarios, the curvature of the vehicle's planned trajectory differs excessively from the curvature of the road.

[0065] Step 2: Evaluation Module Execution. Based on the scenario classification results and abnormal states, the evaluation module calculates and outputs downgrade instructions through the risk calculation submodule, controllability calculation submodule, and downgrade judgment submodule.

[0066] The risk calculation submodule calculates the collision time t_collision. Figure 3 (Time of occurrence of the hazard).

[0067] Specifically, based on predicted information about dynamic and static targets (including future trajectory information of other vehicles and VRUs, and location information of road boundaries and temporary facilities), combined with the vehicle's trajectory information, the collision time between the vehicle and the dynamic and static targets is calculated. The vehicle's trajectory information includes both planned trajectory information and trajectory information predicted by the kinematic model. When the planned trajectory information is not updated in a timely manner, the trajectory information predicted by the kinematic model is used for collision calculation; otherwise, it is not used. The future trajectory information of other vehicles and VRUs can be predicted using the kinematic model or the trajectory prediction results output by the prediction module used in existing autonomous driving systems.

[0068] The controllability calculation submodule calculates the time t_ov that the driver can take over before the collision. That is, based on the collision time t_collision output by the risk calculation module and combined with the abnormality duration t_abnormal output by the detection module, the driver's takeover time t_ov = t_collision - t_abnormal is calculated in the current scenario after the vehicle abnormality occurs.

[0069] The degradation judgment submodule determines whether degradation is necessary by judging the driver's takeover time, the risk level, and the duration of the abnormality. If the time allotted for the driver to take over is less than t_ov_thres s or the duration of the abnormality exceeds t_max, a degradation command is issued immediately.

[0070] Step 3: Degradation Module Execution. Upon receiving the degradation command, the degradation module performs the following operations according to priority: 1. Level 1 Response (0-500ms): Sends a takeover request to the driver via the HMI interface (e.g., a red warning icon on the instrument panel) + audible and visual alerts (e.g., continuous beeping + steering wheel vibration); 2. Level 2 Response (500ms-T): If the driver does not take over, the system performs active deceleration (deceleration -1m / s²). 21. Activate hazard warning lights; 2. Level 3 response (T seconds later, T is preset to 3 seconds): If the system still fails to take over, it will gradually reduce the vehicle speed to 0 (prioritizing stopping in the emergency lane in highway scenarios), and exit intelligent driving mode to switch to manual driving. Key parameter: Ensure longitudinal deceleration < 2m / s² during deceleration. 2 (To avoid passenger discomfort), lateral offset <0.5m (to ensure safety within the lane).

[0071] This application can also comprehensively assess the actual risk of a single failure by combining scenario information (such as highway / curving scenarios, and the status of traffic participants). For example, when millimeter-wave radar experiences a brief data loss, LiDAR data compensation can be used to prevent the vehicle from exhibiting unexpected behavior and thus avoid degrading. This avoids unnecessary degradation operations triggered by minor anomalies that can be covered by the redundant system. Therefore, scenario-based risk assessment avoids degradation triggered by minor anomalies that can be covered by redundancy (such as a brief data loss from a single sensor), reducing unnecessary functional interruptions. For example, in a multi-sensor perception system, when one sensor temporarily fails, data compensation from other sensors can maintain system operation, reducing user experience disruption caused by frequent degradations.

[0072] This application's embodiments use scene-related thresholds instead of static thresholds to solve the problem of missed detections in complex scenarios using traditional fixed thresholds. For example, in low-speed following scenarios (where the distance between the vehicle and the vehicle in front is close), the unexpected acceleration detection threshold is lowered to ensure timely identification under close-range collision risks; in high-speed curve scenarios, the lateral deviation threshold is tightened to reduce the risk of side-impact accidents caused by trajectory deviation. This allows for accurate coverage of safety boundaries and reduces accident risks. This application's embodiments dynamically adjust the detection thresholds based on the scenario risk level to ensure a takeover buffer window in high-risk scenarios. For example, in high-risk scenarios such as high-speed curves, the trajectory anomaly response time is shortened to ensure that the driver's effective takeover time meets safety standards; in complex scenarios such as construction areas, the anomaly detection delay is compressed to avoid takeover failure due to response lag. This ensures sufficient takeover time and enhances human-machine collaborative safety. The scene-related threshold mechanism of this application's embodiments reduces the "convergence-verification" iteration cycle of traditional fixed thresholds and adapts to different vehicle models through scenario-based parameter configuration, reducing multi-platform development costs. At the same time, the risk assessment model can be continuously optimized through OTA (Over-The-Air), reducing reliance on hardware upgrades.

[0073] In other embodiments, this application also provides a non-volatile computer storage medium storing computer-executable instructions that can execute the abnormal behavior detection method and / or the autonomous driving system degradation method in any of the above method embodiments.

[0074] Non-volatile computer-readable storage media may include a stored program area and a stored data area, wherein the stored program area may store an operating system and an application program required for at least one function; the stored data area may store data created based on the use of the autonomous driving system abnormal behavior detection device and / or the autonomous driving system degradation device, etc. Furthermore, the non-volatile computer-readable storage medium may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the non-volatile computer-readable storage medium may optionally include memory remotely configured relative to the processor, and these remote memories may be connected via a network to the autonomous driving system abnormal behavior detection method and / or the autonomous driving system degradation device. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0075] This application also provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer, cause the computer to perform any of the above-described methods for detecting abnormal behavior in an autonomous driving system and / or for downgrading an autonomous driving system.

[0076] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 4 As shown, the device includes one or more processors 410 and a memory 420. Figure 4 Taking a processor 410 as an example, the device for executing the abnormal behavior detection method and / or the autonomous driving system degradation method may further include an input device 430 and an output device 440. The processor 410, memory 420, input device 430, and output device 440 may be connected via a bus or other means. Figure 4 Taking a bus connection as an example, the memory 420 is the aforementioned non-volatile computer-readable storage medium. The processor 410 executes various server functions and data processing by running non-volatile software programs, instructions, and modules stored in the memory 420, thereby implementing the above-described method embodiment's abnormal behavior detection method and / or autonomous driving system degradation method. The input device 430 can receive input digital or character information and generate key signal inputs related to user settings and function control of the autonomous driving system abnormal behavior detection device and / or autonomous driving system degradation device. The output device 440 may include a display device such as a screen.

[0077] This application also provides a mobile platform, which includes: a vehicle body, a power system, and electronic equipment as described in the above embodiments. The power system is installed on the vehicle body and provides power; the principle and implementation of the electronic equipment are consistent with those described in the above embodiments, and will not be repeated here. The electronic equipment may be a controller or other computing device installed on the mobile platform. Optionally, the mobile platform may include at least one of the following: a vehicle, a mobile robot, an unmanned vehicle, an aircraft, etc.

[0078] The above-described product can perform the methods provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for performing the methods. Technical details not described in detail in this embodiment can be found in the methods provided in the embodiments of this application.

[0079] In one embodiment, the electronic device described above is applied in the above-mentioned autonomous driving system abnormal behavior detection device and degrading device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the above-mentioned autonomous driving system abnormal behavior detection methods and / or autonomous driving system degrading methods.

[0080] The electronic devices in this application embodiment exist in various forms, including but not limited to: (1) Mobile communication devices: These devices are characterized by having mobile communication functions and are primarily designed to provide voice and data communication. (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, have computing and processing functions, and generally also have mobile internet access capabilities. (3) Portable entertainment devices: These devices can display and play multimedia content. (4) Servers: Devices that provide computing services. Servers consist of processors, hard disks, memory, system buses, etc. Servers are similar to general computer architectures, but because they need to provide highly reliable services, they have higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability. (5) Other electronic devices with data interaction functions.

[0081] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for detecting abnormal behavior in an autonomous driving system, comprising: Detect scene information and output scene classification labels based on the scene information, wherein the scene information includes road information, traffic participant information, infrastructure information and / or temporary facility information, and the scene classification labels are related to vehicle speed, road information, and traffic participant information; A scene association threshold is determined based on the scene classification label, wherein the scene association threshold is negatively correlated with the severity level of an accident occurring in the scene corresponding to the scene classification label; Based on the scene information and the scene association threshold, it is determined whether the vehicle exhibits abnormal behavior.

2. The method according to claim 1, characterized in that, The scene classification labels include forward hazard scene_vehicle, forward hazard scene_obstacle, forward hazard scene_vulnerable road user, lateral hazard scene_vehicle, or lateral hazard scene_vulnerable road user. The abnormal behavior includes unexpected acceleration. Determining whether the vehicle exhibits abnormal behavior based on the scene information and the scene association threshold includes: In the aforementioned forward hazard scenario_vehicle, if the speed of the vehicle in front is less than the speed of the vehicle itself, and the planned acceleration of the vehicle itself is greater than a first acceleration threshold associated with the forward hazard scenario_vehicle, then it is determined that the vehicle itself is experiencing unexpected acceleration; or, In the aforementioned forward hazard scenario_obstacle, if the planned acceleration of the vehicle is greater than a second acceleration threshold associated with the forward hazard scenario_obstacle, then it is determined that the vehicle is experiencing unexpected acceleration; or, In the aforementioned forward hazard scenario - vulnerable road user, if the planned acceleration of the vehicle exceeds a third acceleration threshold associated with the aforementioned forward hazard scenario - vulnerable road user, then it is determined that the vehicle is experiencing unintended acceleration; or, In the lateral hazard scenario_vehicle, if the speed of the vehicle in front is less than the speed of the vehicle itself and it cuts into the lane where the vehicle is located, and the planned acceleration of the vehicle itself is greater than the fourth acceleration threshold associated with the lateral hazard scenario_vehicle, then it is determined that the vehicle itself has an unexpected acceleration; or, In the lateral hazard scenario - vulnerable traffic participant, if the planned acceleration of the vehicle is greater than the fifth acceleration threshold associated with the lateral hazard scenario - vulnerable traffic participant, then it is determined that the vehicle has an unexpected acceleration.

3. The method according to claim 2, characterized in that, The settings of each acceleration threshold are negatively correlated with the scenario risk level.

4. The method according to claim 1, characterized in that, The scene classification label includes rear-approaching vehicle scenarios, the abnormal behavior includes unexpected deceleration, and the step of determining whether the vehicle exhibits abnormal behavior based on the scene information and the scene association threshold includes: In the scenario of oncoming traffic but not a forward hazard scenario, if the planned deceleration of the vehicle is greater than the deceleration threshold associated with the scenario of oncoming traffic, it is determined that the vehicle has an unexpected deceleration. The forward hazard scenario includes forward hazard scenario_vehicle, forward hazard scenario_obstacle and / or forward hazard scenario_vulnerable road user.

5. The method according to claim 1, characterized in that, The scenario classification label includes scenarios with conflicting vehicles in adjacent lanes, the abnormal behavior includes unexpected lane changes, and the step of determining whether the vehicle exhibits abnormal behavior based on the scenario information and the scenario association threshold includes: In the scenario of conflicting vehicles in the adjacent lane, if the vehicle plans to change lanes and the planned lane change preparation time is less than the preset lane change time threshold, it is determined that the vehicle has an unexpected lane change.

6. The method according to claim 1, characterized in that, The scene classification labels include slope scenes, curve scenes, medium-speed scenes, and / or high-speed scenes; the abnormal behavior includes unexpected vehicle non-control; and the determination of whether the vehicle exhibits abnormal behavior based on the scene information and the scene association threshold includes: In the slope scenario, the curve scenario, the medium speed scenario, and / or the high speed scenario, if the vehicle does not issue a vehicle control request, it is determined that the vehicle has an unexpected failure to control itself.

7. The method according to claim 1, characterized in that, The scene classification label includes curve scenes, the abnormal behavior includes unexpected lateral deviation, and the step of determining whether the vehicle has abnormal behavior based on the scene information and the scene association threshold includes: In the curved scenario, if the difference between the curvature of the trajectory planned by the vehicle and the curvature of the road is greater than a preset curvature difference threshold, it is determined that the vehicle has an unexpected lateral deviation.

8. The method according to any one of claims 1-7, characterized in that, The scene information includes vehicle status information, road information, and / or perception result information. The perception result information includes information on dynamic and static targets around the vehicle. The scene classification label output based on the scene information includes: Based on the vehicle's status information, determine whether the vehicle speed is within a preset speed range, wherein the preset speed range includes medium speed and high speed; if the vehicle speed is within the preset speed range, output a corresponding scene classification label based on the vehicle's speed range; and / or, Based on the road information, it is determined whether to output scene classification labels related to the road information. Specifically, when the slope is greater than a preset slope threshold, scene classification labels related to the slope are output; and / or, when the vehicle is traveling at a constant speed along the tangent of the current lane centerline and the time taken to cross the road boundary is less than a preset time threshold, scene classification labels related to the road curvature are output; and / or, The system iterates through the dynamic and static target information around the vehicle, and filters out a first target located in front of the vehicle in the same lane based on this information. The first target includes categories such as vehicles, obstacles, and vulnerable road users. When the first target exists and the distance between the vehicle and the first target is less than a first preset distance threshold, a scene classification label related to the forward hazard scenario and the category of the first target is output; and / or, The system iterates through the dynamic and static target information around the vehicle, and based on this information, filters out a second target that is in an adjacent lane and is cutting into the vehicle's lane. The second target includes vehicles and vulnerable road users. When the second target exists and the distance between the vehicle and the second target is less than a second preset distance threshold, a scene classification label related to the lateral hazard scenario and the category of the second target is output; and / or, Iterate through the dynamic and static target information around the vehicle, and filter out vehicles behind the vehicle based on the target information. If there is a target vehicle behind the vehicle and the speed of the target vehicle is greater than or equal to the speed of the vehicle, output a scene classification label related to the following vehicle; and / or, Iterate through the dynamic and static target information around the vehicle, and filter out vehicles in adjacent lanes based on the target information; when the speed of a vehicle behind the vehicle in the adjacent lane exceeds a first preset percentage of the vehicle's speed or the speed of a vehicle in front of the vehicle in the adjacent lane is lower than a second preset percentage of the vehicle's speed, output a scene classification label related to the conflicting vehicle in the adjacent lane; and / or, If none of the above scenarios exist, output the scenario category labels related to the regular scenarios.

9. A method for downgrading an autonomous driving system, comprising: The method according to any one of claims 1-8 detects abnormal behavior of the vehicle, and / or detects vehicle malfunction, and outputs the duration of the abnormality, wherein the duration of the abnormality includes the duration of the abnormal behavior and / or the duration of the vehicle malfunction. Predict the collision time between the vehicle and the target object based at least on the scene information; The driver takeover time is determined based on the collision time and the duration of the anomaly. Whether to trigger a downgrade is determined based on the driver takeover time and the duration of the anomaly.

10. The method according to claim 9, characterized in that, The vehicle malfunctions include message loss, abnormal message data exceeding limits, message lag, and / or message data jumps. The detection of vehicle malfunctions includes: If no upstream message is received within the first time threshold, it is determined that the vehicle has experienced message loss. The upstream messages include perception messages, decision-making and planning messages, and / or control execution messages; and / or... If the data parameter values ​​in the upstream message exceed the corresponding preset threshold values, it is determined that the vehicle has an abnormal data overrun; and / or, If the upstream messages remain unchanged except for the timestamp, then it is determined that the vehicle is experiencing message lag; and / or, If the difference between the values ​​of two consecutive frames of data parameters in the upstream message exceeds the corresponding physical limit, it is determined that there is a message data jump in the vehicle.

11. The method according to claim 9, characterized in that, After determining whether a downgrade is triggered, the method further includes: If a downgrade is triggered, the driver will be notified to take over immediately. If the driver fails to take over within the first time period, the driver shall actively slow down and turn on the hazard warning lights within the second time period. If the driver does not take over within the second time frame, the vehicle speed will be gradually reduced to 0, and the automatic driving mode will be discontinued, switching to manual driving.

12. The method according to claim 11, characterized in that, The method further includes: During active deceleration, the longitudinal deceleration of the vehicle is kept less than a preset deceleration threshold, and the lateral offset of the vehicle is kept less than a preset offset threshold.

13. The method according to claim 9, characterized in that, The process of determining whether to trigger a downgrade based on the driver takeover time and the duration of the anomaly includes: If the driver takeover time is less than the takeover time threshold, or the anomaly has lasted for a duration greater than the maximum duration threshold, then a downgrade is triggered. If the driver takeover time is greater than or equal to the takeover time threshold, and the duration of the anomaly is less than or equal to the maximum duration threshold, then a downgrade will not be triggered.

14. The method according to claim 13, characterized in that, The takeover time threshold and / or the maximum duration threshold are related to the scene classification label and are negatively correlated with the severity level of the accident after the scene corresponding to the scene classification label.

15. The method according to claim 9, characterized in that, The prediction of the collision time between the vehicle and the target object based at least on the scene information includes: Based on the scene information, the future trajectory information of the target object is predicted using a kinematic model or the prediction module of an autonomous driving system. Based on the future trajectory information of the target object, combined with the trajectory information of the vehicle, the collision time between the vehicle and the target object is calculated. The trajectory information of the vehicle uses the planned trajectory of the vehicle. When the planned trajectory of the vehicle is not updated in time, the trajectory information predicted by the kinematic model is used.

16. The method according to claim 15, characterized in that, The calculation of the collision time between the vehicle and the target object based on the target object's future trajectory information and the vehicle's trajectory information includes: The timestamps of the starting point of the vehicle's trajectory information and the starting point of the target object's future trajectory information are aligned; Calculate the lateral and longitudinal distances between trajectory points in the vehicle's trajectory information and trajectory points in the future trajectory information of the target object at the same moment; If at any given moment the lateral distance between two trajectory points is less than a lateral threshold or the longitudinal distance between two trajectory points is less than a longitudinal threshold, then it is determined that the vehicle has collided with the target object. The collision time is obtained by subtracting the timestamp of the starting point from the timestamp of the trajectory point at that moment. If at any given moment the lateral distance between two trajectory points is greater than or equal to the lateral threshold and the longitudinal distance between two trajectory points is greater than or equal to the longitudinal threshold, then the default time is determined as the collision time.

17. An electronic device comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1 to 16.

18. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 16.

19. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1 to 16.

20. A portable platform comprising the electronic device as claimed in claim 17.

Images