Autonomous driving system, method and storage medium based on enhanced verification

Abstract

The present application provides an autonomous driving system, method and storage medium based on enhanced verification, which relates to the field of autonomous driving technology. The system includes: an identification module for probabilistically fusing the recognition results of multiple network models to obtain visual perception results; a perception module for generating sensor perception results, and performing enhanced verification on the sensor perception results and visual perception results to obtain perception verification results, and fusing the predicted state corresponding to the perception verification results as an observation quantity to obtain a target perception result; a decision module for determining the corresponding comprehensive risk assessment result based on the target perception result, and determining the corresponding autonomous driving decision based on the comprehensive risk assessment result. In this way, the problems of existing autonomous driving systems such as difficulty in ensuring target recognition and perception accuracy, and large deviation between decisions on driving risks and the driver's psychological expectations can be improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to an autonomous driving system, method, and storage medium based on enhanced verification. Background Art

[0002] The field of intelligent driving faces two challenges. First, environmental factors are intertwined (including traffic participants, obstacles, weather factors, etc.), resulting in high levels of data noise caused by environmental interference. Second, driving scenarios are dynamic and random, and sensors with limited performance often fail to recognize edge cases. The constraints of economical computing platforms make it impossible to apply complex perception and decision-making models. This leads to two issues: poor perception reliability and low confidence in decision-making. Improving perception reliability and confidence in decision-making in complex and dynamic scenarios is a pressing issue for the intelligent driving industry.

[0003] In the field of object recognition, the current mainstream industry approach is to invoke different perception models based on the scenario. This is essentially single-mode perception, and the perception results are generally processed using post-processing algorithms based on expert experience. This makes it difficult to effectively guarantee object recognition accuracy, recall, and latency. In the perception field, the current mainstream approach is multi-source sensor fusion, which associates and fuses objects based on the perception results of different sensor types. Fusion essentially aims to reduce perception noise, but it cannot effectively guarantee the real-time and accuracy of perception results. In the decision-making field, a single, passive decision-making strategy is generally used in limited scenarios. This means that a decision is made only when a certain scenario is triggered and risk detection accumulates to a certain threshold. This type of decision-making is generally difficult to meet human expectations. In complex traffic scenarios, the autonomous driving experience is particularly poor, and decision-making strategies struggle to meet driver expectations, resulting in high takeover rates and low user adoption. Based on these traditional object recognition, perception, and decision-making algorithms, it is difficult to effectively ensure the safety and reliability of intelligent driving systems. Summary of the Invention

[0004] In view of this, the purpose of the embodiments of the present application is to provide an autonomous driving system, method and storage medium based on enhanced verification, which can improve the problems of existing autonomous driving systems such as difficulty in ensuring target recognition and perception accuracy, and large deviation between decisions on driving risks and the driver's psychological expectations.

[0005] To achieve the above technical objectives, the technical solutions adopted in this application are as follows:

[0006] In a first aspect, an embodiment of the present application provides an autonomous driving system based on enhanced verification, the system comprising:

[0007] A recognition module, configured to perform probabilistic fusion of the recognition results of multiple network models to obtain a visual perception result, wherein the recognition result is a result obtained by visually perceiving the target to be recognized by the corresponding network model;

[0008] A perception module is configured to generate a sensor perception result, perform enhanced verification on the sensor perception result and the visual perception result to obtain a perception verification result, and fuse the predicted state corresponding to the perception verification result as an observation to obtain a target perception result, wherein the sensor perception result is a result obtained by the vehicle-mounted sensor identifying the target and fusing it with the perception module;

[0009] A decision module is used to determine a corresponding comprehensive risk assessment result based on the target perception result, and to determine a corresponding autonomous driving decision based on the comprehensive risk assessment result, wherein the autonomous driving decision includes at least one of normal cruising, braking according to the risk level, and braking according to the quantified value of the objective risk.

[0010] In conjunction with the first aspect, in some optional embodiments, the recognition module includes a target recognition unit and a post-processing unit;

[0011] The target recognition unit is used to perform visual perception on the target to be recognized through multiple network models to obtain the recognition result, wherein the target recognition unit includes an optical flow model, a backbone large network model and a backbone small network model, the backbone large network model is an integrated end-to-end BEV perception model, and the backbone small network model is a cascaded multi-task network model;

[0012] The post-processing unit is used to perform probability fusion based on particle filtering on the recognition results to obtain the visual perception results.

[0013] In conjunction with the first aspect, in some optional implementations, the perception module includes a multi-source fusion unit, a semantic enhancement unit, and a scenario prediction unit;

[0014] The multi-source fusion unit is used to fuse the results obtained after the vehicle-mounted sensor recognizes the target to serve as the sensor perception result;

[0015] Alternatively, the multi-source fusion unit is used to use the predicted state as an observation quantity and perform fusion processing to obtain the target perception result;

[0016] The semantic enhancement unit is used to perform enhanced verification on the sensor perception result and the visual perception result to obtain a perception verification result;

[0017] The scenario prediction unit is used to determine the predicted state corresponding to the target based on the perception verification result, wherein the scenario prediction unit includes a trajectory prediction subunit based on the long time domain and an intention prediction subunit based on the short time domain. The trajectory prediction subunit is used to predict the motion trajectory of the target based on the perception verification result, and the intention prediction subunit is used to calculate the target lane change intention and the target intrusion intention.

[0018] In conjunction with the first aspect, in some optional implementations, the decision module includes an objective risk assessment unit, a subjective risk assessment unit, and a decision unit;

[0019] The objective risk assessment unit is used to determine a first assessment result and a second assessment result based on the target identification result, and perform enhanced verification on the first assessment result and the second assessment result to obtain an objective risk quantitative assessment result, wherein the objective risk assessment unit includes a first risk assessment subunit based on field theory and a second risk assessment subunit based on a social force model, the first assessment result is obtained by the first risk assessment subunit performing risk assessment on the target identification result, and the second assessment result is obtained by the second risk assessment subunit performing risk assessment on the target identification result;

[0020] The subjective risk assessment unit is used to map corresponding risk behaviors from a preset risk scenario set based on the longitudinal behavior and lateral behavior of the driver of the vehicle, so as to obtain a subjective risk quantitative assessment result;

[0021] The decision-making unit is used to combine the objective risk quantification assessment result and the subjective risk quantification assessment result to obtain the comprehensive risk assessment result that characterizes the objective environment and the driver's subjective psychological risk, and determine the corresponding autonomous driving decision based on the comprehensive risk assessment result.

[0022] In a second aspect, an embodiment of the present application provides an autonomous driving method based on enhanced verification, which is applied to the above-mentioned system, and the method includes:

[0023] Acquiring, through the target recognition unit in the recognition module, first target state information obtained after the plurality of network models visually perceive the target to be recognized;

[0024] Probabilistically fusing the first target state information through a post-processing unit in the recognition module to obtain a visual perception result;

[0025] The multi-source fusion unit in the perception module fuses the second target state information obtained after the vehicle-mounted sensor recognizes the target to obtain the sensor perception result;

[0026] Performing enhanced verification on the sensor perception result and the visual perception result through the semantic enhancement unit in the perception module to obtain a perception verification result;

[0027] According to the perception verification result, determining, by a scenario prediction unit in the perception module, a prediction state corresponding to the perception verification result;

[0028] The predicted state is used as an observation quantity for fusion processing by the multi-source fusion unit to obtain a target perception result;

[0029] According to the target perception result, a comprehensive risk assessment result corresponding to the target perception result is determined by a decision module;

[0030] Based on the comprehensive risk assessment result, the autonomous driving decision corresponding to the comprehensive risk assessment result is determined by the decision module, wherein the autonomous driving decision includes at least one of normal cruising, braking according to the risk level, and braking according to the quantified value of the objective risk.

[0031] In conjunction with the second aspect, in some optional implementations, performing enhanced verification on the sensor perception result and the visual perception result by a semantic enhancement unit in the perception module to obtain a perception verification result includes:

[0032] By means of the semantic enhancement unit, a position state quantity representing the lateral position of the target in the sensor perception result and the visual perception result is used as a first verification parameter for enhanced verification to obtain a steady-state intrusion amount of the target;

[0033] The semantic enhancement unit performs enhanced verification on a speed state quantity representing the lateral speed of the target in the sensor perception result and the visual perception result as a second verification parameter to obtain a stable lateral speed of the target;

[0034] According to the steady-state intrusion amount and the steady lateral speed, the dangerous state of the target is determined by the semantic enhancement unit, and the dangerous state is used as the perception verification result.

[0035] In conjunction with the second aspect, in some optional implementations, the semantic enhancement unit uses the position state quantity representing the lateral position of the target in the sensor perception result and the visual perception result as a first verification parameter for enhanced verification to obtain the steady-state intrusion amount of the target, including:

[0036] Setting a position state quantity in the first verification parameter as a default steady-state intrusion quantity;

[0037] When the distance between the target and the vehicle is within a first preset distance range, sequentially verifying the default steady-state intrusion amount and the remaining position state amounts in the first verification parameter except the default steady-state intrusion amount;

[0038] When the difference between the default steady-state intrusion amount and the remaining position state amount is within a preset first threshold range, and the default steady-state intrusion amount and the remaining position state amount have the same preset sign, the default steady-state intrusion amount is still output as the steady-state intrusion amount;

[0039] When the difference between the default steady-state intrusion amount and the remaining position state amount exceeds the first threshold range, and the default steady-state intrusion amount and the remaining position state amount have the same preset sign, calculating an average value of the default steady-state intrusion amount and the remaining position state amount as the steady-state intrusion amount output;

[0040] When the default steady-state intrusion amount and the preset signs of the remaining position state amount are different, the default steady-state intrusion amount is verified with the next remaining position state amount until all the remaining position state amounts are verified with the default state intrusion amount.

[0041] In conjunction with the second aspect, in some optional implementations, the semantic enhancement unit uses the velocity state quantity representing the lateral velocity of the target in the sensor perception result and the visual perception result as a second verification parameter for enhanced verification to obtain the stable lateral velocity of the target, including:

[0042] Determine the corresponding lateral speed according to the speed state quantity, wherein the lateral speed V1 is calculated according to the rate of change of the distance between the target and the lane line in the speed state quantity; the lateral speed V2 of the target is calculated according to the rate of change of the distance between the target and the vehicle's driving trajectory in the speed state quantity; the lateral speed V3 of the target is calculated according to the rate of change of the visual line pressure of the target in the speed state quantity; the lateral speed V4 of the target is calculated according to the rate of change of the lateral distance in the speed state quantity; and call the lateral speed V0 of the target output by the fusion in the speed state quantity;

[0043] Set any lateral speed among V0, V1, V2, V3, and V4 as the default stable lateral speed;

[0044] When the longitudinal distance between the target and the vehicle is within a second preset distance range in the sensor perception result, the default stable lateral speed is ignored and the stable lateral speed is determined to be 1 / 3*(V1+V2+V3);

[0045] When the longitudinal distance between the target and the vehicle is within a second preset distance range and the target is a preset vehicle type, the default stable lateral speed is ignored and the stable lateral speed is determined to be 1 / 4*(V1+V2+V3+V4);

[0046] When the sensor perception result shows that the longitudinal distance between the target and the vehicle is within a third preset distance range, the default stable lateral speed is ignored and the stable lateral speed is determined to be 1 / 2*(V1+V2);

[0047] When the longitudinal distance between the target and the vehicle is within a third preset distance range and the target is a preset vehicle type, the default stable lateral speed is ignored and the stable lateral speed is determined to be 1 / 3*(V1+V3).

[0048] In conjunction with the second aspect, in some optional implementations, determining, by a decision module, a comprehensive risk assessment result corresponding to the target perception result based on the target perception result includes:

[0049] According to the target perception result, a first assessment result and a second assessment result are determined by an objective risk assessment unit in a decision module, wherein the objective risk assessment unit includes a first risk assessment subunit based on field theory and a second risk assessment subunit based on a social force model, the first assessment result is obtained by the first risk assessment subunit performing a risk assessment on the target identification result, and the second assessment result is obtained by the second risk assessment subunit performing a risk assessment on the target identification result;

[0050] Performing enhanced verification on the first assessment result and the second assessment result by the objective risk assessment unit to obtain an objective risk quantitative assessment result;

[0051] Through the subjective risk assessment unit in the decision-making module, corresponding risk behaviors are mapped from the preset risk scenarios to obtain the subjective risk quantitative assessment results;

[0052] The objective risk quantification evaluation result and the subjective risk quantification evaluation result are combined by the decision unit in the decision module to obtain the comprehensive risk evaluation result that characterizes the objective environment and the driver's subjective psychological risk.

[0053] In conjunction with the second aspect, in some optional implementations, the objective risk assessment unit performs enhanced verification on the first assessment result and the second assessment result to obtain an objective risk quantitative assessment result, including:

[0054] When, in the target recognition result, the longitudinal distance between the target and the vehicle is within a fourth preset distance range, the first assessment result is used for the first preset target, and the second assessment result is used for the second preset target, as the objective risk quantitative assessment result;

[0055] When, in the target recognition result, the longitudinal distance between the target and the vehicle exceeds the fourth preset distance range, and the difference between the first evaluation result and the second evaluation result is within the second threshold range, the root mean square value of the first evaluation result and the second evaluation result is calculated, and the first preset target and the second preset target both use the root mean square value as the objective risk quantification evaluation result.

[0056] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is run on a computer, the computer executes the above method.

[0057] The invention adopting the above technical solution has the following advantages:

[0058] In the technical solution provided by this application, the recognition module probabilistically fuses the recognition results of multiple network models to obtain visual perception results; the perception module generates sensor perception results, and performs enhanced verification on the sensor perception results and visual perception results to obtain perception verification results. The predicted state corresponding to the perception verification results is fused as an observation quantity to obtain a target perception result; the decision module determines the corresponding comprehensive risk assessment result based on the target perception result, and determines the corresponding autonomous driving decision based on the comprehensive risk assessment result. In this way, through enhanced verification, more reliable observation quantities are added to the sensor fusion process, and the objective risks of the scene and the subjective psychological risks of the driver are combined in the risk assessment and decision-making stages, thereby improving the existing autonomous driving system. The problem of difficult to ensure the accuracy of target recognition and perception, and the large deviation between the decision on driving risk and the driver's psychological expectations. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The present application may be further illustrated by the non-limiting embodiments provided in the accompanying drawings. It should be understood that the following drawings illustrate only certain embodiments of the present application and are therefore not to be construed as limiting the scope of the present application. It is understood that a person skilled in the art can derive other relevant drawings from these drawings without inventive effort.

[0060] Figure 1 A schematic diagram of the structure of an autonomous driving system based on enhanced verification provided in an embodiment of the present application.

[0061] Figure 2A flowchart of the autonomous driving method based on enhanced verification provided in an embodiment of the present application.

[0062] Figure 3 Schematic diagram of the identification module provided in an embodiment of the present application.

[0063] Figure 4 Schematic diagram of the perception module provided in an embodiment of the present application.

[0064] Figure 5 A schematic diagram of the enhanced verification process in the perception module provided in an embodiment of the present application.

[0065] Figure 6 A schematic diagram of the process flow of autonomous driving decision-making provided in an embodiment of the present application.

[0066] Icon: 100-Automatic driving system based on enhanced verification; 101-Recognition module; 102-Perception module; 103-Decision module. DETAILED DESCRIPTION

[0067] The present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that similar or identical parts in the drawings or descriptions are numbered the same. Implementations not shown or described in the drawings are known to those of ordinary skill in the art. In the description of this application, the terms "first," "second," etc. are used solely to distinguish descriptions and are not to be construed as indicating or implying relative importance.

[0068] Please refer to Figure 1 , an embodiment of the present application provides an autonomous driving system 100 based on enhanced verification, the system including an identification module 101, a perception module 102 and a decision module 103.

[0069] The recognition module 101 is used to perform probabilistic fusion on the recognition results of multiple network models to obtain a visual perception result, wherein the recognition result is a result obtained by the corresponding network model performing visual perception on the target to be recognized.

[0070] In this embodiment, the visual perception result can be understood as the relative displacement between the target and the vehicle, such as the target's visual line-of-sight or lateral speed. The target can be the object perceived by the onboard sensors deployed on the vehicle, such as pedestrians, vehicles, and obstacles. The onboard sensors can be ultrasonic radars, cameras, and the like.

[0071] The perception module 102 is used to generate sensor perception results, and perform enhanced verification on the sensor perception results and the visual perception results to obtain perception verification results, and fuse the predicted states corresponding to the perception verification results as observation quantities to obtain target perception results, wherein the sensor perception results are the results obtained after the vehicle-mounted sensors identify the target to be identified and are fused by the perception module 102.

[0072] In this embodiment, the sensor perception result can be understood as the result of the multiple onboard sensors deployed on the vehicle sensing the target's motion and position. Examples include the distance between the target and the vehicle's driving trajectory, the distance between the target and the lane line, and the lateral speed of the target. The driving trajectory can be the vehicle's preset automatic driving route during autonomous driving.

[0073] Enhanced verification can be understood as a rule for selecting state quantities (such as speed and position information). This involves using state quantities from different sources, converting them into representations of the same physical quantity using kinematic formulas, and then comparing them. The state quantity with the highest confidence in each scenario is selected as the enhanced verification result. The specific implementation of enhanced verification for sensor and visual perception results is described below and will not be further elaborated here.

[0074] The decision module 103 is used to determine a corresponding comprehensive risk assessment result based on the target perception result, and to determine a corresponding autonomous driving decision based on the comprehensive risk assessment result, wherein the autonomous driving decision includes at least one of normal cruising, braking according to the risk level, and braking according to the quantified value of the objective risk.

[0075] In this embodiment, the target perception results are used to characterize the relative position and relative velocity of the target and the vehicle. The comprehensive risk assessment results are used to characterize the risks of the objective environment (such as pedestrians, vehicles, or obstacles) and the driver's subjective psychology (such as the driver's acceleration and deceleration behavior or the driver's steering wheel operation).

[0076] In this embodiment, the comprehensive risk assessment results specifically include dividing the risk into three zones, centered on the vehicle's direction of travel: a deterministic risk zone, a potential risk zone, and a zero-risk zone. The risk zones are divided based on the relative distance and relative speed of the target to the vehicle.

[0077] The method for determining the results of the comprehensive analysis and assessment is as follows: Based on the target's steady-state intrusion volume and lateral speed, it is determined whether the target enters the potential risk zone or the deterministic risk zone within a preset time threshold (flexibly set according to actual application, such as 10 seconds or 20 seconds). If so, the target is identified as a potential risk target. Subsequently, based on the time threshold required for the target to reach the deterministic risk zone and the risk assessment results of the objective environment, the risk level and quantitative value are determined. The specific definition of steady-state intrusion volume is mentioned below and is not repeated here.

[0078] The method for determining autonomous driving decisions is as follows: when the target is in a risk-free zone, the vehicle cruises normally; when the target enters a potential risk zone, different levels of deceleration actions are performed in advance according to the level of risk; if the target enters a determined risk zone, different levels of deceleration actions are performed according to the risk quantification value of the objective environment.

[0079] As an optional implementation, please refer to Figure 3 , the recognition module 101 may include a target recognition unit and a post-processing unit;

[0080] The target recognition unit is used to perform visual perception on the target to be recognized through multiple network models to obtain the recognition result, wherein the target recognition unit includes an optical flow model, a backbone large network model and a backbone small network model, the backbone large network model is an integrated end-to-end BEV perception model, and the backbone small network model is a cascaded multi-task network model;

[0081] The post-processing unit is used to perform probability fusion based on particle filtering on the recognition results to obtain the visual perception results.

[0082] As can be understood, the post-processing unit probabilistically fuses the recognition results of the optical flow model, the backbone large network model, and the backbone small network model to obtain the visual perception result. In this way, the advantages of the optical flow model's strong real-time performance, the integrated end-to-end BEV perception model's high accuracy, and the cascaded multi-task network model's fast convergence speed can be combined to dynamically adjust the fused feature parameters, allowing probabilistic fusion to balance real-time and accurate target perception in highly dynamic scenes.

[0083] As an optional implementation, please refer to Figure 4-Figure 5 , the perception module 102 includes a multi-source fusion unit, a semantic enhancement unit and a scenario prediction unit;

[0084] The multi-source fusion unit is used to fuse the results obtained after the vehicle-mounted sensor recognizes the target to serve as the sensor perception result;

[0085] Alternatively, the multi-source fusion unit is used to use the predicted state as an observation quantity and perform fusion processing to obtain the target perception result;

[0086] The semantic enhancement unit is used to perform enhanced verification on the sensor perception result and the visual perception result to obtain a perception verification result;

[0087] The scenario prediction unit is used to determine the predicted state corresponding to the target based on the perception verification result, wherein the scenario prediction unit includes a trajectory prediction subunit based on the long time domain and an intention prediction subunit based on the short time domain. The trajectory prediction subunit is used to predict the motion trajectory of the target based on the perception verification result, and the intention prediction subunit is used to calculate the target lane change intention and the target intrusion intention.

[0088] In this embodiment, the trajectory prediction subunit may specifically include a map prediction model, a perception prediction model, and a physical prediction model, and the prediction results of the three are combined to enable the trajectory prediction subunit to more accurately predict the motion trajectory of the target.

[0089] In this embodiment, the multi-source fusion unit includes a data association subunit based on the principle of topological similarity and a fusion subunit based on graph optimization theory. Among them, the data association subunit and the fusion subunit are conventional components in sensor fusion, and the functions or structures of the data association subunit and the fusion subunit are not repeated here.

[0090] It can be understood that the target-related parameters (such as the distance between the target and the vehicle's driving trajectory, the distance between the target and the lane line, the lateral speed of the target, etc.) obtained after the on-board sensor perceives the target are obtained through the multi-source fusion unit, and the target-related parameters are fused to obtain the sensor perception result. Subsequently, the semantic enhancement unit performs enhanced verification on the sensor perception result and the pre-obtained visual perception result to obtain the perception verification result. The scenario prediction unit predicts the target's motion trajectory, the target's lane changing intention, and the target's intrusion intention based on the perception verification result, and obtains the predicted state of the target by combining the target's motion trajectory, lane changing intention, and intrusion intention. Finally, the predicted state of the target is fed back to the multi-source fusion unit to be fused as the fusion observation quantity of the multi-source fusion unit to improve the accuracy of sensor fusion.

[0091] As an optional implementation, please refer to Figure 6 , the decision module 103 includes an objective risk assessment unit, a subjective risk assessment unit and a decision unit;

[0092] The objective risk assessment unit is used to determine a first assessment result and a second assessment result based on the target identification result, and perform enhanced verification on the first assessment result and the second assessment result to obtain an objective risk quantitative assessment result, wherein the objective risk assessment unit includes a first risk assessment subunit based on field theory and a second risk assessment subunit based on a social force model, the first assessment result is obtained by the first risk assessment subunit performing risk assessment on the target identification result, and the second assessment result is obtained by the second risk assessment subunit performing risk assessment on the target identification result;

[0093] The subjective risk assessment unit is used to map corresponding risk behaviors from a preset risk scenario set based on the longitudinal behavior and lateral behavior of the driver of the vehicle, so as to obtain a subjective risk quantitative assessment result;

[0094] The decision-making unit is used to combine the objective risk quantification assessment result and the subjective risk quantification assessment result to obtain the comprehensive risk assessment result that characterizes the objective environment and the driver's subjective psychological risk, and determine the corresponding autonomous driving decision based on the comprehensive risk assessment result.

[0095] In this embodiment, risk assessments based on both field theory and social forces are applicable to traffic participants such as vehicles and pedestrians. Field theory-based risk assessments have significant advantages in describing vehicle risk fields, but their performance in describing pedestrian risk fields is relatively limited. Therefore, a second risk assessment subunit based on social forces is introduced into the objective risk assessment unit. This second risk assessment subunit has better performance in describing pedestrian risk fields and complements the first risk assessment subunit based on field theory. By means of enhanced verification, appropriate assessment subunits are selected for vehicles and pedestrians, thereby making the objective risk assessment of the target more accurate. For details on the enhanced verification methods for the first and second risk assessment subunits, please refer to the following text and will not be elaborated on here.

[0096] In this embodiment, the subjective risk assessment unit includes a longitudinal behavior mapping subunit based on a Gaussian mixture model and a lateral behavior mapping subunit based on a hybrid logic dynamic model. Lateral behavior refers to the driver's steering behavior, and longitudinal behavior refers to the driver's acceleration and deceleration behavior.

[0097] As can be understood, by combining the driver's subjective psychological risk with the objective environmental risk, a comprehensive risk assessment is generated that represents both the objective environmental risk and the driver's subjective psychological risk. Based on this comprehensive risk assessment, potential risks are predicted, and proactive defensive decisions are made based on the predicted risk. This ensures that autonomous driving decisions reflect both the objective danger level of the scenario and the subjective feelings of the driver and passengers.

[0098] Please refer to Figure 2The present application also provides an enhanced verification-based autonomous driving method, which is applied to the aforementioned enhanced verification-based autonomous driving system 100. The enhanced verification-based autonomous driving method may include the following steps:

[0099] Step 110: obtaining, through the target recognition unit in the recognition module 101, first target state information obtained after the plurality of network models visually perceive the target to be recognized;

[0100] Step 120: performing probabilistic fusion on the first target state information by a post-processing unit in the recognition module 101 to obtain a visual perception result;

[0101] Step 130 , fusing the second target state information obtained after the vehicle-mounted sensor recognizes the target through the multi-source fusion unit in the perception module 102 to obtain a sensor perception result;

[0102] Step 140: Performing enhanced verification on the sensor perception result and the visual perception result by the semantic enhancement unit in the perception module 102 to obtain a perception verification result;

[0103] Step 150: determining a predicted state corresponding to the perception verification result by the scenario prediction unit in the perception module 102 according to the perception verification result;

[0104] Step 160: fusing the predicted state as an observation quantity through the multi-source fusion unit to obtain a target perception result;

[0105] Step 170: Determine the comprehensive risk assessment result corresponding to the target perception result through the decision module 103 according to the target perception result;

[0106] Step 180: Based on the comprehensive risk assessment result, the decision module 103 determines the autonomous driving decision corresponding to the comprehensive risk assessment result, wherein the autonomous driving decision includes at least one of normal cruising, braking according to the risk level, and braking according to the quantified value of the objective risk.

[0107] As an optional implementation, performing enhanced verification on the sensor perception result and the visual perception result by the semantic enhancement unit in the perception module 102 to obtain a perception verification result may include:

[0108] By means of the semantic enhancement unit, a position state quantity representing the lateral position of the target in the sensor perception result and the visual perception result is used as a first verification parameter for enhanced verification to obtain a steady-state intrusion amount of the target;

[0109] The semantic enhancement unit performs enhanced verification on a speed state quantity representing the lateral speed of the target in the sensor perception result and the visual perception result as a second verification parameter to obtain a stable lateral speed of the target;

[0110] According to the steady-state intrusion amount and the steady lateral speed, the dangerous state of the target is determined by the semantic enhancement unit, and the dangerous state is used as the perception verification result.

[0111] In this embodiment, the driving trajectory of the vehicle is used as the reference line, and half the lane width (or an manually set empirical value as the threshold) is extended to both sides with the reference line as the center as the boundary line, and the distance by which the target boundary invades the boundary line is defined as the steady-state intrusion amount of the target.

[0112] In this embodiment, the position state quantity can be the distance between the target and the lane line, the distance between the target and the vehicle's driving trajectory, the target's visual line-crossing amount, the target's lateral distance, etc. The speed state quantity can be the lateral speed calculated based on the distance between the target and the lane line, the fused lateral speed, the lateral speed calculated based on the visual line-crossing amount, the lateral speed calculated based on the lateral distance, or the lateral speed calculated based on the distance between the target and the vehicle's driving trajectory, etc.

[0113] As an optional implementation, the semantic enhancement unit uses the position state quantity representing the lateral position of the target in the sensor perception result and the visual perception result as the first verification parameter for enhanced verification, which may include:

[0114] Setting a position state quantity in the first verification parameter as a default steady-state intrusion quantity;

[0115] When the distance between the target and the vehicle is within a first preset distance range, sequentially verifying the default steady-state intrusion amount and the remaining position state amounts in the first verification parameter except the default steady-state intrusion amount;

[0116] When the difference between the default steady-state intrusion amount and the remaining position state amount is within a preset first threshold range, and the default steady-state intrusion amount and the remaining position state amount have the same preset sign, the default steady-state intrusion amount is still output as the steady-state intrusion amount;

[0117] When the difference between the default steady-state intrusion amount and the remaining position state amount exceeds the first threshold range, and the default steady-state intrusion amount and the remaining position state amount have the same preset sign, calculating an average value of the default steady-state intrusion amount and the remaining position state amount as the steady-state intrusion amount output;

[0118] When the default steady-state intrusion amount and the preset signs of the remaining position state amount are different, the default steady-state intrusion amount is verified with the next remaining position state amount until all the remaining position state amounts are verified with the default state intrusion amount.

[0119] In this embodiment, the first preset distance range is used to represent the longitudinal distance between the target and the vehicle. The value of the first preset distance range can be flexibly set according to the actual application. For example, within 1 time headway, 3 times the time headway, etc. The time headway refers to the distance the vehicle travels after traveling continuously at the current speed and direction for a specified time period. For example, 1 time headway refers to the distance the vehicle travels after traveling at the current speed and direction for 1 second, and 6 times the time headway refers to the distance the vehicle travels after traveling at the current speed and direction for 6 seconds.

[0120] In this embodiment, the first threshold range is used to characterize the difference between the default steady-state intrusion amount and the verified position state amount, and can be flexibly set according to actual conditions, such as 1 meter, 3 meters, 5 meters, etc.

[0121] For example, the default steady-state intrusion amount of the target in a straight road scenario is the distance between the target and the lane line; the default steady-state intrusion amount of the target in a curve scenario is the distance between the target and the vehicle's driving trajectory;

[0122] When the distance between the target and the vehicle is within 1 time interval, the target steady-state intrusion amount is first verified with the target's visual line pressure amount based on the above-mentioned default steady-state intrusion amount. If the difference between the default steady-state intrusion amount and the target's visual line pressure amount is within the first threshold range and the preset signs are the same, the target's steady-state intrusion amount is still output according to the default value; if the difference between the default steady-state intrusion amount and the target's visual line pressure amount exceeds the first threshold range but has the same sign, the target's steady-state intrusion amount is taken as the average of the default steady-state intrusion amount and the target's visual line pressure amount; if the default steady-state intrusion amount and the target's visual line pressure amount have different signs, the target's lateral distance is introduced for verification. If the default steady-state intrusion amount is closer to the target's visual line pressure amount, the steady-state intrusion amount is still the default steady-state intrusion amount; otherwise, the steady-state intrusion amount is taken as the target's visual line pressure amount.

[0123] As an optional implementation, the semantic enhancement unit uses the velocity state quantity representing the target lateral velocity in the sensor perception result and the visual perception result as the second verification parameter for enhanced verification, which may include:

[0124] Determine the corresponding lateral speed according to the speed state quantity, wherein the lateral speed V1 is calculated according to the rate of change of the distance between the target and the lane line in the speed state quantity; the lateral speed V2 of the target is calculated according to the rate of change of the distance between the target and the vehicle's driving trajectory in the speed state quantity; the lateral speed V3 of the target is calculated according to the rate of change of the visual line pressure of the target in the speed state quantity; the lateral speed V4 of the target is calculated according to the rate of change of the lateral distance in the speed state quantity; and call the lateral speed V0 of the target output by the fusion in the speed state quantity;

[0125] Set any lateral speed among V0, V1, V2, V3, and V4 as the default stable lateral speed;

[0126] When the longitudinal distance between the target and the vehicle is within a second preset distance range in the sensor perception result, the default stable lateral speed is ignored and the stable lateral speed is determined to be 1 / 3*(V1+V2+V3);

[0127] When the longitudinal distance between the target and the vehicle is within a second preset distance range and the target is a preset vehicle type, the default stable lateral speed is ignored and the stable lateral speed is determined to be 1 / 4*(V1+V2+V3+V4);

[0128] When the sensor perception result shows that the longitudinal distance between the target and the vehicle is within a third preset distance range, the default stable lateral speed is ignored and the stable lateral speed is determined to be 1 / 2*(V1+V2);

[0129] When the longitudinal distance between the target and the vehicle is within a third preset distance range and the target is a preset vehicle type, the default stable lateral speed is ignored and the stable lateral speed is determined to be 1 / 3*(V1+V3).

[0130] In this embodiment, the second preset distance range and the third preset distance range are used to represent the longitudinal distance between the target and the vehicle, and their values can be flexibly set according to actual applications, such as within 1 time headway, more than 3 time headway, etc.

[0131] For example, the default stable lateral speed of a target on a straight road is V0; the default stable lateral speed of a target on a curve is V1;

[0132] When the longitudinal distance between the target and the vehicle is greater than 2 times the time interval, the default stable lateral speed is ignored, and the stable lateral speed is determined to be 1 / 3*(V1+V2+V3); when the longitudinal distance between the target and the vehicle is greater than 2 times the time interval and the target is a large vehicle, the default stable lateral speed is ignored, and the stable lateral speed is determined to be 1 / 4*(V1+V2+V3+V4); when the target is within 1-2 times the time interval, the default stable lateral speed is ignored, and the stable lateral speed is determined to be 1 / 2*(V1+V2); when the target is within 1-2 times the time interval and the target is a large vehicle, the default stable lateral speed is ignored, and the stable lateral speed is determined to be 1 / 3*(V1+V3).

[0133] As an optional implementation, according to the target perception result, determining the comprehensive risk assessment result corresponding to the target perception result by the decision module 103 may include:

[0134] According to the target perception result, a first assessment result and a second assessment result are determined by an objective risk assessment unit in the decision module 103, wherein the objective risk assessment unit includes a first risk assessment subunit based on field theory and a second risk assessment subunit based on a social force model, the first assessment result is obtained by the first risk assessment subunit performing a risk assessment on the target identification result, and the second assessment result is obtained by the second risk assessment subunit performing a risk assessment on the target identification result;

[0135] Performing enhanced verification on the first assessment result and the second assessment result by the objective risk assessment unit to obtain an objective risk quantitative assessment result;

[0136] Through the subjective risk assessment unit in the decision module 103, corresponding risk behaviors are mapped from the preset risk scenarios to obtain a subjective risk quantitative assessment result;

[0137] The decision unit in the decision module 103 combines the objective risk quantification assessment result and the subjective risk quantification assessment result to obtain the comprehensive risk assessment result that characterizes the objective environment and the driver's subjective psychological risk.

[0138] As an optional implementation manner, performing enhanced verification on the first assessment result and the second assessment result by the objective risk assessment unit to obtain an objective risk quantitative assessment result may include:

[0139] When, in the target recognition result, the longitudinal distance between the target and the vehicle is within a fourth preset distance range, the first assessment result is used for the first preset target, and the second assessment result is used for the second preset target, as the objective risk quantitative assessment result;

[0140] When, in the target recognition result, the longitudinal distance between the target and the vehicle exceeds the fourth preset distance range, and the difference between the first evaluation result and the second evaluation result is within the second threshold range, the root mean square value of the first evaluation result and the second evaluation result is calculated, and the first preset target and the second preset target both use the root mean square value as the objective risk quantification evaluation result.

[0141] In this embodiment, the fourth preset distance range can be flexibly set according to actual circumstances, such as within 1 time interval, more than 1 time interval, etc. The second threshold range can be understood as an empirical value obtained through risk assessment experiments, used to characterize the difference between the first assessment result and the second assessment result, and can be flexibly set according to actual circumstances. The first preset target and the second preset target can be flexibly set according to actual circumstances, such as vehicles, pedestrians, static obstacles, etc.

[0142] It should be noted that those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working process of the autonomous driving system 100 based on enhanced verification described above can refer to the corresponding processes of each step in the aforementioned method, and will not be elaborated on here.

[0143] The present application also provides a computer-readable storage medium that stores a computer program that, when executed on a computer, causes the computer to execute the autonomous driving method based on enhanced verification as described in the above embodiments.

[0144] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented through hardware or by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0145] In summary, the embodiments of the present application provide an autonomous driving system, method, and storage medium based on enhanced verification. In this solution, the recognition module 101 performs probabilistic fusion on the recognition results of multiple network models to obtain visual perception results; the perception module 102 generates sensor perception results, and performs enhanced verification on the sensor perception results and visual perception results to obtain perception verification results, and the predicted state corresponding to the perception verification result is fused as an observation quantity to obtain a target perception result; the decision module 103 determines the corresponding comprehensive risk assessment result based on the target perception result, and determines the corresponding autonomous driving decision based on the comprehensive risk assessment result. In this way, by means of enhanced verification, more reliable observation quantities are added to the sensor fusion process, and the objective risks of the scene and the subjective psychological risks of the driver are combined in the risk assessment and decision-making stages, thereby improving the existing autonomous driving system. The problem that the accuracy of target recognition and perception is difficult to guarantee, and the decision on driving risk deviates greatly from the psychological expectations of the driver.

[0146] In the embodiments provided in the present application, it should be understood that the disclosed devices, systems and methods can also be implemented in other ways. The device, system and method embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of code, and a part of the module, program segment or code includes one or more executable instructions for implementing the specified logical function. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. In addition, the functional modules in the various embodiments of the present application can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part.

[0147] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, various modifications and variations of the present application are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. An automatic driving system based on enhanced verification, characterized in that: The system comprises: A recognition module, configured to perform probabilistic fusion of the recognition results of multiple network models to obtain a visual perception result, wherein the recognition result is a result obtained by visually perceiving the target to be recognized by the corresponding network model; A perception module is configured to generate a sensor perception result, perform enhanced verification on the sensor perception result and the visual perception result to obtain a perception verification result, and fuse the predicted state corresponding to the perception verification result as an observation quantity to obtain a target perception result, wherein the sensor perception result is a result obtained by the on-board sensor identifying the target to be identified and fused by the perception module, and the enhanced verification representation compares and screens the state quantities representing the same physical quantity in the sensor perception result and the visual perception result; a decision module, configured to determine a corresponding comprehensive risk assessment result based on the target perception result, and determine a corresponding autonomous driving decision based on the comprehensive risk assessment result, wherein the autonomous driving decision includes at least one of normal cruising, braking based on a risk level, and braking based on a quantified value of the comprehensive risk; Wherein, the recognition module includes a target recognition unit and a post-processing unit; The target recognition unit is used to perform visual perception on the target to be recognized through multiple network models to obtain the recognition result, wherein the target recognition unit includes an optical flow model, a backbone large network model and a backbone small network model, the backbone large network model is an integrated end-to-end BEV perception model, and the backbone small network model is a cascaded multi-task network model; The post-processing unit is used to perform probability fusion based on particle filtering on the recognition results to obtain the visual perception results; The perception module includes a multi-source fusion unit, a semantic enhancement unit and a scenario prediction unit; The multi-source fusion unit is used to fuse the results obtained after the vehicle-mounted sensor recognizes the target to serve as the sensor perception result; Alternatively, the multi-source fusion unit is used to use the predicted state as an observation quantity and perform fusion processing to obtain the target perception result; The semantic enhancement unit is used to perform enhanced verification on the sensor perception result and the visual perception result to obtain a perception verification result; The scenario prediction unit is used to determine the predicted state corresponding to the target based on the perception verification result, wherein the scenario prediction unit includes a trajectory prediction subunit based on the long time domain and an intention prediction subunit based on the short time domain. The trajectory prediction subunit is used to predict the motion trajectory of the target based on the perception verification result, and the intention prediction subunit is used to calculate the target lane change intention and the target intrusion intention.

2. The system according to claim 1, wherein: The decision-making module includes an objective risk assessment unit, a subjective risk assessment unit and a decision-making unit; The objective risk assessment unit is used to determine a first assessment result and a second assessment result based on the target identification result, and perform enhanced verification on the first assessment result and the second assessment result to obtain an objective risk quantitative assessment result, wherein the objective risk assessment unit includes a first risk assessment subunit based on field theory and a second risk assessment subunit based on a social force model, the first assessment result is obtained by the first risk assessment subunit performing risk assessment on the target identification result, and the second assessment result is obtained by the second risk assessment subunit performing risk assessment on the target identification result; The subjective risk assessment unit is used to map corresponding risk behaviors from a preset risk scenario set based on the longitudinal behavior and lateral behavior of the driver of the vehicle, so as to obtain a subjective risk quantitative assessment result; The decision-making unit is used to combine the objective risk quantification assessment result and the subjective risk quantification assessment result to obtain the comprehensive risk assessment result that characterizes the objective environment and the driver's subjective psychological risk, and determine the corresponding autonomous driving decision based on the comprehensive risk assessment result.

3. An autonomous driving method based on enhanced verification, characterized in that: Applied to the system according to any one of claims 1-2, the method comprising: Acquiring, through the target recognition unit in the recognition module, first target state information obtained after the plurality of network models visually perceive the target to be recognized; Probabilistically fusing the first target state information through a post-processing unit in the recognition module to obtain a visual perception result; The multi-source fusion unit in the perception module fuses the second target state information obtained after the vehicle-mounted sensor recognizes the target to obtain the sensor perception result; Performing enhanced verification on the sensor perception result and the visual perception result through the semantic enhancement unit in the perception module to obtain a perception verification result; According to the perception verification result, determining, by a scenario prediction unit in the perception module, a prediction state corresponding to the perception verification result; The predicted state is used as an observation quantity for fusion processing by the multi-source fusion unit to obtain a target perception result; According to the target perception result, a comprehensive risk assessment result corresponding to the target perception result is determined by a decision module; Based on the comprehensive risk assessment result, the autonomous driving decision corresponding to the comprehensive risk assessment result is determined by the decision module, wherein the autonomous driving decision includes at least one of normal cruising, braking according to the risk level, and braking according to the quantified value of the comprehensive risk.

4. The method according to claim 3, characterized in that Performing enhanced verification on the sensor perception result and the visual perception result by a semantic enhancement unit in the perception module to obtain a perception verification result, including: By means of the semantic enhancement unit, a position state quantity representing the lateral position of the target in the sensor perception result and the visual perception result is used as a first verification parameter for enhanced verification to obtain a steady-state intrusion amount of the target; The semantic enhancement unit performs enhanced verification on a speed state quantity representing the lateral speed of the target in the sensor perception result and the visual perception result as a second verification parameter to obtain a stable lateral speed of the target; According to the steady-state intrusion amount and the steady lateral speed, the dangerous state of the target is determined by the semantic enhancement unit, and the dangerous state is used as the perception verification result.

5. The method according to claim 4, characterized in that The semantic enhancement unit performs enhanced verification on a position state quantity representing the lateral position of the target in the sensor perception result and the visual perception result as a first verification parameter to obtain a steady-state intrusion amount of the target, including: Setting a position state quantity in the first verification parameter as a default steady-state intrusion quantity; When the distance between the target and the vehicle is within a first preset distance range, sequentially verifying the default steady-state intrusion amount and the remaining position state amounts in the first verification parameter except the default steady-state intrusion amount; When the difference between the default steady-state intrusion amount and the remaining position state amount is within a preset first threshold range, and the default steady-state intrusion amount and the remaining position state amount have the same preset sign, the default steady-state intrusion amount is still output as the steady-state intrusion amount; When the difference between the default steady-state intrusion amount and the remaining position state amount exceeds the first threshold range, and the default steady-state intrusion amount and the remaining position state amount have the same preset sign, calculating an average value of the default steady-state intrusion amount and the remaining position state amount as the steady-state intrusion amount output; When the default steady-state intrusion amount and the preset signs of the remaining position state amount are different, the default steady-state intrusion amount is verified with the next remaining position state amount until all the remaining position state amounts are verified with the default steady-state intrusion amount.

6. The method according to claim 4, characterized in that The semantic enhancement unit performs enhanced verification on a speed state quantity representing the lateral speed of the target in the sensor perception result and the visual perception result as a second verification parameter to obtain a stable lateral speed of the target, including: Determine the corresponding lateral speed according to the speed state quantity, wherein the lateral speed V1 is calculated according to the rate of change of the distance between the target and the lane line in the speed state quantity; the lateral speed V2 of the target is calculated according to the rate of change of the distance between the target and the vehicle's driving trajectory in the speed state quantity; the lateral speed V3 of the target is calculated according to the rate of change of the visual line pressure of the target in the speed state quantity; the lateral speed V4 of the target is calculated according to the rate of change of the lateral distance in the speed state quantity; and call the lateral speed V0 of the target output by the fusion in the speed state quantity; Set any lateral speed among V0, V1, V2, V3, and V4 as the default stable lateral speed; When the longitudinal distance between the target and the vehicle is within a second preset distance range in the sensor perception result, the default stable lateral speed is ignored and the stable lateral speed is determined to be 1 / 3*(V1+V2+V3); When the longitudinal distance between the target and the vehicle is within a second preset distance range and the target is a preset vehicle type, the default stable lateral speed is ignored and the stable lateral speed is determined to be 1 / 4*(V1+V2+V3+V4); When the sensor perception result shows that the longitudinal distance between the target and the vehicle is within a third preset distance range, the default stable lateral speed is ignored and the stable lateral speed is determined to be 1 / 2*(V1+V2); When the longitudinal distance between the target and the vehicle is within a third preset distance range and the target is a preset vehicle type, the default stable lateral speed is ignored and the stable lateral speed is determined to be 1 / 3*(V1+V3).

7. The method according to claim 3, characterized in that According to the target perception result, a comprehensive risk assessment result corresponding to the target perception result is determined by a decision module, including: According to the target perception result, a first assessment result and a second assessment result are determined by an objective risk assessment unit in a decision module, wherein the objective risk assessment unit includes a first risk assessment subunit based on field theory and a second risk assessment subunit based on a social force model, the first assessment result is obtained by the first risk assessment subunit performing a risk assessment on the target identification result, and the second assessment result is obtained by the second risk assessment subunit performing a risk assessment on the target identification result; Performing enhanced verification on the first assessment result and the second assessment result by the objective risk assessment unit to obtain an objective risk quantitative assessment result; Through the subjective risk assessment unit in the decision-making module, corresponding risk behaviors are mapped from the preset risk scenarios to obtain the subjective risk quantitative assessment results; The objective risk quantification evaluation result and the subjective risk quantification evaluation result are combined by the decision unit in the decision module to obtain the comprehensive risk evaluation result that characterizes the objective environment and the driver's subjective psychological risk.

8. The method according to claim 7, characterized in that Performing enhanced verification on the first assessment result and the second assessment result by the objective risk assessment unit to obtain an objective risk quantitative assessment result, including: When, in the target recognition result, the longitudinal distance between the target and the vehicle is within a fourth preset distance range, the first assessment result is used for the first preset target, and the second assessment result is used for the second preset target, as the objective risk quantitative assessment result; When, in the target recognition result, the longitudinal distance between the target and the vehicle exceeds the fourth preset distance range, and the difference between the first evaluation result and the second evaluation result is within the second threshold range, the root mean square value of the first evaluation result and the second evaluation result is calculated, and the first preset target and the second preset target both use the root mean square value as the objective risk quantification evaluation result.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is run on a computer, the computer is caused to execute the method according to any one of claims 3 to 8.

Images