Definition and online monitoring method and system for autonomous driving operational design domain

Abstract

A definition and online monitoring method and system for an autonomous driving operational design domain (ODD). Taking a lane keeping function as an example, the method comprises: acquiring an observation data set, establishing a structure cause-and-effect diagram between lane line detection precision of an autonomous driving system and ODD elements on the basis of the observation data set, and fitting a structure cause-and-effect equation of the structure cause-and-effect diagram on the basis of an observation sample; acquiring various operating environment combinations on the basis of an operating parameter space of an autonomous vehicle, applying the acquired structure cause-and-effect equation to perform counterfactual reasoning on an autonomous driving performance under the various operating environment combinations, and selecting an operating environment combination having a counterfactual calculation result that is greater than a threshold value, so as to represent an ODD; and if the lane line detection precision in autonomous driving performance presentation indices is continuously less than a set threshold value, enabling a function degradation policy or reminding a driver to take over. The safety of autonomous vehicles is effectively improved by means of ODD online monitoring.

Description

Definition and online monitoring method and system for autonomous driving operation design domain Technical Field

[0001] The present invention relates to the field of artificial intelligence and autonomous driving technology, and in particular to a method, system, terminal, and computer-readable storage medium for defining and online monitoring a design domain for autonomous driving operations. Background Art

[0002] Autonomous vehicles face significant safety challenges in complex and unpredictable environments. To address this, the industry has proposed the concept of an Operational Design Domain (ODD), which attempts to reduce traffic accidents by specifying the environment in which autonomous vehicles must operate safely.

[0003] However, current ODD definitions often rely on subjective experience and lack clear operational guidelines. Furthermore, the vehicle's driving environment is constantly changing, meaning that deployed autonomous vehicles often encounter operating conditions outside the defined ODD range. Therefore, the design of a real-time ODD monitoring system is crucial. This system can assess whether the current driving environment exceeds safety boundaries online and switch to the lowest-risk mode (or alert the human driver to take over) when the autonomous driving system's performance degrades.

[0004] Currently, there is no rational design solution specifically for online ODD monitoring. While some studies have reported on online verification of autonomous driving subsystems, these solutions suffer from limitations such as limited adaptability, high computational complexity, and excessive reliance on engineering experience, making them inadequate for online ODD monitoring in autonomous vehicles.

[0005] Therefore, the existing technology still needs to be improved and developed.

[0006] Summary of the Invention

[0007] The main purpose of the present invention is to provide a method, system, terminal and computer-readable storage medium for defining and online monitoring of the operation design domain of autonomous driving, aiming to solve the problems in the existing technology of online monitoring of the operation design domain, such as poor adaptability, high computational complexity and excessive reliance on engineering experience.

[0008] To achieve the above objectives, the present invention provides a method for defining and online monitoring a design domain for autonomous driving operations. The method comprises the following steps:

[0009] Obtaining an observation data set, establishing a structural causal graph between lane detection accuracy of the autonomous driving system and elements of the operational design domain based on the observation data set, and fitting a structural causal equation of the structural causal graph based on the observation samples;

[0010] Based on the operating parameter space of the autonomous vehicle, various operating environment combinations are obtained. The obtained structural causal equations are used to perform counterfactual reasoning on the autonomous driving performance under various operating environment combinations. The environmental combination with the counterfactual calculation result greater than the threshold is selected to represent the design operating domain.

[0011] During actual deployment, if the lane line detection accuracy in the autonomous driving performance indicator continues to be less than the set threshold, the function degradation strategy will be activated or the driver will be reminded to take over. The function degradation strategy uses radar to infer the relative position of the vehicle in front in the same lane and obtains a reference path through historical trajectory fitting.

[0012] Optionally, the method for defining and online monitoring the autonomous driving operation design domain, wherein the step of obtaining an observation dataset, establishing a structural causal graph between the lane detection accuracy of the autonomous driving system and elements of the operation design domain based on the observation dataset, and fitting a structural causal equation of the structural causal graph based on the observation samples, specifically includes:

[0013] The environmental variables to be investigated are combined into conditions, and a certain number of images are captured for each combination;

[0014] Clean and label the collected images, and extract no-reference image quality assessment features from the images to obtain the observation dataset;

[0015] Identify key operational design domain elements from high-dimensional parameter space based on causal structure discovery algorithms;

[0016] Based on the observation data set, a causal structure discovery method is used to establish a structural causal graph between the lane line detection accuracy of the autonomous driving system and the operation design domain elements, and a structural causal equation of the structural causal graph is fitted according to the observation samples.

[0017] Optionally, the method for defining and online monitoring a design domain for autonomous driving operation further includes:

[0018] The causal relationship between a set of variables is represented by a directed acyclic graph G(ν,ν), where the node ν represents a random variable and the edge ε represents the causal path between the variables;

[0019] The process of constructing a structural causal diagram starts from a blank graph G init Start by adding or removing edges gradually until the BIC score function converges.

[0020] Optionally, in the method for defining and online monitoring the design domain for autonomous driving operations, the scoring function used is the Bayesian Information Criterion: BIC = 2·ln P(data|θ,G)-c·k·ln(n)

[0021] Where θ represents the parameters of the directed acyclic graph G(ν,ε), c is a constant equal to 1, k is the number of graph parameters, and n represents the sample size of the observed data;

[0022] The structural equation is:

[0023] Among them, X represents the causal variable, H represents the mediating variable, and Y represents the response variable; U X 、U H 、U Y are the values ​​of the exogenous variables corresponding to X, H, and Y respectively; α, β, and γ represent the causal transmission path;

[0024] Structural equations are fitted using linear or nonlinear functions;

[0025] When an autonomous vehicle faces a new operating environment, it performs counterfactual calculations including:

[0026] Find the closest running environment X from known observation data new Records of {X obs ,H obs ,Y} to obtain the value of the exogenous variable U:

[0027] Among them, obs represents the corresponding item in the observation data set;

[0028] Set the causal variable to the current running environment variable, the new observation environment X new Down The counterfactual result is expressed as:

[0029] in, Indicates that the causal variable X takes the value X new The value of the response variable Y when M x represents the counterfactual causal model; u represents the specific value of the exogenous variable;

[0030] New EnvironmentX new The lane line detection accuracy counterfactual estimation result under is:

[0031]

[0032] Structural equations are parameterized representations of structural causal diagrams.

[0033] Optionally, the method for defining and online monitoring the design domain for autonomous driving operations, wherein the method obtains various operating environment combinations based on the operating parameter space of the autonomous driving vehicle, applies the obtained structural causal equations to perform counterfactual reasoning on the autonomous driving performance under the various operating environment combinations, and selects an environmental combination whose counterfactual calculation result is greater than a threshold value to represent the design operating domain, specifically includes:

[0034] Modeling the operational design domain definition as a probability distribution of a safe subspace of the autonomous driving operating environment;

[0035] If φ all :=φ(X) is the joint probability density distribution function of the given parameter space, φ ODD :=φ(X|c>Thr) is the joint probability density distribution function of the operation design domain subspace, Thr is the lane line detection accuracy threshold based on vision, φ all and φ OSS Represent the joint probability density distribution of the given parameter space and the safe subspace of the operational design domain respectively;

[0036] Use the statistical mathematical tool Copulas to accurately model the dependencies between marginal distributions;

[0037] If F(X) is the joint cumulative distribution function, F(X i ) marginal joint cumulative distribution function, then:

[0038] F(X)=C(F(X1),F(X2),…,F(X d ))

[0039] Where C:[0,1] d →[0,1] is the Copulas function, d represents the dimension, and the corresponding probability density function is:

[0040] φ=c(F(X1),F(X2),…,F(X d ))·f(X1)…f(X d )

[0041] Where: c(·) is the probability density of the Copulas function, f(X i ) is the marginal joint probability density distribution function.

[0042] Optionally, in the method for defining and online monitoring the autonomous driving operational design domain, during actual deployment, if the lane detection accuracy in the autonomous driving performance indicator is continuously less than a set threshold, a function degradation strategy is activated or the driver is prompted to take over. The function degradation strategy uses radar to infer the relative position of the vehicle ahead in the same lane and obtains a reference path through historical trajectory fitting, specifically including:

[0043] The extended Kalman filter combines radar information as observations with the prediction model. The prediction equation and measurement equation of the extended Kalman filter are expressed as follows:

[0044] Among them, x t Represents the state vector, the state vector x t = [x, y, θ, v] represents the longitudinal distance, lateral distance, heading angle and speed of the leading vehicle relative to the main vehicle, z t represents the observation vector, Q and R represent the noise matrices in the prediction model and measurement model respectively, H is the unit measurement equation matrix, T represents the sampling time, and F represents the state transfer matrix;

[0045] Within a time step, the motion of the ego vehicle is approximated as uniform circular motion, and the historical positions x′ and y′ of the preceding vehicle in the main lane are converted to the coordinate system of the current ego vehicle:

[0046] Where Δx and Δy represent the position changes in the ego-vehicle coordinate system, and α is the heading angle offset, which is updated using the following equation:

[0047] Among them, ω ego and ω′ ego are the yaw rates of the current time step and the previous time step respectively;

[0048] Polynomial curve fitting is used to convert the trajectory points into the current main vehicle coordinate system, and the fitted curve is used as the reference path.

[0049] Optionally, the method for defining and online monitoring the design domain for autonomous driving operations, wherein the prediction equation is used to estimate the system state at the next time step based on the dynamic model of the system; the measurement equation is used to compare sensor measurement data with the predicted state and generate a corrected estimate of the system state; the measurement equation corrects errors in the prediction by comparing the measurement information with the predicted state.

[0050] In addition, to achieve the above-mentioned objectives, the present invention further provides a system for defining and online monitoring a design domain for autonomous driving operations, wherein the system for defining and online monitoring a design domain for autonomous driving operations comprises:

[0051] a causal graph construction module, configured to obtain an observation dataset, establish a structural causal graph between lane detection accuracy of the autonomous driving system and elements of the operational design domain based on the observation dataset, and fit a structural causal equation of the structural causal graph based on the observation samples;

[0052] The counterfactual reasoning module is used to obtain various operating environment combinations based on the operating parameter space of the autonomous vehicle, apply the obtained structural causal equations to perform counterfactual reasoning on the autonomous driving performance under various operating environment combinations, and select the environment combination with the counterfactual calculation result greater than the threshold to represent the design operating domain;

[0053] The monitoring degradation module is used in actual deployment. If the lane line detection accuracy in the autonomous driving performance indicator is continuously less than the set threshold, the function degradation strategy is activated or the driver is reminded to take over. The function degradation strategy uses radar to infer the relative position of the vehicle in front in the same lane and obtains a reference path through historical trajectory fitting.

[0054] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a definition and online monitoring program for the autonomous driving operation design domain stored in the memory and runnable on the processor, and when the definition and online monitoring program for the autonomous driving operation design domain is executed by the processor, the steps of the definition and online monitoring method for the autonomous driving operation design domain as described above are implemented.

[0055] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a definition and online monitoring program for an autonomous driving operation design domain, and when the definition and online monitoring program for an autonomous driving operation design domain is executed by a processor, the steps of the definition and online monitoring method for an autonomous driving operation design domain as described above are implemented.

[0056] The present invention obtains an observation dataset, establishes a structural causal graph between the lane detection accuracy of the autonomous driving system and elements of the operational design domain based on the observation dataset, and fits a structural causal equation to the structural causal graph based on the observation samples. Various operating environment combinations are obtained based on the operating parameter space of the autonomous vehicle, and the obtained structural causal equations are applied to perform counterfactual reasoning on the autonomous driving performance under various operating environment combinations. Environmental combinations whose counterfactual calculation results are greater than a threshold are selected to represent the designed operating domain. During actual deployment, if the lane detection accuracy of the autonomous driving performance indicator remains below a set threshold, a functional degradation strategy is activated or the driver is prompted to take over. The functional degradation strategy uses radar to infer the relative position of the preceding vehicle in the same lane and obtains a reference path through historical trajectory fitting. This present invention is the first in the industry to apply causal inference theory to deeply explore the causal relationship between environmental conditions and algorithm performance, thereby more accurately evaluating the impact of different environmental factors on the algorithm. Compared with existing technologies, it is more scientific and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] FIG1 is a flow chart of a preferred embodiment of a method for defining and online monitoring a design domain for autonomous driving operation according to the present invention;

[0058] FIG2 is a method framework diagram of a preferred embodiment of the method for defining and online monitoring the autonomous driving operation design domain according to the present invention;

[0059] FIG3 is a schematic diagram of the observation data set construction process in a preferred embodiment of the method for defining and online monitoring the autonomous driving operation design domain of the present invention;

[0060] FIG4 is a schematic diagram of a causal graph structure discovery step in a preferred embodiment of the method for defining and online monitoring the autonomous driving operation design domain of the present invention;

[0061] FIG5 is a schematic diagram of an original causal graph M of a causal counterfactual reasoning diagram in a preferred embodiment of the method for defining and online monitoring the autonomous driving operation design domain of the present invention;

[0062] FIG6 is a schematic diagram of a modified causal counterfactual causal graph Mx of the causal counterfactual reasoning diagram in a preferred embodiment of the method for defining and online monitoring the autonomous driving operation design domain of the present invention;

[0063] FIG7 is a schematic diagram of an ODD definition process in a preferred embodiment of the method for defining and online monitoring an autonomous driving operation design domain according to the present invention;

[0064] FIG8 is a flow chart of an ODD online monitoring algorithm based on causal counterfactuals in a preferred embodiment of the method for defining and online monitoring the autonomous driving operation design domain of the present invention;

[0065] FIG9 is a schematic diagram of a lane fitting strategy for functional degradation based on radar information in a preferred embodiment of the definition and online monitoring system for the autonomous driving operation design domain of the present invention;

[0066] FIG10 is a schematic diagram of updating the historical position of the preceding vehicle to the current vehicle coordinates in a preferred embodiment of the definition of the autonomous driving operation design domain and the online monitoring system of the present invention;

[0067] FIG11 is a schematic diagram showing a preferred embodiment of the definition and online monitoring system for the autonomous driving operation design domain according to the present invention.

[0068] FIG12 is a schematic diagram of an operating environment of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0069] Current methods for defining the ODD of autonomous vehicles are generally ambiguous and lack universal solutions. Furthermore, existing online monitoring solutions, such as skill graphs and diagnostic graphs, cannot adequately handle dynamic road conditions. The high computational complexity of sequential logic poses significant challenges for practical application in complex scenarios.

[0070] This paper proposes a method for defining and online monitoring the ODD of autonomous vehicles based on causal inference. It can reasonably, stably and scalably identify the key ODD factors that affect the performance of the autonomous driving system, and characterize the ODD space in the form of probability distribution. Based on causal counterfactual reasoning calculation, the performance evaluation of the autonomous driving system in a random dynamic environment is realized with extremely low computational burden, thereby establishing an ODD online monitor. Taking the lane keeping task as an example, the paper further proposes a new functional degradation strategy based on the historical trajectory of the preceding vehicle.

[0071] In order to make the purpose, technical solutions and advantages of the present invention more clear and distinct, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0072] The method for defining and online monitoring the autonomous driving operation design domain according to a preferred embodiment of the present invention, as shown in FIG1 and FIG2 , comprises the following steps:

[0073] Step S10: Obtain an observation data set, establish a structural causal graph between the lane line detection accuracy of the autonomous driving system and the elements of the operation design domain based on the observation data set, and fit a structural causal equation of the structural causal graph based on the observation samples.

[0074] Specifically, the environmental variables that need to be examined are constituted into combination conditions, and a certain number of images are captured for each combination; the collected images are cleaned and labeled, and reference-free image quality assessment features are extracted from the images to obtain an observation data set; based on the causal structure discovery algorithm, key operational design domain elements are identified from the high-dimensional parameter space; based on the observation data set, a causal structure discovery method is used to establish a structural causal graph between the lane line detection accuracy of the autonomous driving system and the operational design domain elements, and the structural causal equation of the structural causal graph is fitted according to the observation samples.

[0075] The present invention first uses a causal structure discovery algorithm to identify key operational design domain (ODD) elements from a high-dimensional parameter space and then constructs a causal graph between lane detection accuracy and ODD elements. Typically, this requires obtaining data from randomized controlled trials, where the environmental variables to be investigated are randomly assigned to either a control or analysis group. However, obtaining randomized controlled trials is expensive or even impractical.

[0076] To this end, the present invention proposes using a combination testing principle to collect observational datasets (i.e., the observational dataset in Figure 2). Specifically, as shown in Figure 3, the environmental variables to be examined are combined into combination conditions, and a certain number of images are captured for each combination. Subsequently, the collected data is cleaned and labeled, and no-reference image quality assessment features are extracted from the images to measure the impact of various environmental factors on image quality.

[0077] Furthermore, based on the observation data set and integrating domain knowledge, a causal structure diagram of the impact of environmental factors on perception performance is constructed, and a structural causal equation of the structural causal diagram is fitted based on the observation samples.

[0078] Specifically, based on the collected observation data set, the present invention uses the causal structure discovery method to establish a structural causal graph represented by a Bayesian graph. The causal structure discovery method focuses on describing the causal relationship between a set of variables, which is represented by a directed acyclic graph G(ν,ε), where the node ν represents a random variable and the edge ε represents the causal path between the variables. The process of constructing a causal graph starts from a blank graph G init Start by adding or deleting edges gradually until the BIC score function reaches convergence. The construction process is shown in Figure 4. Start: Given an empty graph G init Add edges, step 1: check for loops or disconnections, step 2: determine whether the BIC score is improved, step 3: if the score is improved, add or delete edges and update the causal structure loop to enter step 1, otherwise end.

[0079] The scoring function used in the present invention is the Bayesian Information Criterion (BIC):

[0080] BIC=2·ln P(data|θ,G)-c·k·ln(n)

[0081] Where θ represents the parameters of the directed acyclic graph G(ν,ε), c is a constant equal to 1, k is the number of graph parameters, and n represents the sample size of the observed data.

[0082] Generally speaking, a higher BIC value means that the discovered causal graph can provide a more reliable causal explanation for the observed dataset without over-complication. It should be noted that in practical applications, pruning the causal graph can still be combined with expert testing to obtain more reliable results.

[0083] The resulting causal graph provides a valuable analytical foundation for lane detection accuracy. In practical applications, a quantitative description of the causal relationships between nodes in a causal graph is often required. Structural equations, as a mathematical representation, describe the underlying mechanisms and dependencies between endogenous variables in a causal model. These equations embody the original mechanism and the connection between cause and effect. By instantiating structural equations through a nonlinear regression model, they theoretically establish the mathematical foundation for counterfactual reasoning.

[0084] Causal counterfactual reasoning aims to infer the potential impact on lane detection accuracy when the values ​​of certain ODD factors change. This enables, for the first time, the quantitative and interpretable calculation of lane detection performance under different operating conditions from a causal perspective, without the need for exhaustive testing. For ease of description, we assume the existence of a causal graph structure as shown in Figure 4.

[0085] As shown in Figure 5, the structural equation is:

[0086] Among them, X represents the causal variable, H represents the mediating variable, and Y represents the response variable; U X 、U H 、U Y are the values ​​of the exogenous variables corresponding to X, H, and Y respectively; α, β, and γ represent the causal transmission path;

[0087] Structural equations are fitted using linear or nonlinear functions.

[0088] When an autonomous vehicle faces a new operating environment, it performs counterfactual calculations including:

[0089] Find the closest running environment X from known observation data new Records of {X obs ,H obs ,Y} to obtain the value of the exogenous variable U:

[0090] Among them, obs represents the corresponding item in the observation dataset.

[0091] Set the causal variables to the current operating environment variables, that is, modify the causal graph, as shown in Figure 6. New observation environment X new Down The counterfactual result is expressed as:

[0092] in, Indicates that the causal variable X takes the value X new The value of the response variable Y when M xrepresents the counterfactual causal model; u represents the specific value of the exogenous variable. Using the modified counterfactual causal model and the values ​​of the exogenous variables obtained in the first step, we can answer the following counterfactual question: "What would be the lane detection accuracy if the autonomous vehicle encountered a new ODD environment instead of known observational data?"

[0093] According to the causal diagram in Figure 6, the new environment X new The lane line detection accuracy counterfactual estimation result under is:

[0094] In summary, structural equations (i.e., causal path functions, α, β, and γ, fitted using linear / nonlinear models) are parameterized representations of causal graphs, where each equation explains the process by which a causal variable acquires its value.

[0095] This invention provides a prediction method based on causal inference theory. It compares historical data and infers algorithm performance under new operating conditions. Once autonomous vehicles have this capability, they will be like adding a real-time monitor. If the estimated algorithm performance falls short of expectations, it can promptly alert humans to take over or activate a backup plan. This will further improve the safety of autonomous vehicles.

[0096] Step S20: Obtain various operating environment combinations based on the operating parameter space of the autonomous driving vehicle, apply the obtained structural causal equations to perform counterfactual reasoning on the autonomous driving performance under the various operating environment combinations, and select an environmental combination whose counterfactual calculation results are greater than a threshold to represent the design operating domain.

[0097] Specifically, this paper proposes to model the ODD definition as a probability distribution of the safe subspace of the autonomous driving operating environment, that is, the autonomous vehicle can safely operate under various combinations of conditions sampled within the safe subspace. Unlike existing fuzzy specifications, a performance alternative indicator c is defined: ( represents the domain of positive real numbers) to assist in defining the ODD space, as shown in Figure 7. In the ODD definition, various combinations of environmental factors are no longer uniformly sampled, but are sampled according to real distributions. For example, the amount of rainfall in a year is not uniformly distributed, but follows a certain probability distribution.

[0098] In which, assuming φ all :=φ(X) is the joint probability density function (PDF) of the given parameter space, φ OSS :=φ(X|v>Thr) is the joint probability density distribution function of the operation design domain subspace, Thr is the lane line detection accuracy threshold based on vision, φ all and φODD Represent the joint probability density distribution of the given parameter space and the safe subspace of the operational design domain, respectively.

[0099] In order to obtain the probability distribution φ in the ODD space ODD , using causal counterfactual reasoning techniques, inferring from a given parameter space φ all The lane detection accuracy under the generated environment combination is calculated, and samples with prediction values ​​greater than a given threshold are used to reconstruct φ ODD .

[0100] It is worth noting that the marginal distributions of ODD elements are not independent, which means that we cannot simply multiply the marginal probability distributions directly to obtain φ all and φ ODD To this end, the present invention introduces the statistical mathematical tool Copulas to accurately model the dependencies between various marginal distributions.

[0101] If F(X) is the Cumulative Distribution Function (CDF), F(X i ) is the marginal joint cumulative distribution function, then:

[0102] F(X)=C(F(X1),F(X2),…,F(X d ))

[0103] Where C:[0,1] d →[0,1] is the Copulas function, d represents the dimension, and the corresponding probability density function is:

[0104] φ=c(F(X1),F(X2),…,F(X d ))·f(X1)…f(X d )

[0105] Where: c(·) is the probability density of the Copulas function, f(X i ) is the marginal joint probability density distribution function.

[0106] Step S30: During actual deployment, if the lane line detection accuracy in the autonomous driving performance indicator continues to be less than the set threshold, a function degradation strategy is activated or the driver is reminded to take over. The function degradation strategy uses radar to infer the relative position of the vehicle in front in the same lane and obtains a reference path through historical trajectory fitting.

[0107] Specifically, based on the aforementioned causal graph structure and counterfactual calculation method, the present invention proposes a new ODD online monitoring method. If the autonomous driving performance continues to be less than the set threshold, the function degradation strategy is activated or the driver is reminded to take over to improve the safety of the autonomous driving car. As shown in Figure 8, counter and flag represent the count cache variable and the flag bit of whether the function degradation strategy is activated. Initialize the variables counter and flag to 0 and obtain the current driving environment X. new , judge whether counter is less than or equal to 5, if it is, flag = True, and end, if it is, find the same as X from the observation data set new The closest match, further using causal counterfactuals to infer lane detection accuracy Then determine whether If yes, counter+1, and then re-obtain the current driving environment X new If not, then re-acquire the current driving environment X new .

[0108] If the autonomous driving performance continues to be lower than the set threshold, a function degradation strategy will be activated or the driver will be reminded to take over to improve the safety of the autonomous driving vehicle. The autonomous driving performance refers to the use of the multi-dice score as an indicator to measure the accuracy of visual lane line detection.

[0109] Causal inference theory provides a formal framework for modeling the causal relationship between autonomous driving system performance and different ODD elements. This study uses counterfactual reasoning techniques to infer the performance of autonomous vehicles in unknown and random dynamic environments, thereby addressing the problem of existing research that relies too much on engineering experience in the definition of ODD and online monitoring.

[0110] Specifically, millimeter-wave radar is very effective in all weather conditions due to its ability to penetrate fog, smoke and dust. Therefore, the present invention proposes a lane keeping strategy for functional degradation to solve the problem of continuous performance degradation of the visual perception algorithm when the operating environment exceeds the ODD range. The strategy uses radar to infer the relative position of the vehicle in front of the same lane and derives the reference path through historical trajectory fitting. The detailed processing flow is shown in Figure 9. The input radar point cloud information is mapped to the pixel coordinate system, and the DBSCAN-based clustering algorithm is used to obtain the target area of ​​interest. After that, the position of the cluster center is extracted, and the trajectory points are stored in a queue through the extended Kalman filter and the coordinate transformation is performed. Finally, the output is subjected to polynomial fitting to obtain the reference path.

[0111] Among them, the extended Kalman filter (EKF) combines radar information as observation values ​​(which can be understood as the location of the cluster center) with the prediction model. The prediction equation and measurement equation of the extended Kalman filter are expressed as follows:

[0112] Among them, x t Represents the state vector, the state vector x t = [x, y, θ, v] represents the longitudinal distance, lateral distance, heading angle and speed of the leading vehicle relative to the main vehicle, z t represents the observation vector, Q and R represent the noise matrices in the prediction model and measurement model, respectively, H is the unit measurement equation matrix, T represents the sampling time, and F represents the state transition matrix. Considering that the ego vehicle is in a continuous motion state, it is necessary to convert the historical trajectory coordinates of the preceding vehicle into the current host vehicle coordinate system in real time. As shown in Figure 10, X′O′Y′ and XOY are the coordinate systems of the previous and current timestamps of the ego vehicle, respectively.

[0113] The prediction equation estimates the system state at the next time step based on the system's dynamic model. It generates a predicted state based on the previous state estimate and the system's control input. By accounting for the system's dynamics, the prediction equation allows the filter to update the system state over time to predict the future state. The observation equation compares sensor measurement data with the predicted state and generates a corrected estimate of the system state. It accounts for measurement noise and uncertainty in the system state to generate a corrected state estimate. By comparing measurement information with the predicted state, the observation equation corrects for errors in the prediction, thereby improving the accuracy of the state estimate.

[0114] Within a time step, the motion of the ego vehicle is approximated as uniform circular motion, and the historical positions x′ and y′ of the preceding vehicle in the main lane are converted to the coordinate system of the current ego vehicle:

[0115] Where Δx and Δy represent the position change of the ego vehicle in the coordinate system, and α is the heading angle offset, which is updated by the following equation:

[0116] Among them, ω ego and ω′ ego are the yaw angular rates of the current time step and the previous time step, respectively, and T represents the sampling time.

[0117] Polynomial curve fitting is used to convert the trajectory points into the current host vehicle coordinate system, and the fitted curve is used as the reference path for the downstream lane keeping control algorithm.

[0118] The operating environment of autonomous driving is dynamic and random, and it is impossible to rely solely on testing to cover all driving conditions. Once a self-driving car encounters a new operating condition that has never been seen before, the key point of this invention is to propose a self-driving car ODD definition and online monitoring method based on causal reasoning. Specifically, it includes qualitative mining of causal relationships, instantiation of causal transmission paths (represented by structural equations); defining the probability distribution of the ODD space from the global parameter space; designing an online ODD monitor based on counterfactual calculations; and a lane keeping function degradation strategy based on radar information. In summary, this invention proposes a new ODD definition and monitoring method that can effectively improve the safety of self-driving cars.

[0119] Compared with the best existing technology, the present invention has the following advantages (i.e. beneficial effects):

[0120] (1) In-depth exploration of causal relationships: This invention is the first in the industry to apply causal inference theory to deeply explore the causal relationship between environmental conditions and algorithm performance, thereby enabling a more accurate assessment of the impact of different environmental factors on the algorithm, which is more scientific and reliable than existing technologies.

[0121] (2) Reasonable ODD definition method: This invention is the first in the industry to propose a clear and quantitative definition method for ODD, which solves the drawbacks of existing methods that mainly rely on vague specifications and engineering experience.

[0122] (3) Interpretable ODD real-time online monitoring method: This invention is the first in the industry to propose an interpretable, easily scalable, and low-computational complexity ODD online monitoring method. This method uses the causal structure discovered above and instantiates the structural equation, and provides a method for using causal counterfactuals to predict autonomous driving performance in a new environment.

[0123] (4) A novel lane keeping function degradation strategy: This invention utilizes all-weather radar to record the preceding vehicle's historical trajectory information and converts it to the current coordinate system of the vehicle in real time. It then uses polynomial fitting to obtain the lane keeping reference trajectory. This strategy has high accuracy and good real-time performance, and has great potential as a functional degradation strategy when vision algorithms degrade.

[0124] In summary, the present invention has significant advantages in causal relationship mining, ODD definition method, explainable ODD real-time online monitoring method, lane keeping function degradation strategy, etc., and can more accurately evaluate and verify the performance of visual algorithms in complex environments.

[0125] In addition, the method described in the present invention has been subjected to a large number of experiments. Compared with the scheme without ODD definition and monitoring, the accident rate of autonomous driving vehicles under this scheme has dropped by three orders of magnitude.

[0126] Furthermore, as shown in FIG11 , based on the above-mentioned definition and online monitoring method for the autonomous driving operation design domain, the present invention also provides a definition and online monitoring system for the autonomous driving operation design domain, wherein the definition and online monitoring system for the autonomous driving operation design domain includes:

[0127] a causal graph construction module 51 for acquiring an observation data set, establishing a structural causal graph between lane detection accuracy of the autonomous driving system and elements of the operational design domain based on the observation data set, and fitting a structural causal equation of the structural causal graph based on the observation samples;

[0128] a counterfactual reasoning module 52 for obtaining various operating environment combinations based on the operating parameter space of the autonomous vehicle, applying the obtained structural causal equation to perform counterfactual reasoning on the autonomous driving performance under the various operating environment combinations, and selecting an environmental combination for which the counterfactual calculation result is greater than a threshold value to represent the design operating domain;

[0129] The monitoring degradation module 53 is used in actual deployment. If the lane line detection accuracy in the autonomous driving performance indicator is continuously less than the set threshold, the function degradation strategy is activated or the driver is reminded to take over. The function degradation strategy uses radar to infer the relative position of the vehicle in front in the same lane and obtains a reference path through historical trajectory fitting.

[0130] Furthermore, as shown in FIG12 , based on the above-described definition of the autonomous driving operation design domain and the online monitoring method and system, the present invention also provides a terminal, which includes a processor 10, a memory 20, and a display 30. FIG12 illustrates only some of the components of the terminal, but it should be understood that implementation of all illustrated components is not required, and more or fewer components may be implemented instead.

[0131] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Furthermore, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, the memory 20 stores a definition and online monitoring program 40 for the autonomous driving operation design domain, which can be executed by the processor 10, thereby realizing the definition and online monitoring method for the autonomous driving operation design domain in this application.

[0132] In some embodiments, the processor 10 can be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the definition and online monitoring method of the autonomous driving operation design domain.

[0133] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0134] In one embodiment, when the processor 10 executes the definition and online monitoring program 40 for the autonomous driving operation design domain in the memory 20, the steps of the definition and online monitoring method for the autonomous driving operation design domain as described above are implemented.

[0135] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a definition and online monitoring program for an autonomous driving operation design domain, and when the definition and online monitoring program for an autonomous driving operation design domain is executed by a processor, the steps of the definition and online monitoring method for an autonomous driving operation design domain as described above are implemented.

[0136] In summary, the present invention provides a method and related equipment for defining and online monitoring an autonomous driving operational design domain. The method comprises: obtaining an observation dataset, establishing a structural causal graph based on the observation dataset between the lane detection accuracy of the autonomous driving system and operational design domain elements, and fitting structural causal equations to the structural causal graph based on observation samples; obtaining various operating environment combinations based on the autonomous vehicle's operating parameter space, applying the obtained structural causal equations to perform counterfactual reasoning on the autonomous driving performance under each operating environment combination, and selecting an environmental combination whose counterfactual calculation result exceeds a threshold to represent the design operational domain; and, during actual deployment, if the lane detection accuracy of the autonomous driving performance indicator remains below a set threshold, a functional degradation strategy is activated or the driver is prompted to take over. The functional degradation strategy uses radar to infer the relative position of the preceding vehicle in the same lane and obtains a reference path through historical trajectory fitting. The present invention obtains a lane keeping reference trajectory through online ODD monitoring, with high accuracy and real-time performance, effectively improving the safety of autonomous vehicles.

[0137] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or terminal comprising the element.

[0138] Of course, those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware (such as a processor, controller, etc.) through a computer program. The program can be stored in a computer-readable storage medium that can be read by a computer. When the program is executed, it can include the processes in the above-described method embodiments. The computer-readable storage medium can be a memory, a magnetic disk, an optical disk, etc.

[0139] It should be understood that the application of the present invention is not limited to the above examples. For those skilled in the art, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for defining and online monitoring of an autonomous driving operation design domain, characterized in that: The definition and online monitoring method for the autonomous driving operation design domain includes: Acquire an observation data set, establish a structural causal graph between lane detection accuracy of the autonomous driving system and elements of the operation design domain according to the observation data set, and fit a structural causal equation of the structural causal graph according to the observation sample; Obtain various operating environment combinations based on the operating parameter space of the autonomous vehicle, apply the obtained structural causal equations to perform counterfactual reasoning on the autonomous driving performance under various operating environment combinations, and select environmental combinations with counterfactual calculation results greater than a threshold to represent the design operating domain; During actual deployment, if the lane line detection accuracy in the autonomous driving performance indicator continues to be less than the set threshold, the function degradation strategy will be enabled or the driver will be reminded to take over. The function degradation strategy uses radar to infer the relative position of the vehicle in front in the same lane and obtains the reference path through historical trajectory fitting.

2. The method for defining and online monitoring the autonomous driving operation design domain according to claim 1, characterized in that: The step of obtaining an observation data set, establishing a structural causal graph between lane detection accuracy of the autonomous driving system and elements of the operation design domain according to the observation data set, and fitting a structural causal equation of the structural causal graph according to the observation sample specifically includes: The environmental variables to be investigated are combined into conditions, and a certain number of images are captured for each combination; Clean and label the collected images, and extract no-reference image quality assessment features from the images to obtain an observation dataset; Identify key operational design domain elements from high-dimensional parameter space based on causal structure discovery algorithms; According to the observation data set, a causal structure discovery method is used to establish a structural causal graph between the lane line detection accuracy of the autonomous driving system and the elements of the operation design domain, and a structural causal equation of the structural causal graph is fitted according to the observed samples.

3. The method for defining and online monitoring the autonomous driving operation design domain according to claim 2, characterized in that: The definition and online monitoring method for the autonomous driving operation design domain also includes: The causal relationship between a set of variables is represented by a directed acyclic graph G(ν,ε), where the node ν represents a random variable and the edge ε represents the causal path between the variables; The process of constructing a structural causal graph starts from a blank graph G. init Start with and gradually add or remove edges until the BIC score function converges.

4. The method for defining and online monitoring the autonomous driving operation design domain according to claim 3, characterized in that: The scoring function used is the Bayesian Information Criterion: BIC = 2·ln P(data|θ,G)-c·k·ln(n) Among them, θ represents the parameters of the directed acyclic graph G(ν,ε), c is a constant equal to 1, k is the number of graph parameters, and n represents the sample size of the observed data; The structural equation is: Among them, X represents the causal variable, H represents the mediating variable, and Y represents the response variable; U X , U H , U Y are the values ​​of the exogenous variables corresponding to X, H, and Y respectively; α, β, and γ represent the causal transmission paths; Structural equations are fitted using linear or nonlinear functions; When an autonomous vehicle faces a new operating environment, it performs counterfactual calculations including: Find the closest operating environment X from known observational data new Records of {X obs ,H obs ,Y} to obtain the value of the exogenous variable U: Among them, obs represents the corresponding item in the observation data set; Set the causal variable to the current running environment variable, the new observation environment X new Down The counterfactual result is expressed as: in, Indicates that the causal variable X takes the value X new The value of the response variable Y when M x represents the counterfactual causal model; u represents the specific value of the exogenous variable; New EnvironmentX new The lane line detection accuracy counterfactual estimation result under is: Structural equations are parameterized representations of structural causal diagrams.

5. The method for defining and online monitoring the autonomous driving operation design domain according to claim 4, characterized in that: The method of obtaining various operating environment combinations according to the operating parameter space of the autonomous driving vehicle, applying the obtained structural causal equation to perform counterfactual reasoning on the autonomous driving performance under various operating environment combinations, and selecting an environment combination whose counterfactual calculation result is greater than a threshold to represent the design operating domain specifically includes: Modeling the operational design domain definition as a probability distribution of a safe subspace of the autonomous driving operating environment; If φ all :=φ(X) is the joint probability density distribution function of the given parameter space, φ ODD :=φ(X|c>Thr) is the joint probability density distribution function of the operation design domain subspace, Thr is the lane line detection accuracy threshold based on vision, φ all and φ ODD Represent the joint probability density distribution of the given parameter space and the safe subspace of the operational design domain respectively; Use the statistical mathematical tool Copulas to accurately model the dependencies between various marginal distributions; If F(X) is the joint cumulative distribution function, F(X i ) is the marginal joint cumulative distribution function, then: F(X)=C(F(X1),F(X2),…,F(X d )) Where C:[0,1] d →[0,1] is the Copulas function, d represents the dimension, and the corresponding probability density function is: φ=c(F(X1),F(X2),…,F(X d ))·f(X1)…f(X d ) Where: c(·) is the probability density of the Copulas function, f(X i ) is the marginal joint probability density distribution function.

6. The method for defining and online monitoring the autonomous driving operation design domain according to claim 5, characterized in that: During the actual deployment, if the lane detection accuracy in the autonomous driving performance indicator is continuously less than the set threshold, the function degradation strategy is activated or the driver is reminded to take over. The function degradation strategy uses radar to infer the relative position of the vehicle in front of the same lane and obtains a reference path through historical trajectory fitting, specifically including: The extended Kalman filter combines radar information as observations with the prediction model. The prediction equation and measurement equation of the extended Kalman filter are expressed as follows: Among them, x t represents the state vector, the state vector x t = [x, y, θ, v] represents the longitudinal distance, lateral distance, heading angle and speed of the front vehicle relative to the main vehicle, z t represents the observation vector, Q and R represent the noise matrices in the prediction model and measurement model respectively, H is the unit measurement equation matrix, T represents the sampling time, and F represents the state transfer matrix; In one time step, the motion of the ego vehicle is approximated as uniform circular motion, and the historical positions x′ and y′ of the front vehicle in the main lane are converted to the coordinate system of the current ego vehicle: Among them, Δx and Δy represent the position change of the vehicle in the coordinate system, and α is the heading angle offset, which is updated by the following equation: Among them, ω ego and ω′ ego are the yaw rate of the current time step and the previous time step respectively; Polynomial curve fitting is used to convert the trajectory points into the current main vehicle coordinate system, and the fitted curve is used as the reference path.

7. The method for defining and online monitoring the autonomous driving operation design domain according to claim 1, characterized in that: The prediction equation is used to estimate the system state at the next time step based on the dynamic model of the system; the measurement equation is used to compare the sensor measurement data with the predicted state and generate a corrected estimate of the system state; the measurement equation corrects errors in the prediction by comparing the measurement information with the predicted state.

8. A system for defining and online monitoring the design domain of autonomous driving operations, characterized in that: The definition and online monitoring system for the autonomous driving operation design domain includes: A causal graph construction module is used to obtain an observation data set, establish a structural causal graph between the lane line detection accuracy of the autonomous driving system and the elements of the operation design domain according to the observation data set, and fit a structural causal equation of the structural causal graph according to the observation sample; The counterfactual reasoning module is used to obtain various operating environment combinations based on the operating parameter space of the autonomous driving vehicle, apply the obtained structural causal equations to perform counterfactual reasoning on the autonomous driving performance under various operating environment combinations, and select the environment combination whose counterfactual calculation result is greater than the threshold to represent the design operating domain; The monitoring degradation module is used in actual deployment. If the lane line detection in the autonomous driving performance indicator If the measurement accuracy is continuously less than the set threshold, the function degradation strategy is activated or the driver is reminded to take over. The function degradation strategy uses radar to infer the relative position of the vehicle in front of the same lane and obtains a reference path through historical trajectory fitting.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a definition and online monitoring program for an autonomous driving operation design domain stored in the memory and executable on the processor, wherein the definition and online monitoring program for an autonomous driving operation design domain, when executed by the processor, implements the steps of a definition and online monitoring method for an autonomous driving operation design domain as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a definition and an online monitoring program for an autonomous driving operation design domain. When the definition and the online monitoring program for the autonomous driving operation design domain are executed by a processor, the steps of the definition and online monitoring method for the autonomous driving operation design domain as described in any one of claims 1 to 7 are implemented.

Images