Method and device for enhancing interpretability of automatic driving scene

Abstract

The invention relates to the technical field of automatic driving, and discloses an explanatory enhancement method and device for an automatic driving scene, and the method comprises the steps: obtaining a driving scene label and sensor performance data corresponding to the driving scene label, and determining a plurality of weight combinations based on the driving scene label and the sensor performance data; performing experimental simulation training based on the sensor performance data and the plurality of weight combinations to generate experimental record data; performing anti-fact simulation on the experimental record data to generate anti-fact simulation record data; and finally, based on the experimental record data and the anti-fact simulation record data, performing causal model verification optimization to obtain a sensor weight distribution strategy in the automatic driving scene. According to the method, dynamic evaluation and optimization of sensor performance data in different driving scenes are realized, the problem of'black box 'in an automatic driving system is solved, and interpretability and transparency of a sensor fusion process are realized.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method and device for enhancing the interpretability of an autonomous driving scenario. Background Art

[0002] With the continuous advancement of autonomous driving technology, sensor fusion has become a core component of intelligent decision-making and perception systems; related autonomous driving systems perform environmental perception and decision-making by integrating data from different sensors (such as lidar, cameras, and radars).

[0003] However, relevant intelligent decision-making and perception systems still face the "black box" problem, that is, it is impossible to clearly explain how the autonomous driving system makes decisions based on sensor data in a specific driving scenario, especially how to assign weights and importance to different sensors in the perception process. Summary of the invention

[0004] In view of this, the present invention provides a method and device for enhancing the interpretability of autonomous driving scenarios to solve the "black box" problem faced by related intelligent decision-making and perception systems.

[0005] In a first aspect, the present invention provides a method for enhancing the interpretability of an autonomous driving scenario, the method comprising:

[0006] Acquire a driving scenario label and sensor performance data corresponding to the driving scenario label, and determine a plurality of weight combinations based on the driving scenario label and the sensor performance data;

[0007] Conduct experimental simulation training based on sensor performance data and multiple weight combinations to generate experimental record data;

[0008] Performing counterfactual simulation on experimental record data to generate counterfactual simulation record data;

[0009] Causal model verification and optimization are performed based on experimental recorded data and counterfactual simulation recorded data to obtain the sensor weight allocation strategy in the autonomous driving scenario.

[0010] The method for enhancing the interpretability of autonomous driving scenarios provided in this embodiment obtains driving scenario labels and sensor performance data corresponding to the driving scenario labels, and determines multiple weight combinations based on the driving scenario labels and the sensor performance data; performs experimental simulation training based on the sensor performance data and the multiple weight combinations to generate experimental record data; performs counterfactual simulation on the experimental record data to generate counterfactual simulation record data; performs causal model verification and optimization based on the experimental record data and the counterfactual simulation record data to obtain a sensor weight allocation strategy in the autonomous driving scenario; verifies and optimizes the causal model through the experimental record data and the counterfactual simulation record data, realizes dynamic evaluation and optimization of sensor performance data in different driving scenarios, solves the "black box" problem in the autonomous driving system, and realizes the interpretability and transparency of the sensor fusion process.

[0011] In an optional implementation, the driving scene labels and the sensor performance data corresponding to the driving scene labels are obtained, and a plurality of weight combinations are determined based on the driving scene labels and the sensor performance data, including:

[0012] Collect the target vehicle's positioning information, driving environment characteristics and weather conditions, classify and annotate the positioning information, driving environment characteristics and weather conditions, and obtain multiple driving scene labels;

[0013] Collect sensor performance data corresponding to various driving scenario labels respectively;

[0014] Construct a weight matrix based on driving scenario labels and sensor performance data;

[0015] Random perturbations are added to the weight matrix to generate multiple sets of weight combinations, and the multiple sets of weight combinations are stored according to driving scenario labels.

[0016] The method for enhancing the interpretability of autonomous driving scenarios provided in this embodiment constructs a weight matrix through driving scenario labels and sensor performance data; adds random perturbations to the weight matrix to generate multiple sets of weight combinations, and stores the multiple sets of weight combinations according to driving scenario labels, thereby generating multiple sets of weight combinations covering diverse driving scenarios, providing a data basis for analysis and characterization for weight combination optimization and driving scenario adaptation.

[0017] In an optional implementation, experimental simulation training is performed based on sensor performance data and multiple weight combinations to generate experimental record data, including:

[0018] Set a simulated driving scenario, and select a current weight combination based on a driving scenario label corresponding to the simulated driving scenario;

[0019] Perform multimodal data fusion on the sensor performance data corresponding to the simulated driving scenario to obtain a fused feature vector;

[0020] Select the optimal algorithm based on the simulated driving scenario and fused feature vector;

[0021] Based on the fusion feature vector and the current weight combination, the optimal algorithm is used to generate obstacle avoidance decision instructions and performance indicators;

[0022] The experimental record data is determined based on the simulated driving scenario, the current weight combination, the obstacle avoidance decision instructions and the performance indicators.

[0023] The method for enhancing the interpretability of autonomous driving scenarios provided in this embodiment sets a simulated driving scenario, selects the current weight combination based on the driving scenario label corresponding to the simulated driving scenario, and then experimentally simulates the performance of the vehicle under different driving scenarios and current weight combinations one by one, analyzes the impact of the simulated driving scenario, sensor performance data and weight combination on obstacle avoidance decision instructions and performance indicators, and provides basic data support for optimizing the causal model.

[0024] In an optional implementation, performing counterfactual simulation on experimental record data to generate counterfactual simulation record data includes:

[0025] Preprocess the experimental record data, and create counterfactual scenarios based on the preprocessed experimental record data;

[0026] Adjusting multiple weight combinations to obtain a weight combination of the oscillation sensor;

[0027] Generate simulated obstacle avoidance decisions and simulated indicators based on counterfactual scenarios and shock sensor weight combinations;

[0028] Determine counterfactual simulation record data based on counterfactual scenarios, shock sensor weight combinations, simulated obstacle avoidance decisions, and simulation indicators.

[0029] The method for enhancing the interpretability of autonomous driving scenarios provided in this embodiment determines counterfactual simulation record data based on counterfactual scenarios, oscillation sensor weight combinations, simulated obstacle avoidance decisions and simulation indicators, thereby revealing the impact of counterfactual scenarios and oscillation sensor weight combinations on obstacle avoidance decision instructions and performance indicators, laying the foundation for the subsequent optimization of causal models.

[0030] In an optional implementation, a causal model verification optimization is performed based on experimental record data and counterfactual simulation record data to obtain a sensor weight allocation strategy in an autonomous driving scenario, including:

[0031] Construct a causal model; a causal model consists of a set of variables and a causal path;

[0032] The causal model is verified based on experimental record data and counterfactual simulation record data to obtain verification bias;

[0033] The sensor weight distribution is adjusted based on the verification deviation to obtain the sensor weight distribution strategy in the autonomous driving scenario.

[0034] The method for enhancing the interpretability of the autonomous driving scenario provided in this embodiment adjusts the sensor weight distribution by verifying the deviation, and obtains the sensor weight distribution strategy in the autonomous driving scenario, so that each adjustment of the sensor weight has a clear basis and logical chain, and the decision-making process of each step can be clearly traced, which significantly improves the transparency of the autonomous driving system, avoids the unknowability and potential risks brought by the black box algorithm, improves the safety and credibility of the autonomous driving system, and can provide developers, operators and end users with more intuitive and easy-to-understand decision-making basis when making key decisions and handling abnormal driving scenarios.

[0035] In an optional implementation, the causal model verification optimization is performed based on the experimental record data and the counterfactual simulation record data to obtain the sensor weight allocation strategy in the autonomous driving scenario, which also includes:

[0036] The sensor weight allocation strategy in the autonomous driving scenario is iteratively verified, and the sensor weight allocation strategy is adjusted based on the iterative verification results.

[0037] The interpretability enhancement method for the autonomous driving scenario provided in this embodiment refines the sensor weight allocation strategy by iteratively verifying the sensor weight allocation strategy in the autonomous driving scenario and adjusting the sensor weight allocation strategy based on the iterative verification result, thereby ensuring that the sensor weight allocation strategy meets the actual needs of sensor performance data and avoiding waste of resources.

[0038] In a second aspect, the present invention provides an explanatory enhancement device for an autonomous driving scene, the device comprising:

[0039] A determination module, used to obtain a driving scene label and sensor performance data corresponding to the driving scene label, and determine a plurality of weight combinations based on the driving scene label and the sensor performance data;

[0040] A training module, used to perform experimental simulation training based on sensor performance data and multiple weight combinations to generate experimental record data;

[0041] A simulation module, used to perform counterfactual simulation on experimental record data to generate counterfactual simulation record data;

[0042] The optimization module is used to perform causal model verification and optimization based on experimental record data and counterfactual simulation record data to obtain the sensor weight allocation strategy in the autonomous driving scenario.

[0043] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method for enhancing the interpretability of the autonomous driving scenario of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0044] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method for enhancing the interpretability of an autonomous driving scenario according to the first aspect or any corresponding embodiment thereof.

[0045] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions, wherein the computer instructions are used to enable a computer to execute the method for enhancing the interpretability of an autonomous driving scenario according to the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 is a flow chart of an interpretative enhancement method for an autonomous driving scenario according to an embodiment of the present invention;

[0048] Figure 2 is a flowchart of specific steps of an interpretative enhancement method for an autonomous driving scenario according to an embodiment of the present invention;

[0049] Figure 3 is a flow chart of another method for enhancing the interpretability of an autonomous driving scenario according to an embodiment of the present invention;

[0050] Figure 4 is a flowchart of another method for enhancing the interpretability of an autonomous driving scenario according to an embodiment of the present invention;

[0051] Figure 5 is a flowchart of another method for enhancing the interpretability of an autonomous driving scenario according to an embodiment of the present invention;

[0052] Figure 6 is a schematic diagram of a flow chart of module connections in an explanatory enhancement system for an autonomous driving scenario according to an embodiment of the present invention;

[0053] Figure 7is a structural block diagram of an explanatory enhancement device for an autonomous driving scenario according to an embodiment of the present invention;

[0054] Figure 8 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0056] Related sensor fusion methods often rely on weighted averages based on fixed models or simple voting mechanisms. The above methods do not fully consider the differences in sensor performance in specific scenarios. For example, in environments with low light or bad weather, cameras may be affected, while radars may be more effective. However, related technologies often find it difficult to dynamically adjust the weight distribution between sensors, and cannot provide a clear explanation as to why a certain sensor has a higher weight at certain moments, making the decision-making process difficult to understand and verify.

[0057] In addition, although related methods have attempted to introduce a sensor weight adjustment mechanism based on scene recognition, most methods still rely on fixed decision models and cannot fully incorporate the dynamic characteristics of the scene; specifically, the system usually does not have the ability to flexibly adjust the sensor configuration and weight distribution according to the complexity of the driving environment and the real-time changes in sensor performance, resulting in the system's perception performance not being optimally utilized in complex scenarios; for example, in complex urban road environments, cameras and lidar may be the main perception tools, while in highway scenarios, the role of radar may be more prominent, but these dynamic changes cannot be accurately reflected in related technologies.

[0058] In order to solve the above technical problems, the embodiment of the present invention provides an interpretability enhancement method for autonomous driving scenarios. Through a multi-level analysis framework, the sensor performance is closely combined with the needs of specific driving scenarios to form a dynamic and interpretable sensor weight allocation mechanism, which not only improves the decision transparency of the system, but also enhances the adaptability and safety of the autonomous driving system in different scenarios; a scenario-based dynamic baseline sensor weight configuration is proposed, which realizes the automatic selection of the optimal sensor combination under different environmental conditions through real-time recognition of different driving scenarios and comprehensive evaluation of sensor data; by introducing a scene perception mechanism, combined with real-time sensor performance, the configuration and weight distribution between sensors are flexibly adjusted, thereby ensuring that the system can select the most appropriate sensor combination under different environmental conditions. This mechanism not only improves the perception accuracy of the autonomous driving system, but also enhances the system's adaptability in complex dynamic scenarios, ensuring the reliability and accuracy of the decision-making process.

[0059] According to an embodiment of the present invention, an embodiment of an explanatory enhancement method for an autonomous driving scenario is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0060] In this embodiment, a method for enhancing the interpretability of an autonomous driving scenario is provided, which can be used in the above-mentioned electronic device. Figure 1 is a flow chart of a method for enhancing the interpretability of an autonomous driving scenario according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0061] Step S101: Acquire a driving scenario label and sensor performance data corresponding to the driving scenario label, and determine a plurality of weight combinations based on the driving scenario label and the sensor performance data.

[0062] Step S102, performing experimental simulation training based on sensor performance data and multiple weight combinations to generate experimental record data.

[0063] Specifically, the system's performance under different driving scenarios and weight combinations is tested one by one through experimental simulation. The impact of scenarios, sensor performance data, and weight distribution on obstacle avoidance decisions are analyzed based on the test results, providing basic data support for subsequent optimization.

[0064] Specifically, Figure 2As shown in the figure, a diversified algorithm library containing multiple algorithms is constructed to achieve the purpose of experimental simulation training; the algorithm library covers methods such as rule control, deep learning and reinforcement learning, providing support for autonomous driving decision-making in different scenarios; multimodal sensor data is processed through the fusion layer module, providing standardized multimodal input for the algorithm library so that its weight distribution can be adjusted, and the algorithm performance is quantified through formulated performance indicators.

[0065] Among them, the above-mentioned algorithm library includes three types of algorithms: rule control algorithm: suitable for stable scenarios such as highways, and achieves obstacle avoidance through preset logical conditions (such as threshold detection); deep learning algorithm: uses neural networks to process complex unstructured data (such as pedestrian and vehicle recognition), suitable for dynamic interactive scenarios; reinforcement learning algorithm: optimizes strategies through interactive learning, especially suitable for rare or unknown environments; the algorithm library manages and uses rule control algorithms, deep learning algorithms and reinforcement learning algorithms in a modular manner; the above algorithms exist as independent modules in the algorithm library, and are integrated through unified interfaces and input / output standards; each algorithm is dynamically called according to driving scenario requirements and data characteristics.

[0066] Furthermore, the autonomous driving system manages the algorithm library through a modular architecture. The main modules include a fusion layer module, an algorithm selection module, and an algorithm execution module. The fusion layer module is used to process multimodal sensor data and generate a unified fusion feature vector. The algorithm selection module selects the optimal algorithm (including rule control algorithm, deep learning algorithm, and reinforcement learning algorithm) based on the driving scene label L(S) and the fusion feature vector output by the fusion layer module. The algorithm execution module is used to generate obstacle avoidance decision instructions. Specifically, the fusion layer module receives multimodal sensor data, such as lidar, camera, millimeter wave radar, and other sensor data, and extracts the data through standardization and feature extraction. Generate a unified fusion feature vector X′; the fusion feature vector X′ ensures that sensor data from different sources can be seamlessly integrated and supports the use of multiple types of algorithms. The feature vector X′ output by the fusion layer module is directly used as the input of the algorithm library, which is further processed by the algorithm library according to the driving scenario requirements; the algorithm library inputs the driving scenario label L(S) and the fusion layer module outputs the fusion feature vector X′; the driving scenario and the algorithm category are matched according to the driving scenario label L(S); the algorithm execution module calls the matching algorithm, uses the current weight combination W to adjust the fusion feature vector X′ output by the fusion layer module, and generates a standardized obstacle avoidance decision instruction u={a,δ} and performance indicators.

[0067] Furthermore, in addition to the modular architecture for managing algorithm libraries, the autonomous driving system can also use the following methods for experimental simulation training: Containerization technology: Encapsulate the various algorithms and related dependencies of the autonomous driving system in independent containers. Each container is like an independent operating environment, which contains everything needed for the operation of a specific algorithm, including code, runtime environment, system tools, libraries, etc.; Microservice architecture: Split the huge functions of the autonomous driving system into multiple small, independent microservices. Each microservice focuses on completing a specific business function. The above microservices collaborate with each other through a lightweight communication mechanism to form a complete autonomous driving system; Model management tools: Use specialized model management tools to perform version management, training tracking, and deployment management on machine learning models in the autonomous driving system.

[0068] Step S103, performing counterfactual simulation on the experimental record data to generate counterfactual simulation record data.

[0069] Step S104, performing causal model verification optimization based on the experimental record data and the counterfactual simulation record data to obtain a sensor weight allocation strategy in the driving scenario.

[0070] Specifically, Figure 2 As shown in the figure, through counterfactual simulation and feedback mechanism, the causal association model of "scenario-sensor performance-weight distribution" is verified and optimized based on the experimental record data; the accuracy of the association model is verified, that is, the causal relationship between sensor performance data, weight combination and driving scenario is explored, and it is ensured that the scenario adaptability and decision-making performance can be improved by adjusting the weight combination; the weight distribution rules are optimized, and the best sensor weight combination configuration under each driving scenario is found through counterfactual simulation, and the causal model between scenario-sensor performance-weight distribution is optimized and improved; the actual contribution of the sensor is identified, and the impact of each sensor performance data on the decision-making performance under the driving scenario is evaluated, providing a theoretical basis for sensor configuration optimization and dynamic adjustment.

[0071] The method for enhancing the interpretability of the autonomous driving scenario provided in this embodiment obtains a driving scenario label and sensor performance data corresponding to the driving scenario label, determines multiple weight combinations based on the driving scenario label and the sensor performance data; performs experimental simulation training based on the sensor performance data and the multiple weight combinations to generate experimental record data; performs counterfactual simulation on the experimental record data to generate counterfactual simulation record data; performs causal model verification and optimization based on the experimental record data and the counterfactual simulation record data to obtain the sensor weight allocation strategy in the autonomous driving scenario; verifies and optimizes the causal model through the experimental record data and the counterfactual simulation record data to obtain the sensor weight allocation strategy in the autonomous driving scenario, realizes dynamic evaluation and optimization of sensor performance data under different driving scenarios, solves the "black box" problem in the autonomous driving system, realizes the interpretability and transparency of the sensor fusion process, so that each adjustment of the sensor weight has a clear basis and logical chain, can clearly trace the decision-making process of each step, significantly improves the transparency of the autonomous driving system, avoids the unknowability and potential risks brought by the black box algorithm, improves the safety and credibility of the autonomous driving system, and can provide developers, operators and end users with more intuitive and easy-to-understand decision-making basis when dealing with key decisions and abnormal driving scenarios.

[0072] In this embodiment, a method for enhancing the interpretability of an autonomous driving scenario is provided, which can be used in the above-mentioned electronic device. Figure 3 is a flow chart of a method for enhancing the interpretability of an autonomous driving scenario according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0073] Step S301: Acquire a driving scenario label and sensor performance data corresponding to the driving scenario label, and determine a plurality of weight combinations based on the driving scenario label and the sensor performance data.

[0074] Specifically, by collecting the vehicle's positioning information, environmental characteristics and weather conditions, and combining artificial intelligence algorithms to classify and subcategory predict driving scenarios, we can achieve classification extraction of driving scenarios, generate driving scenario labels, and then form structured data records; by generating weight distribution combinations under different scenarios, we can provide weight configuration support for experimental simulation.

[0075] The above step S301 includes:

[0076] Step S3011, collecting the positioning information, driving environment characteristics and weather conditions of the target vehicle, classifying and labeling the positioning information, driving environment characteristics and weather conditions, and obtaining multiple driving scene labels.

[0077] Specifically, the vehicle's geographic location (x, y, z) and road type R are obtained through the GPS (Global Positioning System) module and high-definition map; x, y represent the longitude and latitude of the vehicle, z represents the altitude of the vehicle, and R represents the road type marked according to the high-definition map, including urban roads, rural roads and highways; and the vehicle's driving status and surrounding environment characteristics are collected, including dynamic information and environmental information, wherein the vehicle's dynamic information includes vehicle speed v, acceleration a and steering angle θ; driving environment characteristics include obstacle distance d obstacle , traffic signal status signal and light intensity L env , record the weather state w, which includes sunny days, nights and light rain, and pass the weather correction factor α w The sensor data is adjusted, and the specific adjustment formula is as follows:

[0078] D′ sensor =D sensor α w (1)

[0079] Among them, D′ sensor is the corrected sensor data, D sensor For sensor data.

[0080] Furthermore, artificial intelligence algorithms are used to predict specific subcategories of driving scenarios and generate labels for each driving scenario to generate driving scenario labels.

[0081] Furthermore, a feature data set for scene classification is input, including road type, vehicle dynamic information, driving environment information and weather conditions; the data D in the feature data set is normalized, and the calculation formula for the normalization is as follows:

[0082]

[0083] Among them, D′ is the normalized data, D min is the minimum value in the data, D max is the maximum value in the data.

[0084] Furthermore, the normalized data is input into the artificial intelligence model, and a pre-trained artificial intelligence model (e.g., a classifier or regression model based on deep learning) is used to predict the specific subcategory of the scene, and the subcategory label (i.e., the driving scene label L(S)) is output; wherein the input data of the artificial intelligence model is:

[0085] Y={R,v,a,θ,d obstacle ,L env ,w} (3)

[0086] Among them, Y is the normalized feature dataset.

[0087] Furthermore, the driving scenario labels L(S) include: smooth urban main roads during sunny daytimes, congested urban commercial areas during rainy nights, urban rush hour roads with traffic accidents in foggy mornings, small roads around farmlands during sunny daytimes, rural main roads with heavy rain at night, smooth rural roads in foggy mornings, smooth highway main roads during sunny daytimes, smooth highway main roads with slight traffic jams during sunny daytimes, and highway main roads with multiple vehicle accidents on heavy rain nights.

[0088] Specifically, the following expression is used as the training target of the artificial intelligence model to identify the external environment in which the vehicle is traveling:

[0089]

[0090] Among them, L(S) is the driving scene label, Predict labels for the model, is the loss function, and the loss function can adopt cross entropy loss.

[0091] Furthermore, the driving scene label is output according to the artificial intelligence model and stored in association with the collected data; for example, if R=city, v=10m / s, w=rainy is input, the driving scene label L(S) is output.

[0092] Specifically, the driving scene labels and related data are stored in a standardized manner to provide support for subsequent analysis and training; the data storage fields include the timestamp T corresponding to the data collection time, the scene label L(S) obtained by the classification result predicted by the model, the geographic location information (x, y, z), the dynamic information (v, a, θ), and the environmental information (d obstacle , L env , signal) and weather correction factor α w .

[0093] Step S3012, collecting sensor performance data corresponding to multiple driving scene labels respectively.

[0094] Specifically, the sensor types include: LiDAR: used for high-precision three-dimensional point cloud data; millimeter-wave radar: detects moving targets and distance data around the vehicle; camera: obtains color images and identifies traffic signals, pedestrians and other data; ultrasonic sensor Ultrasonic: short-distance obstacle detection data.

[0095] Furthermore, the sensor performance data collected by different types of sensors include: the number N of each type of sensor on the vehicle, the installation position P represented by three-dimensional coordinates (x, y, z) i, sensor detection range R, sensor spatial or temporal resolution Δ, sensor data update frequency f update , SNR (Signal-to-Noise Ratio) and the sensor delay τ from signal input to data output; where the sensor performance data collected by different types of sensors are:

[0096] Sensor_i={N,P i ,R,Δ,f update ,SNR,τ} (5)

[0097] Among them, Sensor_i is the sensor type and number.

[0098] Specifically, the performance data of each sensor is collected under different driving scenarios. The sensor performance data covers different weather, lighting conditions and road types. The collection content includes: the validity of each data, the frequency of data collection of each sensor and the data processing time; the data processing time includes the signal delay τ of each sensor sensor , and its calculation formula is as follows:

[0099] τ sensor =t out -t in (6)

[0100] Among them, t in is the signal input time, t out is the signal output time.

[0101] Furthermore, the collected sensor performance data is stored in a standardized format, including the sensor type and number Sensor_i, the collection scene associated with the driving scene label L(S), and the environmental conditions (including weather w and light intensity L env ); the data is classified and stored according to the driving scene label and sensor type. For example, if the driving scene label is urban road, the sensor performance data includes lidar performance data and camera performance data; if the driving scene label is highway, the sensor performance data includes millimeter wave radar performance data; if the driving scene label is special environment (such as night and rainy days), the sensor performance data includes the signal validity and noise impact of the sensor.

[0102] Step S3013, constructing a weight matrix based on the driving scenario label and sensor performance data.

[0103] Specifically, the driving scene labels are used as rows in the weight matrix, and the sensor types are used as columns in the weight matrix. The weight matrix is ​​defined as:

[0104]

[0105] Each row represents a specific driving scenario label, including smooth urban trunk roads during sunny daytime, congested urban commercial areas during rainy nights, urban peak sections with traffic accidents in foggy mornings, small roads around farmland during sunny daytime, rural trunk roads with heavy rain at night, smooth rural roads on sunny mornings, smooth highway main roads during sunny daytime, slightly congested highway main roads during sunny daytime, and highway main roads with multiple vehicles accidents during heavy rain at night; each column represents the contribution ratio of a specific sensor, including LiDAR W LiDAR 、Camera W Camera 、Millimeter wave radar W Radar and ultrasonic sensor W Ultrasonic .

[0106] Furthermore, the calculation formula of the weight constraint is as follows:

[0107] ∑ sensor W sensor =1,W sensor ∈[0,1] (8)

[0108] Among them, ∑ sensor represents the sum of the sensor weights, W sensor Represents the weight of the sensor.

[0109] Step S3014, adding random disturbances to the weight matrix to generate multiple groups of weight combinations, and storing the multiple groups of weight combinations according to driving scene labels.

[0110] Specifically, the weight combination corresponding to each driving scene label L(S) is initialized to obtain the initial weight W sensor , the initial weight is the average weight. sensor On this basis, random disturbance ΔW is added to generate multiple sets of weight combinations. The calculation formula for generating weight combinations is as follows:

[0111] W s ' ensor =W sensor +ΔW, ΔW~u(-∈,∈) (9)

[0112] Among them, W s ' ensor represents the weight combination of the corresponding driving scene label, and u(-∈,∈) represents the range of random disturbance.

[0113] Furthermore, the above weight combinations are stored as weight libraries according to driving scenario labels. The weight combination storage format Weight_Record is as follows:

[0114] Weight_Record = {L(S),W LiDAR ,W Camera,W Radar ,W Ultrasonic} (10)

[0115] For example, the weight combination storage format for a highway that is unobstructed during the daytime on a sunny day is:

[0116] {Sunny day, smooth main highway, 0.4, 0.4, 0.1, 0.1}(11)

[0117] Furthermore, the generated weight combinations are analyzed for coverage to ensure that each driving scenario label L(S) has at least k sets of weights and each set of weight combinations meets the diversity requirements and covers possible sensor configurations.

[0118] Step S302: Perform experimental simulation training based on sensor performance data and multiple weight combinations to generate experimental record data. Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0119] Step S303, perform counterfactual simulation on the experimental record data to generate counterfactual simulation record data. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0120] Step S304: Optimize the causal model verification based on the experimental record data and the counterfactual simulation record data to obtain the sensor weight allocation strategy in the autonomous driving scenario. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0121] The method for enhancing the interpretability of autonomous driving scenarios provided in this embodiment constructs a weight matrix through driving scenario labels and sensor performance data; adds random perturbations to the weight matrix to generate multiple sets of weight combinations, and stores the multiple sets of weight combinations according to driving scenario labels, thereby generating multiple sets of weight combinations covering diverse driving scenarios, providing a data basis for analysis and characterization for weight combination optimization and driving scenario adaptation.

[0122] In this embodiment, a method for enhancing the interpretability of an autonomous driving scenario is provided, which can be used in the above-mentioned electronic device. Figure 4 : is a flow chart of an interpretability enhancement method for an autonomous driving scenario according to an embodiment of the present invention. As shown in the figure, the process includes the following steps:

[0123] Step S401, obtaining a driving scene label and sensor performance data corresponding to the driving scene label, and determining multiple weight combinations based on the driving scene label and the sensor performance data. Figure 3 Step S301 of the illustrated embodiment will not be described in detail here.

[0124] Step S402: Performing experimental simulation training based on sensor performance data and multiple weight combinations to generate experimental record data.

[0125] Specifically, for each set of experimental scenarios and weight combinations, all algorithms in the above algorithm library are run one by one, and the vehicle driving state, environmental characteristics and obstacles are simulated based on the simulation environment to obtain experimental record data.

[0126] The above step S402 includes:

[0127] Step S4021, setting a simulated driving scenario, and selecting a current weight combination based on a driving scenario label corresponding to the simulated driving scenario.

[0128] Specifically, experimental scenarios are selected according to the driving scenario labels, and the experimental scenarios include: urban scenarios: smooth urban main roads during sunny daytimes, congested urban commercial areas during rainy nights, and urban peak hours with traffic accidents in foggy mornings; rural scenarios: small roads around farmlands during sunny daytimes, rural main roads during heavy rain at night, and smooth rural roads in zero mornings; highway scenarios: smooth highway main roads during sunny daytimes, highway main roads with slight traffic jams during sunny daytimes, and highway main roads with multiple vehicle accidents on heavy rain nights.

[0129] Furthermore, the weight combination W corresponding to each experimental scenario is obtained according to the driving scenario label corresponding to the experimental scenario, ensuring a comprehensive analysis of all possible weight combinations.

[0130] Step S4022: Perform multimodal data fusion on the sensor performance data corresponding to the simulated driving scenario to obtain a fused feature vector.

[0131] Furthermore, the fusion layer module obtains various sensor performance data in the simulated driving scenario, which is expressed as:

[0132] X={D LiDAR ,D Camera ,D Radar ,D Ultrasonic} (12)

[0133] Among them, X is the sensor performance data, D LiDAR is the laser radar point cloud data, D Camera is the camera image data, D Radar is the millimeter wave radar detection data, D Ultrasonic It is the ultrasonic sensor data.

[0134] Furthermore, after aligning the timestamps of different sensors, the data of various sensors are normalized and spliced, and the normalized data are spliced ​​into a fused feature vector. The fused feature vector can be expressed as:

[0135] X′=[D′ LiDAR ,D′Camera ,D′ Radar ,D′ Ultrasonic ] (13)

[0136] Among them, D′ LiDAR represents the normalized lidar point cloud data, D′ Camera represents the normalized camera image data, D′ Radar represents the normalized millimeter-wave radar detection data, D′ Ultrasonic Represents the normalized ultrasonic sensor data.

[0137] Step S4023, selecting the optimal algorithm based on the simulated driving scenario and the fused feature vector.

[0138] Specifically, the algorithm selection module selects a suitable algorithm category through rules or heuristic methods according to the simulated driving scenario and the fusion feature vector X′ output by the fusion layer module, that is, selects the optimal algorithm among the rule-controlled algorithm, deep learning algorithm or reinforcement learning algorithm.

[0139] Furthermore, the rule control algorithm makes decisions based on preset logical conditions or thresholds, such as vehicle speed and obstacle distance, and is suitable for stable scenarios such as highways. When the simulated driving scenario is a stable scenario such as "clear highway main road during sunny daytime", the algorithm receives the fusion feature vector X' output by the fusion layer module and performs rule-based logical judgment, and its expression is as follows:

[0140] if d obstacle <threshold,then a=0,δ=0 (14)

[0141] Among them, d obstacle is the obstacle distance, threshold is the threshold, a is the acceleration, and δ is the steering angle.

[0142] Furthermore, the deep learning algorithm uses deep neural networks, such as convolutional neural networks or recurrent neural networks, to process complex and unstructured data and is suitable for dynamic interactive scenarios. When the simulated driving scenario is "complex dynamic scenarios such as congested urban commercial areas on rainy nights", the algorithm receives the fused feature vector X' of the fusion layer module, and inputs the fused feature vector X' into a deep learning model, such as ResNet (Residual Network) or LSTM (Long Short-Term Memory), to predict and output obstacle avoidance decision instructions.

[0143] Furthermore, the reinforcement learning algorithm is used to achieve intelligent obstacle avoidance through interactive learning optimization strategies in rare or unknown environments. When the simulated driving scenario is "unknown or special scenarios such as multi-vehicle accidents on highways at night with heavy rain", the algorithm combines the current environment state s and the reward function R(s,a), and uses algorithms such as deep Q learning or policy gradient to optimize the strategy π(a|s) to dynamically learn and make decisions based on the fusion feature vector X′ of the fusion layer module.

[0144] Step S4024, based on the fused feature vector and the current weight combination, an optimal algorithm is used to generate obstacle avoidance decision instructions and performance indicators.

[0145] Specifically, the current weight combination W represents the contribution ratio of different sensors in the fusion feature vector X′. The expression of the current weight combination is as follows:

[0146] W={W LiDAR ,W Camera ,W Radar ,W Ultrasonic} (15)

[0147] Furthermore, for each current weight combination W, all algorithms in the fusion layer module, algorithm selection module, and algorithm execution module in the algorithm library are run one by one, the fusion feature vector X′ and the current weight combination W are input into the algorithm library, and the obstacle avoidance decision instruction u={a,δ} and performance indicators are output. The performance indicators include the obstacle avoidance success rate P success , path smoothness S path , Decision delay T decision and path deviation D deviation , where the obstacle avoidance success rate P success The calculation formula is as follows:

[0148]

[0149] Among them, N success is the number of successful obstacle avoidance, N total is the total number of experiments.

[0150] Path smoothness S path The calculation formula is as follows:

[0151]

[0152] Among them, δ i is the steering angle corresponding to the i-th segment in the path, δ i-1 is the steering angle corresponding to the i-1th segment in the path.

[0153] Decision delay T decision The calculation formula is as follows:

[0154] Tdecision =t output -t input (18)

[0155] Among them, t output is the decision instruction output time, t input Accept the fusion feature vector moment; path deviation D deviation The calculation formula is as follows:

[0156]

[0157] Among them, p i For the test path, is the target path.

[0158] Step S4025, determining the experimental record data based on the simulated driving scenario, the current weight combination, the obstacle avoidance decision instruction and the performance index.

[0159] Specifically, each run result is stored in a standardized format, and the experimental record data Record is:

[0160] Record={L(S),X′,W,u,P success ,S path ,T decision ,D deviation} (20)

[0161] Step S403, performing counterfactual simulation on the experimental record data to generate counterfactual simulation record data.

[0162] Specifically, the above step S403 includes:

[0163] Step S4031, preprocessing the experimental record data, and creating a counterfactual scenario based on the preprocessed experimental record data.

[0164] Specifically, data are grouped according to the simulated driving scenarios and the current weight combination W in the experimental record data, data points with extreme deviations are filtered, and the performance indicators are normalized to ensure data consistency.

[0165] Furthermore, counterfactual simulation includes: counterfactual scenario design: creating multiple hypothetical counterfactual scenarios L′(S) for simulated driving scenarios; simulating environmental changes: such as severe weather such as heavy rain and dense fog, and changes in lighting conditions; simulating sensor performance degradation: such as sensor failure, increased noise, and data delay.

[0166] Furthermore, assuming that the simulated driving scene in the experimental recording data is a scene of a main road in a city on a sunny day, the lidar-related assumptions include: signal weakening: simulating the decrease in the density of the lidar point cloud to generate L′(S); noise increase: introducing random noise into the point cloud data to generate L″(S); viewing angle reduction: limiting the viewing angle range of the lidar (such as 120° in front) to generate L (3) (S); Camera-related assumptions include: Resolution reduction: The simulated image resolution is reduced to generate L (4) (S); Motion blur: simulates the motion blur caused by camera shaking and generates L (5) (S); Lighting changes: simulate changes in light intensity (such as strong light or shadow interference) to generate L (6) (S); millimeter-wave radar related assumptions include: Signal loss: simulate the millimeter-wave radar for a short period of time to generate L (7) (S), distance error: Introducing distance measurement error to generate L (8) (S); Assumptions related to ultrasonic sensors include: Short-distance obstacle detection anomaly: simulate obstacle detection too early or too late, generate L (9) (S).

[0167] Furthermore, assuming that the simulated driving scene in the experimental recording data is a rural main road scene on a rainy night, the lidar-related assumptions include: Raindrop interference: simulated raindrops cause uneven density of lidar point clouds, generating L (10) (S), distance measurement is unstable: dynamic distance measurement error is introduced to generate L (11) (S); Camera-related assumptions include: Image blur: simulate rain attachment to cause blur, generate L (12) (S), Image noise: simulates the increase in image noise under low light conditions, generates an increase in image noise under low light conditions L (13) (S); millimeter wave radar related assumptions include: multi-target interference: simulating multiple obstacles to produce signal overlap and generate L (14) (S); Assumptions related to ultrasonic sensors include: Rain echo interference: simulate false detection caused by raindrop echo signals to generate L (14) (S).

[0168] Furthermore, assuming that the simulated driving scene in the experimental record data is a traffic accident on a highway on a foggy morning, the lidar-related assumptions include: fog Attenuation: simulates the attenuation of point cloud data intensity and generates L (15) (S); Camera-related assumptions include: Image clarity reduction: simulated haze causes the image contrast to decrease, generating L (16) (S); millimeter wave radar related assumptions include: measurement offset: simulate the signal strength fluctuations caused by haze conditions to generate L (17) (S).

[0169] Step S4032, adjusting the multiple weight combinations to obtain the oscillation sensor weight combination.

[0170] Specifically, multiple weight combinations are adjusted to obtain the following calculation formula for the weight combination of the oscillation sensor:

[0171] W′=W+ΔW, ΔW~u(-∈,∈) (21)

[0172] Wherein, W′ is the weight combination of the oscillation sensor.

[0173] Step S4033, generating a simulated obstacle avoidance decision and a simulated index based on the counterfactual scenario and the oscillation sensor weight combination.

[0174] Specifically, the algorithm library is run under each counterfactual scenario L′(S) and oscillation sensor weight combination W′ to generate simulated obstacle avoidance decisions u′={a′,δ′} and simulated indicators.

[0175] Step S4034, determining counterfactual simulation record data based on the counterfactual scenario, the oscillation sensor weight combination, the simulated obstacle avoidance decision and the simulation index.

[0176] Specifically, the counterfactual simulation record data is expressed as:

[0177] Record'={L′(S),W′,u′,P s ' uccess ,S′ path ,T d ' ecision ,D′ deviation ,P c ' ollision} (twenty two)

[0178] Among them, Record' is the counterfactual simulation record data.

[0179] Step S404: Optimize the causal model verification based on the experimental record data and the counterfactual simulation record data to obtain the sensor weight allocation strategy in the autonomous driving scenario. Figure 3 Step S304 of the illustrated embodiment will not be described in detail here.

[0180] The method for enhancing the interpretability of autonomous driving scenarios provided in this embodiment sets a simulated driving scenario, selects the current weight combination based on the driving scenario label corresponding to the simulated driving scenario, and then experimentally simulates the performance of the vehicle under different driving scenarios and current weight combinations one by one, analyzes the impact of driving scenarios, sensor performance data and weight combinations on obstacle avoidance decision instructions and performance indicators, and provides basic data support for optimizing causal models; by determining counterfactual simulation record data based on counterfactual scenarios, oscillating sensor weight combinations, simulated obstacle avoidance decisions and simulated indicators, it reveals the impact of counterfactual scenarios and oscillating sensor weight combinations on obstacle avoidance decision instructions and performance indicators, laying a foundation for the subsequent optimization of causal models.

[0181] In this embodiment, a method for enhancing the interpretability of an autonomous driving scenario is provided, which can be used in the above-mentioned electronic device. Figure 5 is a flow chart of a method for enhancing the interpretability of an autonomous driving scenario according to an embodiment of the present invention. Figure 5 As shown, the process includes the following steps:

[0182] Step S501: Obtain a driving scene label and sensor performance data corresponding to the driving scene label, and determine multiple weight combinations based on the driving scene label and the sensor performance data. Figure 4 Step S401 of the illustrated embodiment will not be described in detail here.

[0183] Step S502: Perform experimental simulation training based on sensor performance data and multiple weight combinations to generate experimental record data. Figure 4 Step S402 of the illustrated embodiment will not be described in detail here.

[0184] Step S503, perform counterfactual simulation on the experimental record data to generate counterfactual simulation record data. Figure 4 Step S403 of the illustrated embodiment will not be described in detail here.

[0185] Step S504, performing causal model verification optimization based on the experimental record data and the counterfactual simulation record data to obtain a sensor weight allocation strategy in the autonomous driving scenario.

[0186] Specifically, the sensor weight allocation strategy in the autonomous driving scenario is iteratively verified, and the sensor weight allocation strategy is adjusted based on the iterative verification result.

[0187] Specifically, the above step S504 includes:

[0188] Step S5041, constructing a causal model; the causal model is composed of a variable set and a causal path.

[0189] Specifically, the causal model is used to represent the relationship between driving scenarios, sensor performance data, weight combinations, and decision performance. The expression of the causal model is as follows:

[0190] G=(V,E) (23)

[0191] Among them, V is a variable set, E is a causal path; the variable set includes the driving scene label L(S), the fusion feature vector X′, the weight combination W, the decision instruction u, and the obstacle avoidance success rate P success Path S path Smoothness, decision delay T decision and path deviation D deviation ; A causal path refers to a specific route or path consisting of a series of events, processes or mechanisms from cause to effect in a causal relationship, which describes the specific mode of action and transmission link between cause and effect; for example, the expression of a causal path is: L(S)→P sensor →W→P success , P sensor The causal path indicates that the driving scene conditions affect the sensor performance, and the sensor performance affects the obstacle avoidance success rate through weight adjustment.

[0192] Step S5042, verifying the causal model based on the experimental record data and the counterfactual simulation record data to obtain the verification deviation.

[0193] Specifically, the experimental records and counterfactual simulation records are compared to verify whether the causal path is accurate. The verification deviation includes obstacle avoidance success rate deviation and decision delay deviation.

[0194] Furthermore, the calculation formula of the obstacle avoidance success rate deviation is as follows:

[0195] ΔP success =P s ' uccess -P success (twenty four)

[0196] Among them, P s ' uccess is the obstacle avoidance success rate obtained by counterfactual simulation.

[0197] Furthermore, the calculation formula of decision delay deviation is as follows:

[0198] ΔT decision =T d ' ecision -T decision (25)

[0199] Among them, T d ' ecision is the decision delay bias obtained from the counterfactual simulation.

[0200] Furthermore, if the causal path verification fails, the deviation is recorded and the adjustment suggestions are analyzed, and the variables or causal paths that need to be optimized are marked.

[0201] Step S5043: Adjust the sensor weight distribution based on the verification deviation to obtain the sensor weight distribution strategy in the autonomous driving scenario.

[0202] Specifically, the weight distribution is adjusted according to the verification deviation. The optimized weight combination must meet the normalization condition. The calculation formula of the optimized weight combination is as follows:

[0203]

[0204] in, is the optimized weight combination, and k is the adjustment coefficient.

[0205] Furthermore, for each driving scene label L(S), an optimized weight allocation rule is generated. The optimized weight allocation rule is the relationship between scene-sensor performance-weight allocation. The optimized weight allocation rule is:

[0206] W optimized =f(L(S),P sensor ) (27)

[0207] Among them, W optimized is the optimized weight combination.

[0208] Furthermore, the optimized weight combination can also be expressed as:

[0209] W optimized = {W LiDAR ,W Camera ,W Radar ,W Ultrasonic} (28)

[0210] Furthermore, the optimized weight configuration is applied to new counterfactual simulations to verify the optimization effect and further refine the rules.

[0211] Furthermore, the optimized causal model G′(V,E′), the weight allocation rules corresponding to each scenario, and the performance comparison report before and after model optimization are output. The optimized causal path diagram is also displayed, showing the correlation strength between optimized variables, the impact of the weight adjustment process on performance indicators, and the scenario-weight allocation relationship diagram, which can ultimately intuitively display the sensor weight contribution of each driving scenario.

[0212] The method for enhancing the interpretability of the autonomous driving scenario provided in this embodiment adjusts the sensor weight distribution by verifying the deviation, and obtains the sensor weight distribution strategy in the autonomous driving scenario, so that each adjustment of the sensor weight has a clear basis and logical chain, and the decision-making process of each step can be clearly traced, which significantly improves the transparency of the autonomous driving system, avoids the unknowability and potential risks brought by the black box algorithm, improves the safety and credibility of the autonomous driving system, and can provide developers, operators and end users with more intuitive and easy-to-understand decision-making basis when making key decisions and handling abnormal driving scenarios.

[0213] The following is a specific example to illustrate the specific steps of a runoff forecasting method based on data statistical analysis.

[0214] Embodiment 1:

[0215] Taking the scenario of “congested urban commercial district on rainy night” (L(S)) as an example, the simulation of counterfactual scenarios and the verification process are as follows:

[0216] The weight combination corresponding to the simulation experiment scene with the driving scene label of a congested urban commercial district on a rainy night is: W = {W LiDAR =0.2,W Camera =0.4,W Radar =0.3,W Ultrasonic =0.1}; in the performance index, the obstacle avoidance success rate is P success =85%, path smoothness is S path = 0.15 and the decision delay is T decision =180ms.

[0217] Assumption 1: If the camera resolution is reduced by 50%, the corresponding camera weight is reduced to (W′ Camera =0.2), the weight of millimeter wave radar increases to (W′ Radar =0.5);

[0218] Assumption 2: When the laser radar noise increases, the signal stability decreases, and the corresponding laser radar weight changes from W LiDAR =0.2 reduced to W′ LiDAR =0.1.

[0219] Counterfactual simulation data record: Assumption 1: Obstacle avoidance success rate P′ success =80%; Decision delay T′ decision =200ms; collision rate increases to P' collision =20%; Assumption 2: Obstacle avoidance success rate P′ success = 82%, path smoothness S′ path =0.18.

[0220] Optimize and adjust the weight combination of the oscillation sensor: During the weight optimization process, increase the weight of the millimeter wave radar to W Radar =0.6, and reduce the camera weight to W Camera =0.2.

[0221] Performance improvement after optimization: obstacle avoidance success rate Decision delay The collision rate is reduced to

[0222] The above simulation experiments can clarify: 1) Clarify the role of sensors in the scene: In rainy night scenes, millimeter-wave radar has significant advantages in penetrating rain and fog. Counterfactual simulation is used to clarify its contribution to the obstacle avoidance success rate and collision rate; 2) Optimize the weight allocation rules, adjust the weight combination through simulation, form scene optimization rules, ensure that the weight allocation meets the actual needs of sensor performance, and avoid waste of resources; 3) Build data-driven decision support, based on counterfactual simulation and experimental data, establish a causal model between scene-weight-performance, and improve the practical application capability of the autonomous driving obstacle avoidance system.

[0223] Embodiment 2:

[0224] like Figure 6 As shown, the system for realizing the explanation enhancement of the autonomous driving scenario includes:

[0225] 1) The data acquisition module includes a multi-sensor data acquisition module and a sensor health monitoring module; the multi-sensor data acquisition module is responsible for collecting various sensor data from inside and outside the vehicle, including cameras, radars, lidars and GPS; the sensor health monitoring module is used to monitor the working status of each sensor in real time, detect whether there are abnormalities in the data or sensor failures, so as to switch the data source or make corrections when the sensor fails.

[0226] 2) The data preprocessing and anomaly detection module includes a data cleaning and standardization module and an abnormal data detection module; the data cleaning and standardization module is used to clean the collected raw data, remove noise and erroneous data, and uniformly format the data from different sensors to ensure the accuracy of subsequent analysis; the abnormal data detection module is used to detect abnormal values ​​in sensor data, such as data errors caused by sensor failure or environmental factors, and perform correction or weighted processing.

[0227] 3) The data fusion and explainability analysis module includes a multimodal data fusion module and a decision explainability module; the multimodal data fusion module is used to combine data from different sensors to perform feature extraction and data fusion to form a unified perception result. During the data fusion process, the above module can explain the contribution of each sensor data in the final decision; the decision explainability module is used to display the impact of different sensor data on the final driving decision in detail through a causal reasoning model, and provide a transparent decision-making process, especially the decision-making basis when the data is abnormal or the sensor fails.

[0228] 4) The scene recognition and decision-making module includes an environmental perception module and a decision reasoning module; the environmental perception module is used to perceive the environment based on multi-source data, including the identification of information such as roads, obstacles, pedestrians, and traffic signs, to form a dynamic environmental model; the decision reasoning module is used to make driving decisions based on sensor data and environmental information, and output an explainable report of the decision basis and various judgment parameters to help understand the decision logic of the system.

[0229] 5) The sensor fault and redundancy management module includes a sensor fault detection and repair module and a redundant sensor management module; the sensor fault detection and repair module is used to detect whether each sensor has a fault or data anomaly, automatically switch to a redundant sensor or repair solution, and fill in the missing data through weighted averaging, interpolation and other methods; the redundant sensor management module is used to manage redundant sensors (such as multiple cameras, radars, etc.) to ensure seamless switching when a sensor fails to avoid affecting the decision-making process.

[0230] 6) The user interaction and feedback module includes a visualization module and a feedback collection and learning module; the visualization module is used to provide the driver or system operator with visualized decision-making basis and sensor data contribution so as to understand how the system makes the final decision; the feedback collection and learning module optimizes the decision model through feedback from the driver and passengers, so that the system can gradually improve its accuracy and reliability.

[0231] 7) The data storage and security module includes a data storage module and a privacy protection module; the data storage module is used to store sensor data, decision logs, abnormal data, etc., and ensure the integrity and timeliness of the data; the privacy protection module is used to ensure that user data (such as vehicle location, driving behavior, etc.) is encrypted during the upload and storage process to prevent leakage.

[0232] 8) The cloud synchronization and update module includes a real-time cloud synchronization module and a model update and optimization module; the real-time cloud synchronization module is used to upload the vehicle's real-time decision data, sensor status, anomaly detection results and other information to the cloud for big data analysis and model training; the model update and optimization module is used to regularly update the algorithm model based on feedback data from the cloud and newly collected driving data to improve the system's adaptability and performance.

[0233] In the above embodiments, the "black box" problem in the existing autonomous driving system is solved by dynamically evaluating and optimizing the performance of sensors in different driving scenarios, and the interpretability and transparency of the sensor fusion process are achieved; specifically, by introducing a scene perception mechanism, combined with real-time sensor performance, the configuration and weight distribution between sensors are flexibly adjusted, thereby ensuring that the system can select the most appropriate sensor combination under different environmental conditions. This mechanism not only improves the perception accuracy of the autonomous driving system, but also enhances the system's adaptability in complex dynamic scenarios, ensuring the reliability and accuracy of the decision-making process. In addition, the interpretability feature enables each adjustment of the sensor weight to have a clear basis and logical chain, and can clearly trace the decision-making process of each step, significantly improving the transparency of the system, and avoiding the unknowability and potential risks brought by traditional black box algorithms. This technical gain improves the safety and credibility of the autonomous driving system, especially when dealing with key decisions and abnormal scenarios, and can provide developers, operators and end users with more intuitive and easy-to-understand decision-making basis. Finally, autonomous driving systems usually make decisions based on fixed sensor configurations or preset weight distributions. When faced with complex and dynamic driving environments, they are often unable to fully respond to sudden changes or fluctuations in sensor performance. Therefore, the scenario-based dynamic baseline sensor weight configuration, through real-time recognition of different driving scenarios and comprehensive evaluation of sensor data, can achieve automatic selection of the optimal sensor combination under different environmental conditions, or provide a combination with better average performance. For example, in rainy, snowy weather or low visibility conditions, radar or infrared sensors may be more dependent, while in clear weather, visual sensors may provide more decision-making information, thereby ensuring that the perception system always operates in the best state, that is, quantifying the above-mentioned subjective feelings or fuzzy black box points.

[0234] In this embodiment, an explanatory enhancement device for an autonomous driving scenario is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. As used below, the term "module" may be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0235] This embodiment provides an explanatory enhancement device for an autonomous driving scenario, such as Figure 7 As shown, including:

[0236] The determination module 701 is used to obtain a driving scene label and sensor performance data corresponding to the driving scene label, and determine a plurality of weight combinations based on the driving scene label and the sensor performance data.

[0237] The training module 702 is used to perform experimental simulation training based on sensor performance data and multiple weight combinations to generate experimental record data.

[0238] The simulation module 703 is used to perform counterfactual simulation on the experimental record data to generate counterfactual simulation record data.

[0239] The optimization module 704 is used to perform causal model verification optimization based on the experimental record data and the counterfactual simulation record data to obtain the sensor weight allocation strategy in the autonomous driving scenario.

[0240] In some optional implementations, the determining module 701 includes:

[0241] The labeling unit is used to collect the positioning information, driving environment characteristics and weather conditions of the target vehicle, and classify and label the positioning information, driving environment characteristics and weather conditions to obtain multiple driving scene labels.

[0242] The collection unit is used to collect sensor performance data corresponding to multiple driving scene labels respectively.

[0243] The first construction unit is used to construct a weight matrix based on the driving scene label and the sensor performance data.

[0244] An adding unit is used to add random perturbations to the weight matrix, generate multiple sets of weight combinations, and store the multiple sets of weight combinations according to driving scenario labels.

[0245] In some optional implementations, the training module 702 includes:

[0246] The setting unit is used to set the simulated driving scene and select the current weight combination based on the driving scene label corresponding to the simulated driving scene.

[0247] The fusion unit is used to perform multi-modal data fusion on the sensor performance data corresponding to the simulated driving scene to obtain a fused feature vector.

[0248] The selection unit is used to select the optimal algorithm based on the simulated driving scenario and the fused feature vector.

[0249] The first generating unit is used to generate obstacle avoidance decision instructions and performance indicators by using an optimal algorithm based on the fused feature vector and the current weight combination.

[0250] The first determination unit is used to determine the experimental record data based on the simulated driving scene, the current weight combination, the obstacle avoidance decision instruction and the performance index.

[0251] In some optional implementations, the simulation module 703 includes:

[0252] The preprocessing unit is used to perform data preprocessing on the experimental record data and create a counterfactual scenario based on the experimental record data after the data preprocessing.

[0253] The first adjustment unit is used to adjust the multiple weight combinations to obtain the oscillation sensor weight combination.

[0254] The second generation unit generates simulated obstacle avoidance decisions and simulation indicators based on the counterfactual scenario and the oscillation sensor weight combination.

[0255] The second determination unit determines the counterfactual simulation record data based on the counterfactual scenario, the oscillation sensor weight combination, the simulated obstacle avoidance decision and the simulation index.

[0256] In some optional implementations, the optimization module 704 includes:

[0257] The second construction unit is used to construct a causal model; the causal model is composed of a variable set and a causal path.

[0258] The verification unit is used to verify the causal model based on the experimental record data and the counterfactual simulation record data to obtain the verification deviation.

[0259] The second adjustment unit is used to adjust the sensor weight distribution based on the verification deviation to obtain a sensor weight distribution strategy in the autonomous driving scenario.

[0260] In some optional implementations, the optimization module 704 is further used to iteratively verify the sensor weight allocation strategy in the autonomous driving scenario, and adjust the sensor weight allocation strategy based on the iterative verification result.

[0261] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0262] The explanatory enhancement device for the autonomous driving scenario in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above-mentioned functions.

[0263] The embodiment of the present invention also provides a computer device having the above Figure 7Explanatory augmentation device for the autonomous driving scenario shown.

[0264] See also Figure 8 , Figure 8 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 8 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 10 is taken as an example.

[0265] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0266] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0267] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0268] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0269] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0270] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0271] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0272] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for enhancing the interpretability of an autonomous driving scenario, characterized in that: The method comprises: Acquire a driving scenario label and sensor performance data corresponding to the driving scenario label, and determine a plurality of weight combinations based on the driving scenario label and the sensor performance data; Performing experimental simulation training based on the sensor performance data and the multiple weight combinations to generate experimental record data; Performing counterfactual simulation on the experimental record data to generate counterfactual simulation record data; Causal model verification and optimization are performed based on the experimental recorded data and the counterfactual simulation recorded data to obtain a sensor weight allocation strategy in the autonomous driving scenario.

2. The method according to claim 1, characterized in that The acquired driving scene labels and sensor performance data corresponding to the driving scene labels are determined based on the driving scene labels and the sensor performance data to obtain a plurality of weight combinations, including: Collecting the positioning information, driving environment characteristics and weather conditions of the target vehicle, and classifying and labeling the positioning information, the driving environment characteristics and the weather conditions to obtain a plurality of driving scene labels; respectively collecting sensor performance data corresponding to a plurality of driving scene labels; Constructing a weight matrix based on the driving scenario label and the sensor performance data; Random disturbances are added to the weight matrix to generate multiple groups of weight combinations, and the multiple groups of weight combinations are stored according to the driving scene labels.

3. The method according to claim 1, characterized in that: The performing experimental simulation training based on the sensor performance data and the multiple weight combinations to generate experimental record data includes: Setting a simulated driving scenario, and selecting a current weight combination based on a driving scenario label corresponding to the simulated driving scenario; Performing multimodal data fusion on the sensor performance data corresponding to the simulated driving scenario to obtain a fused feature vector; Selecting an optimal algorithm based on the simulated driving scenario and the fused feature vector; Based on the fused feature vector and the current weight combination, using the optimal algorithm to generate obstacle avoidance decision instructions and performance indicators; The experimental record data is determined based on the simulated driving scenario, the current weight combination, the obstacle avoidance decision instruction and the performance indicator.

4. The method according to claim 1, characterized in that The performing counterfactual simulation on the experimental record data to generate counterfactual simulation record data includes: Performing data preprocessing on the experimental record data, and creating a counterfactual scenario based on the experimental record data after the data preprocessing; Adjusting the multiple weight combinations to obtain a shock sensor weight combination; generating a simulated obstacle avoidance decision and a simulated index based on the counterfactual scenario and the oscillation sensor weight combination; The counterfactual simulation record data is determined based on the counterfactual scenario, the oscillation sensor weight combination, the simulated obstacle avoidance decision and the simulation index.

5. The method according to claim 1, characterized in that The causal model verification optimization is performed based on the experimental recorded data and the counterfactual simulation recorded data to obtain a sensor weight allocation strategy in the autonomous driving scenario, including: Constructing a causal model; the causal model is composed of a variable set and a causal path; Verifying the causal model based on the experimental record data and the counterfactual simulation record data to obtain a verification deviation; The sensor weight allocation is adjusted based on the verification deviation to obtain a sensor weight allocation strategy in the autonomous driving scenario.

6. The method according to claim 5, characterized in that The causal model verification optimization is performed based on the experimental recorded data and the counterfactual simulation recorded data to obtain a sensor weight allocation strategy in the autonomous driving scenario, and further includes: Iteratively verify the sensor weight allocation strategy in the autonomous driving scenario, and adjust the sensor weight allocation strategy based on the iterative verification result.

7. An interpretability enhancement device for an autonomous driving scenario, characterized in that: The device comprises: a determination module, configured to obtain a driving scenario label and sensor performance data corresponding to the driving scenario label, and determine a plurality of weight combinations based on the driving scenario label and the sensor performance data; A training module, used for performing experimental simulation training based on the sensor performance data and the multiple weight combinations to generate experimental record data; A simulation module, used for performing counterfactual simulation on the experimental record data to generate counterfactual simulation record data; The optimization module is used to perform causal model verification optimization based on the experimental record data and the counterfactual simulation record data to obtain a sensor weight allocation strategy in the autonomous driving scenario.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for enhancing the interpretability of the autonomous driving scenario according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which are used to enable a computer to execute the method for enhancing the interpretability of the autonomous driving scenario according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the method for enhancing the interpretability of an autonomous driving scenario according to any one of claims 1 to 6.

Images