Safety comprehensive evaluation method and device of automatic driving system and related equipment

Abstract

The invention provides a safety comprehensive evaluation method and device of an automatic driving system and related equipment, and relates to the technical field of automatic driving evaluation, and the method comprises the steps: obtaining time sequence data; noise information and the time sequence data serve as system input, and interference testing is conducted on the automatic driving system; fault information serves as system input, and fault recovery testing is conducted on the automatic driving system; taking confrontation information and the time sequence data as system input, and carrying out confrontation attack test on the automatic driving system; the information output by the interference test, the information output by the fault recovery test and the information output by the anti-attack test are analyzed, evaluation information is obtained, the evaluation information comprises a safety score, and the higher the safety score is, the higher the safety of the automatic driving system is. According to the invention, multi-dimensional safety assessment is carried out on the automatic driving system, so that the finally output safety assessment information is more accurate and reliable.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving evaluation, and particularly to a comprehensive safety evaluation method, device and related equipment for an autonomous driving system. Background Art

[0002] With the rapid development of autonomous driving technology, safety has become one of the most critical issues in this field. The autonomous driving system relies on the collaborative work of multiple technologies such as sensors, decision algorithms, and control modules. Therefore, in the face of various complex environments and abnormal situations, the robustness and safety of the system have received extensive attention. Currently, the safety evaluation of autonomous driving systems mainly focuses on traditional functional verification, ignoring the protection ability of the system against interference and malicious attacks in a dynamic environment.

[0003] Currently, many studies focus on the single-function testing and system performance optimization of autonomous driving systems, especially in the case of a higher environmental complexity. Some previous studies have proposed robustness testing methods based on deep learning, but most methods only consider input perturbations and do not fully consider the impact of adversarial attacks. In addition, most traditional safety evaluation methods do not comprehensively consider the coping strategies after system failure and the real-time response to malicious attacks.

[0004] Some previous studies only focus on the impact of environmental perturbations on the system and ignore the protection against adversarial attacks.

[0005] That is to say, based on the existing solutions for the safety evaluation of autonomous driving systems, the reliability of the final evaluation results is poor. Summary of the Invention

[0006] The purpose of the present invention is to provide a comprehensive safety evaluation method, device and related equipment for an autonomous driving system, which is used to solve the technical problem of poor reliability of the evaluation results existing in the prior art in the safety evaluation of autonomous driving systems.

[0007] In a first aspect, an embodiment of the present invention provides a comprehensive safety evaluation method for an autonomous driving system, and the method includes:

[0008] Obtain time-series data, where each data point in the time-series data is obtained by fusing multiple sensor data corresponding to a corresponding time point, and the multiple sensor data correspond one by one to multiple vehicle-mounted sensors associated with the autonomous driving system;

[0009] Using the noise information and the timing data as system inputs, perform an interference test on the autonomous driving system to obtain first test information, where the noise perturbation data is used to represent the external interference corresponding to the autonomous driving system; the first test information is used to represent the sensitivity of the autonomous driving system to the external interference;

[0010] Using the fault information as a system input, perform a fault recovery test on the autonomous driving system to obtain second test information, where the fault information is used to represent the fault problems corresponding to the autonomous driving system; the second test information is used to represent the recovery probability of the autonomous driving system after encountering the fault problems;

[0011] Using the adversarial information and the timing data as system inputs, perform an adversarial attack test on the autonomous driving system to obtain third test information, where the adversarial information is used to represent the malicious attacks corresponding to the autonomous driving system; the third test information is used to represent the defense performance of the autonomous driving system against the malicious attacks;

[0012] Analyze the first test information, the second test information, and the third test information to obtain evaluation information, where the evaluation information includes a safety score, and the higher the safety score, the higher the safety of the autonomous driving system.

[0013] In a second aspect, an embodiment of the present invention further provides a comprehensive safety evaluation device for an autonomous driving system, where the device includes:

[0014] A data acquisition module, configured to acquire timing data, where each data point in the timing data is obtained by fusing multiple sensor data corresponding to a corresponding time point, and the multiple sensor data correspond one-to-one to multiple vehicle-mounted sensors associated with the autonomous driving system;

[0015] An interference test module, configured to use the noise information and the timing data as system inputs to perform an interference test on the autonomous driving system to obtain first test information, where the noise perturbation data is used to represent the external interference corresponding to the autonomous driving system; the first test information is used to represent the sensitivity of the autonomous driving system to the external interference;

[0016] A fault test module, configured to use the fault information as a system input to perform a fault recovery test on the autonomous driving system to obtain second test information, where the fault information is used to represent the fault problems corresponding to the autonomous driving system; the second test information is used to represent the recovery probability of the autonomous driving system after encountering the fault problems;

[0017] An adversarial testing module, configured to use adversarial information and the timing data as system inputs to perform an adversarial attack test on the autonomous driving system, obtaining third test information, where the adversarial information is used to represent: a malicious attack corresponding to the autonomous driving system; and the third test information is used to represent: the defense performance of the autonomous driving system against the malicious attack.

[0018] A security evaluation module, configured to analyze the first test information, the second test information, and the third test information to obtain evaluation information, where the evaluation information includes: a security score, and the higher the security score, the higher the security of the autonomous driving system.

[0019] In a third aspect, an embodiment of the present invention further provides an electronic device, including a processor, a memory, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, the steps of the above-mentioned comprehensive security evaluation method for an autonomous driving system are implemented.

[0020] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned comprehensive security evaluation method for an autonomous driving system are implemented.

[0021] In the embodiments of the present invention, by separately performing interference testing, fault recovery testing, and adversarial testing on the autonomous driving system, the robustness of the autonomous driving system when encountering external interference, the recovery performance after encountering a fault problem, and the defense performance when encountering a malicious attack are respectively evaluated; and then through comprehensive analysis of the three, a multi-dimensional security evaluation of the autonomous driving system is realized, which can make the finally output evaluation information more accurate and reliable. Description of the Drawings

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments of the present invention. Obviously, the following described drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0023] Figure 1 is a flowchart of a comprehensive security evaluation method for an autonomous driving system provided by an embodiment of the present invention;

[0024] Figure 2 is a schematic structural diagram of a comprehensive security evaluation device for an autonomous driving system provided by an embodiment of the present invention;

[0025] Figure 3It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0026] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0027] An embodiment of the present invention provides a comprehensive safety evaluation method for an autonomous driving system, as Figure 1 shown, the method includes:

[0028] Step 101, obtain time-series data.

[0029] Among them, each data point in the time-series data is obtained by fusing multiple sensor data corresponding to a corresponding time point, and the multiple sensor data correspond one-to-one to multiple vehicle-mounted sensors associated with the autonomous driving system.

[0030] It should be noted that the time-series data corresponds to a continuous period of time (such as a period of time with a set duration in a historical period), the continuous time includes multiple time points distributed at equal intervals, and the multiple time points correspond one-to-one to multiple data points in the time-series data.

[0031] Exemplarily, the multiple vehicle-mounted sensors include but are not limited to: environmental perception sensors (such as cameras, lidars, millimeter-wave radars), vehicle state sensors (such as wheel speed sensors, accelerometers, gyroscopes), and power system sensors (such as oxygen sensors, knock sensors, temperature sensors).

[0032] The autonomous driving system can be understood as: a complex system that realizes the vehicle's autonomous perception of the environment, decision-making, and execution without human intervention through the coordinated work of sensors, algorithms, and control systems. The corresponding levels range from low to high as L0-L5. L0 means that the autonomous driving system only has basic warning functions (such as tire pressure monitoring), and L5 means that the autonomous driving system can support driverless driving in any scenario.

[0033] Step 102, use the noise information and the time-series data as system inputs to perform an interference test on the autonomous driving system to obtain first test information.

[0034] Among them, the noise perturbation data is used to represent: the external interference corresponding to the autonomous driving system; the first test information is used to represent: the sensitivity of the autonomous driving system to the external interference.

[0035] Exemplarily, the external interferences include: environmental changes (such as heavy rain, heavy snow, thick fog, dust, strong light, etc.), electromagnetic interferences (such as signal distortion or failure caused by the influence of devices such as base stations and high-voltage power lines), and sensor contamination (such as the camera being blocked).

[0036] Step 103: Use the fault information as the system input to perform a fault recovery test on the autonomous driving system, and obtain second test information.

[0037] Among them, the fault information is used to represent: the fault problem corresponding to the autonomous driving system; the second test information is used to represent: the recovery probability of the autonomous driving system after encountering the fault problem.

[0038] Exemplarily, the fault problems include: equipment aging, inaccurate calibration of sensor parameters, equipment failures, etc.

[0039] Step 104: Use the adversarial information and the timing data as the system input to perform an adversarial attack test on the autonomous driving system, and obtain third test information.

[0040] Among them, the adversarial information is used to represent: the malicious attack corresponding to the autonomous driving system; the third test information is used to represent: the defense performance of the autonomous driving system against the malicious attack.

[0041] Exemplarily, the malicious attacks include: malicious identification setting (such as pasting a specially designed pattern on a traffic sign or the road surface to deceive the neural network of the camera into misidentifying), laser attack (such as using a high-power laser emitter to emit a laser signal towards the in-vehicle camera to cause the in-vehicle camera to overexpose or hardware damage), and noise attack (such as emitting ultrasonic noise to interfere with the ranging work of the ultrasonic radar).

[0042] Step 105: Analyze the first test information, the second test information, and the third test information to obtain evaluation information.

[0043] The evaluation information includes: a safety score, and the higher the safety score, the higher the safety of the autonomous driving system.

[0044] In the embodiment of the present invention, by respectively performing an interference test, a fault recovery test, and an adversarial test on the autonomous driving system, to respectively evaluate the robustness of the autonomous driving system when encountering external interferences, the recovery performance after encountering fault problems, and the defense performance when encountering malicious attacks; and then through comprehensive analysis of the three, to achieve a multi-dimensional safety evaluation of the autonomous driving system, which can make the finally output evaluation information more accurate and reliable.

[0045] In one embodiment, the obtaining of the timing data corresponding to continuous time includes:

[0046] Obtain multiple groups of original data, where the multiple groups of original data correspond one-to-one to multiple time points of the time series data, and each group of original data includes multiple sensor data corresponding to the corresponding time point. The sensor data is data obtained based on the corresponding vehicle-mounted sensors, and / or the sensor data is data obtained based on the Lagrange interpolation algorithm;

[0047] Among the multiple groups of original data, perform normalization processing on the multiple sensor data included in each group of original data to obtain multiple groups of normalized data. Among them, the multiple groups of normalized data correspond one-to-one to the multiple groups of original data. The normalized data includes multiple normalized values, and the multiple normalized values correspond one-to-one to the multiple sensor data of the corresponding original data. Moreover, different normalized values included in the same normalized data correspond to the same numerical interval;

[0048] Among the multiple groups of normalized data, perform weighted calculation on the multiple normalized values included in each group of normalized data to obtain the time series data.

[0049] It should be noted that the normalization processing methods corresponding to different sensor data can be the same or different. The same numerical interval corresponding to different normalized values can be the interval [0, 1].

[0050] Among them, the normalized value corresponding to the lidar can be used to represent: the point cloud distribution and the distance between point clouds represented by the point cloud data collected by the lidar (which can be completed by global coordinate normalization or spherical coordinate normalization); the normalized value corresponding to the camera can be used to represent: the pixel characteristics or intensity of the image data collected by the camera (which can be completed by mean standard deviation normalization); the normalized value corresponding to the millimeter-wave radar can be used to represent: indicators such as distance, speed, and reflection intensity included in the electromagnetic wave data collected by the millimeter-wave radar.

[0051] Exemplarily, the multiple groups of normalized data can be expressed as:

[0052] X t =[x 1,t ,x 2,t ,…,x n,t T

[0053] In the above formula, x i,t represents the eigenvalue (i.e., the normalized value) corresponding to the data observed by the i-th vehicle-mounted sensor (i = 1, 2, …, n) at time t.

[0054] The process of performing weighted calculation on the multiple normalized values included in each group of normalized data among the multiple groups of normalized data can be expressed as:

[0055] Among them ​

[0056] In the above formula, w i (t) represents the weight coefficient corresponding to the i-th vehicle-mounted sensor at time t, which can be determined based on the data fluctuation of the i-th vehicle-mounted sensor within a set time period before time t. If the data fluctuation is larger, the corresponding weight coefficient is smaller; X fused (t) represents the time-series data.

[0057] The weight coefficient w i (t) here can be determined by a dynamic adaptive method, and its mathematical definition is as follows: First, define the observation variance of the i-th vehicle-mounted sensor within a past time window as Then the information weight w i (t) of the i-th sensor at the current moment is normalized through the information entropy function:

[0058]

[0059] Among them, E i (t) is the information entropy measure defined based on the variance of the sensor data The sensor data with a higher information entropy indicates stable data quality, smaller variance, and a relatively higher weight coefficient, thus contributing more to the fused data. On the contrary, the variance of the sensor data with a lower information entropy is larger, with strong volatility, and the weight is correspondingly reduced, which can avoid the negative impact of abnormal data of individual sensors on the fusion result.

[0060] Among them, the data obtained based on the corresponding vehicle-mounted sensor can be understood as: the point cloud data collected by the lidar, the image data collected by the camera, the electromagnetic wave data collected by the millimeter-wave radar, etc.; the data obtained based on the Lagrange interpolation algorithm can be understood as: in the case of missing or significantly abnormal data obtained based on the corresponding vehicle-mounted sensor, the data obtained by filling in the missing data through the Lagrange interpolation algorithm to overcome the data missing problem caused by occlusion, packet loss, software and hardware failures, etc., and ensure the integrity and continuity of the data.

[0061] The Lagrange interpolation algorithm selects the previous and subsequent m + 1 known data points {(t j , y j )}, and the corresponding interpolation function can be:

[0062]

[0063] In the above formula, L(t) represents the data predicted and compensated based on the high-order Lagrange interpolation algorithm at time t.

[0064] In this embodiment, for different types of data collected by different vehicle-mounted sensors, by normalizing them to the same numerical range, the format of different types of data collected by different vehicle-mounted sensors is unified, so as to avoid the situation that some sensor data overly affects subsequent processing due to format differences, and that some sensor data is not fully utilized, making the evaluation information formed thereby more accurate and reliable. Among them, the weighted processing is to combine multiple sensor data into a higher-dimensional data, which can not only reduce the amount of data to be processed subsequently, but also reduce the mixing of interference, so as to focus on the processing of key information in the data, making the finally output evaluation information more accurate.

[0065] In one embodiment, in the multiple groups of normalized data, performing weighted calculation on multiple normalized values included in each group of normalized data to obtain the time series data, including:

[0066] In the multiple groups of normalized data, respectively performing standardization processing on multiple normalized values included in each group of normalized data to obtain multiple groups of standard data, where the multiple groups of standard data and the multiple groups of normalized data correspond one by one, the standard data includes multiple standard values, the multiple standard values and the multiple normalized values of the corresponding normalized data correspond one by one, and the standard value is obtained by performing Z-Score standardization processing on the corresponding normalized value;

[0067] In the multiple groups of standard data, performing weighted calculation on multiple standard values included in each group of standard data to obtain the time series data, the weight coefficient of the standard value is determined according to the corresponding vehicle-mounted sensor, and the smaller the data fluctuation of the vehicle-mounted sensor in the historical period, the higher the corresponding weight coefficient.

[0068] Exemplarily, the Z-Score standardization processing formula can be expressed as:

[0069]

[0070] In the above formula, z i,t represents the standard value (dimensionless data) corresponding to the data observed by the i-th vehicle-mounted sensor at time t, μ i represents the mean value of multiple historical data of the i-th vehicle-mounted sensor in the historical period before time t, and σ i represents the standard deviation of multiple historical data of the i-th vehicle-mounted sensor in the historical period before time t.

[0071] It should be noted that the weight coefficient of the standard value refers to the relevant description of the aforementioned w i (t). To avoid repetition, it will not be elaborated here.

[0072] In this embodiment, in addition to performing data normalization based on the data characteristics of each vehicle-mounted sensor (which refers to establishing a mapping relationship between the data range of a certain sensor and the aforementioned numerical range), data standardization is further performed according to the different data characteristics of different vehicle-mounted sensors (establishing a mapping relationship between the different data ranges of multiple sensors and the aforementioned numerical range) to obtain dimensionless standard values, fully eliminating the data scale differences between different sensors, ensuring the scale consistency of the fused data, and making the finally output evaluation information more accurate.

[0073] Based on the coordinated settings of the above normalization, standardization, and weighted calculation and other measures, the present invention can accurately reflect the safety performance and risk exposure degree of the autonomous driving system in a real and complex traffic environment, providing key basic data support for the quantitative evaluation of the safety of the autonomous driving system.

[0074] In one embodiment, using the noise information and the timing data as system inputs to perform an interference test on the autonomous driving system to obtain first test information, including:

[0075] Based on the multiple noise data included in the noise information and the timing data, generate multiple noise input data. The multiple noise data follow a normal distribution, and the multiple noise input data correspond one-to-one with the multiple noise data. The noise input data is used to represent the timing data with the corresponding noise data added;

[0076] According to the multiple noise input data and the timing data, generate multiple robustness values. The multiple robustness values correspond one-to-one with the multiple noise input data. The robustness value is the ratio of the output norm to the input norm of the corresponding noise input data. The output norm is used to represent the norm of the difference between the noise mapping value and the non-noise mapping value. The noise mapping value is the mapping value of the corresponding noise input data in the autonomous driving system, and the non-noise mapping is the mapping value of the timing data in the autonomous driving system; the input norm is the norm of the timing data;

[0077] Calculate the probability distribution characteristics of the multiple robustness values, and form the first test information based on the probability distribution characteristics. Among them, the probability distribution characteristics include an expected value. The larger the expected value, the higher the sensitivity of the autonomous driving system to the external interference.

[0078] In this embodiment, based on the multiple noise data to simulate the random errors that may be introduced by the environment and sensors of the autonomous driving system, and restricting the multiple noise data to follow a normal distribution (which can also be understood as Gaussian noise), unifying the complex perturbation factors as zero-mean random noise can simplify the analysis and help evaluate the robustness of the autonomous driving system.

[0079] Exemplarily, the original input corresponding to the time series data can be expressed as The perturbation input corresponding to the noise input data can be expressed as x ′ , x ′ = x + σ·z, where the perturbation terms corresponding to multiple noise data can be expressed σ is the noise standard deviation, and its magnitude directly determines the expected perturbation intensity: when σ is small, only weak noise is introduced; while when σ is large, a more significant random offset will be generated to the input.

[0080] If the mapping function of the autonomous driving system is set as Under the original input x, its output is y = f(x), while under the perturbation input x ′ = x + σ·z, its output becomes y ′ = f(x + σ·z). At this time, the robustness value can be expressed as R, and R is calculated through the following formula:

[0081]

[0082] In the above formula, ‖·‖ p and ‖·‖ q are the selected p-norm and q-norm respectively (usually p = q = 2), used to measure the normalized relationship between the output deviation and the input scale, and ‖f(x + σ·z) - f(x)‖ p can be understood as the aforementioned output norm, and ‖x‖ q can be understood as the aforementioned input norm. This index reflects the degree of relative deviation caused by a small change in the input to the output of the autonomous driving system. The smaller R is, the stronger the system's resistance to perturbations; on the contrary, the larger R is, the more sensitive the system is to perturbations.

[0083] Since the perturbation term δ is essentially a random variable, the output f(x + σ·z) also has uncertainty. To comprehensively evaluate the sensitivity of the autonomous driving system to perturbations, the Monte Carlo simulation method can be used for statistical analysis: under the preset noise standard deviation σ, from N samples are independently sampled Calculate the corresponding robustness index (which can be understood as the aforementioned multiple robustness values)

[0084]

[0085] Based on these samples, the expected value and variance of the robustness index can be further calculated (which can be understood as the aforementioned probability distribution characteristics of the multiple robustness values), and are respectively defined as

[0086]

[0087] where reflects the typical magnitude of the system output deviation at a given noise level, while Var(R) characterizes the stability of the output response. Based on the above statistics, a threshold R can also be set τ as a robustness criterion: when the system is considered to have sufficient robustness at this perturbation level; if it indicates that the system is too sensitive to the perturbation and does not meet the expected robustness standard.

[0088] It should be noted that the selection of the noise standard deviation σ needs to balance the noise level that may be encountered in practical applications and the model performance. Although a larger σ can more strictly test the system robustness limit, it may also introduce extreme perturbations beyond the scope of the real scenario, resulting in a significant deviation of the model output from the normal level.

[0089] In this embodiment, by introducing Gaussian noise to uniformly perturb the input for modeling and using the Monte Carlo statistical method to quantitatively analyze the output response, the ability of the system to resist random perturbations can be comprehensively and accurately evaluated.

[0090] In one embodiment, taking the fault information as the system input, the fault recovery test is performed on the autonomous driving system, and the second test information is obtained, including:

[0091] Determine the fault factors based on the fault information, where the fault information includes: the performance degradation probability distribution corresponding to each vehicle-mounted sensor among multiple vehicle-mounted sensors; the fault factors are used to represent: the fault probability of each vehicle-mounted sensor among multiple vehicle-mounted sensors under the fault problem;

[0092] Determine the time factor, effectiveness factor, and redundancy factor based on the autonomous driving system, where the time factor is used to represent: the response time of the backup mechanism of the autonomous driving system under the corresponding fault problem; the effectiveness factor is used to represent: the effectiveness degree of the backup mechanism of the autonomous driving system under the corresponding fault problem; the redundancy factor is used to represent: the redundancy degree corresponding to the multiple vehicle-mounted sensors;

[0093] Analyze the fault factors, the time factor, the effectiveness factor, and the redundancy factor to obtain the second test information.

[0094] Among them, the backup mechanism can be implemented through the data collected by redundant sensors or the backup positioning algorithm, etc.

[0095] Exemplarily, the process of determining the fault factors based on the fault information can be:

[0096] Using the Monte Carlo simulation algorithm, a large number of random samples are generated by random sampling from a preset probability distribution of sensor performance degradation (which can be understood as the probability distribution of performance degradation corresponding to the aforementioned vehicle-mounted sensors). These random samples represent different degrees of accuracy decline or data loss that the sensors may experience.

[0097] Specifically, let the probability density function of the sensor performance index S (such as the accuracy of lidar point cloud or the clarity of camera images) be f(S), and its cumulative distribution function is defined as:

[0098] where and φ(s)=0. By the inverse transform sampling method, using the random variable U uniformly drawn from the interval [0,1] i (i.e., an equivalent sampling formula can be constructed as follows:

[0099]

[0100] Thus, the simulated sensor state S in the i-th simulation can be obtained i . In a large number of N repeated simulations, using the indicator function to represent sensor failure (i.e., taking the value 1 when S < S th and 0 otherwise), the failure probability of each type of sensor in the system is:

[0101]

[0102] In the above formula, P fail can be understood as the aforementioned failure factor.

[0103] Exemplarily, the process of determining the time factor, effectiveness factor, and redundancy factor based on the autonomous driving system can be as follows:

[0104] Define the system safety recovery function as:

[0105]

[0106] In the above formula, represents the exponential decay effect of the standby mechanism response time on the recovery ability (which can be understood as the aforementioned time factor), can be understood as the aforementioned effectiveness factor - used to represent the reliability and precision of the standby mechanism, and Φ(n,R) describes the comprehensive effectiveness of the sensor redundancy design (its specific form depends on the sensor configuration and layout, which can be understood as the aforementioned redundancy factor), n represents the number of sensors, R represents the redundancy structure, and the integral part in the parentheses uses the gamma function relationship to ensure that its value is always equal to 1, thus not changing the physical meaning of the overall expression.

[0107] The recovery probability of the autonomous driving system after encountering the fault problem can be expressed as:

[0108]

[0109] In the above formula, t r,i represents the response time of the backup mechanism of the system after a sensor fault in the i-th simulation, and P safe The higher the value, the higher the recovery probability of the autonomous driving system after encountering the fault problem, that is, it indicates that the system can enable the backup mechanism more timely and effectively after encountering a fault, so as to maintain a safe operating state.

[0110] In this embodiment, the fault factor is used as a negative impact factor, and the time factor, effectiveness factor, and redundancy factor are used as positive impact factors. Considering comprehensively the response time of the backup mechanism after the autonomous driving system encounters a sensor fault, the availability of the corresponding algorithm of the backup mechanism, and the richness of the corresponding entity structure of the backup mechanism, and comprehensively considering the probability of the autonomous driving system recovering safety, so as to comprehensively determine the recovery probability of the autonomous driving system after encountering the fault problem from multiple aspects, and ensure the data reliability of the obtained second test information.

[0111] In one embodiment, using the adversarial information and the timing data as system inputs, an adversarial attack test is performed on the autonomous driving system to obtain third test information, including:

[0112] Based on the timing data and the deep neural network model included in the autonomous driving system, initial information is generated, where the non-adversarial information includes: the output corresponding to the timing data in the deep neural network model and model parameters;

[0113] Based on the Fast Gradient Sign Method (FGSM), the initial information and the adversarial information are fused to obtain adversarial data;

[0114] The adversarial data is processed by the deep neural network model to obtain an adversarial output;

[0115] Analyze the difference between the adversarial output and the output corresponding to the timing data in the deep neural network model to obtain the third test information.

[0116] Among them, the deep neural network model included in the autonomous driving system is a composite model composed of one or more models with deep neural network structures. The models with deep neural network structures can be used for: environmental perception (such as identifying obstacles), semantic segmentation (identifying lane lines), spatial positioning (positioning the vehicle in three-dimensional space based on point cloud data), autonomous driving strategy output (such as autonomous driving route output, vehicle control output, emergency mechanism output, etc.).

[0117] Exemplarily, in the deep neural network model included in the autonomous driving system, for the model part included therein for environmental perception (used to identify whether there are obstacles in a certain frame of image and set corresponding labels based on the recognition result), the original sample (i.e., the model input) corresponding to the time series data can be defined as The target label is (which can be understood as the output corresponding to the aforementioned time series data in the deep neural network model), and the model loss function is denoted as J(θ, x, y), where θ represents the model parameters. Based on the Fast Gradient Sign Method (FGSM), the generation formula of adversarial samples is further complicated on the basis of the original formula as follows:

[0118]

[0119] In the above formula, ∈ is the perturbation intensity control parameter, e i represents the standard basis vector, ∈ and e i are both included in the adversarial information, and the above limit operation ensures the precise definition of the sign function on each component, thereby ensuring that the generated adversarial sample x adv (which can be understood as the aforementioned adversarial data) can induce the model to produce misjudgments during the perception or decision-making stage.

[0120] Based on the original sample x, a series of adversarial sample sets with different perturbation intensities are generated using the above formula and they are respectively input into the aforementioned deep neural network model to record the model output results for comparative analysis, that is, to compare the differences between the model outputs of the original sample and the adversarial samples. The greater the difference between the two, the worse the defense performance of the autonomous driving system against the malicious attack, and vice versa.

[0121] In this embodiment, based on the construction of the Fast Gradient Sign Method and the corresponding formula, the fine construction of adversarial samples is realized, so as to better simulate the malicious attack behaviors that may be encountered in the application of the autonomous driving system, and further ensure that the third test information obtained accordingly is more accurate and reliable.

[0122] In one embodiment, the analyzing the difference between the adversarial output and the output corresponding to the time series data in the deep neural network model to obtain the third test information includes:

[0123] Taking the output corresponding to the timing data in the deep neural network model as the reference object and the adversarial output as the object to be processed, calculate the misclassification rate and the execution failure rate, where the misclassification rate is used to represent: the proportion of the part of the object to be processed that is different from the reference object; the execution failure rate is used to represent: the proportion of the part of the object to be processed that matches the abnormal execution situation;

[0124] Perform a weighted calculation on the misclassification rate and the execution failure rate to obtain the third test information.

[0125] Exemplarily, the misclassification rate (Misclassification Rate, MR) can be expressed as:

[0126]

[0127] In the above formula, f(·) is the model mapping function, y (i) is the correct label of the i-th sample, and δ(a, b) represents the Kronecker δ function, which is defined as taking 1 when a = b and 0 otherwise; this formula is used to count the proportion of incorrect labels output by the system under adversarial attacks.

[0128] The execution failure rate (Execution Failure Rate, EFR) is used to measure the abnormal situation of decision execution by the system (specifically, the model part corresponding to the output of the autonomous driving strategy in the system) under adversarial perturbations, and its expression can be defined as

[0129]

[0130] where ζ(·) is a function indicating normal system execution. If the system can correctly trigger the emergency or execution mechanism at the input then otherwise it is 0.

[0131] In this case, the defense performance of the autonomous driving system against the malicious attack can be expressed as R adv R adv is also called the attack-resistant robustness index, and this index can be defined as

[0132]

[0133] where α and β are adjustable weight parameters, satisfying α + β = 1, used to reflect the relative contributions of the misclassification rate and the execution failure rate in the evaluation of the attack-resistant robustness index; R advThe closer the value is to 1, the better the stability of the algorithm under adversarial attacks and the stronger its robustness. Conversely, it indicates that the system has significant vulnerabilities under malicious perturbations.

[0134] In this embodiment, based on the above settings, it is possible to quantitatively represent the defense performance of the autonomous driving system against the malicious attack, and at the same time systematically reveal the potential safety hazards of the autonomous driving system when facing tiny maliciously designed perturbations, providing a clear and accurate theoretical basis for identifying the weak links of the system and optimizing and improving it.

[0135] In one embodiment, the first test information, the second test information, and the third test information are analyzed to obtain evaluation information, including:

[0136] Analyze the first test information, the second test information, and the third test information through the analytic hierarchy process to obtain the evaluation information.

[0137] In one example, the first test information includes at least the robustness index (R), the second test information includes at least the fail-safe probability (P safe ), and the third test information includes at least the adversarial attack anomaly rate (M). The weight coefficients of R, P safe and M are determined to be w_1, w_2, and w_3 respectively by the analytic hierarchy process (AHP). Then the safety score (S total ) has the following mathematical expression:

[0138] First, determine the weight coefficients through the analytic hierarchy process, and define the pairwise comparison matrix A:

[0139]

[0140] Next, perform normalization processing on the matrix A to calculate the weight coefficients w i :

[0141]

[0142] After obtaining w_1, w_2, and w_3 through the above formula, calculate the overall system safety score S total :

[0143] S total = w1·R + w2·P safe + w3·(1 - M)

[0144] Among them, R represents the robustness score measured for the autonomous driving system in different complex traffic emergency scenarios (such as pedestrians crossing the road, sudden braking of vehicles, and interference from other vehicles) in the aforementioned simulation environment. The specific calculation of R can be found in the previous description; P safeIt represents the probability P of triggering the backup mechanism in time to maintain safe driving when a critical sensor (such as lidar or camera) in the system fails. safe For the specific calculation of safe , refer to the foregoing description; while the adversarial attack anomaly rate M is the probability that the system becomes abnormal after generating adversarial samples through the Fast Gradient Sign Method (FGSM) algorithm and attacking the system in the previous step. For the specific calculation of M, refer to the foregoing description (obtained by weighted calculation of MR and EFR).

[0145] In the application, to ensure that the constructed judgment matrix A meets the consistency requirement, a consistency check can be further carried out. Let the maximum eigenvalue of matrix A be λ max , which satisfies the characteristic equation:

[0146] Aw = λ max w, where w = (w1, w2, w3) T

[0147] The consistency index CI is defined as:

[0148]

[0149] Meanwhile, a consistency ratio CR is introduced to further evaluate the consistency of the judgment matrix:

[0150]

[0151] Among them, RI is the random consistency index at the corresponding order (usually when n = 3, RI ≈ 0.58). When CR < 0.1, it is considered that the consistency of the judgment matrix is reasonable, thus ensuring that the calculated weight vector has a high reliability.

[0152] In this embodiment, based on the Analytic Hierarchy Process, the safety assessment is carried out from multiple dimensions, which can combine quantitative and qualitative ideas, clearly list the specific numerical indicators obtained in each safety assessment stage, including the specific scenarios and scores of the robustness test, the safety recovery probabilities in the case of various sensor failures, as well as the misclassification rate and execution failure rate in the adversarial attack scenario, and clarify the contribution of each indicator to the overall score. Specifically, the importance degree of the indicators is reflected by the weight coefficient, so as to further reveal the weak links and risk areas of the safety of the autonomous driving system while evaluating the overall safety of the autonomous driving system.

[0153] It can be seen that based on the settings of the above scheme, the present invention can more comprehensively capture various complex situations that the autonomous driving system may encounter in actual applications, thereby providing a more accurate basis for the safety verification and optimization of the system.

[0154] See Figure 2 , Figure 2 is a comprehensive safety assessment device for an autonomous driving system provided by an embodiment of the present invention, asFigure 2 As shown in the figure, the comprehensive safety evaluation device 200 of the autonomous driving system includes:

[0155] A data acquisition module 201, configured to acquire time-series data, wherein each data point in the time-series data is obtained by fusing multiple sensor data corresponding to a corresponding time point, and the multiple sensor data correspond one by one to multiple vehicle-mounted sensors associated with the autonomous driving system;

[0156] An interference test module 202, configured to use noise information and the time-series data as system inputs to perform an interference test on the autonomous driving system to obtain first test information, wherein the noise perturbation data is used to represent: the external interference corresponding to the autonomous driving system; the first test information is used to represent: the sensitivity of the autonomous driving system to the external interference;

[0157] A fault test module 203, configured to use fault information as a system input to perform a fault recovery test on the autonomous driving system to obtain second test information, wherein the fault information is used to represent: the fault problems corresponding to the autonomous driving system; the second test information is used to represent: the recovery probability of the autonomous driving system after encountering the fault problems;

[0158] An adversarial test module 204, configured to use adversarial information and the time-series data as system inputs to perform an adversarial attack test on the autonomous driving system to obtain third test information, wherein the adversarial information is used to represent: the malicious attacks corresponding to the autonomous driving system; the third test information is used to represent: the defense performance of the autonomous driving system against the malicious attacks;

[0159] A safety evaluation module 205, configured to analyze the first test information, the second test information, and the third test information to obtain evaluation information, and the evaluation information includes: a safety score, and the higher the safety score, the higher the safety of the autonomous driving system.

[0160] In one embodiment, the data acquisition module 201 includes:

[0161] A data acquisition unit, configured to acquire multiple groups of original data, wherein the multiple groups of original data correspond one by one to multiple time points of the time-series data, and each group of original data includes multiple sensor data corresponding to a corresponding time point, and the sensor data is data obtained based on a corresponding vehicle-mounted sensor, and / or, the sensor data is data obtained based on the Lagrange interpolation algorithm;

[0162] A data normalization unit, which is used to perform normalization processing on multiple sensor data included in each group of original data in the multiple groups of original data, so as to obtain multiple groups of normalized data. Among them, the multiple groups of normalized data and the multiple groups of original data correspond one by one. The normalized data includes multiple normalized values, and the multiple normalized values and the multiple sensor data of the corresponding original data correspond one by one. Moreover, different normalized values included in the same normalized data correspond to the same numerical range;

[0163] A data weighting unit, which is used to perform weighted calculation on multiple normalized values included in each group of normalized data in the multiple groups of normalized data to obtain the time series data.

[0164] In one embodiment, the data weighting unit is specifically used for:

[0165] In the multiple groups of normalized data, perform standardization processing on multiple normalized values included in each group of normalized data to obtain multiple groups of standard data. Among them, the multiple groups of standard data and the multiple groups of normalized data correspond one by one. The standard data includes multiple standard values, and the multiple standard values and the multiple normalized values of the corresponding normalized data correspond one by one. The standard value is obtained by performing Z-Score standardization processing on the corresponding normalized value;

[0166] In the multiple groups of standard data, perform weighted calculation on multiple standard values included in each group of standard data to obtain the time series data. The weight coefficient of the standard value is determined according to the corresponding vehicle-mounted sensor. The smaller the data fluctuation of the vehicle-mounted sensor in the historical period, the higher the corresponding weight coefficient.

[0167] In one embodiment, the interference test module 202 is specifically used for:

[0168] Based on the multiple noise data included in the noise information and the time series data, generate multiple noise input data. The multiple noise data follow a normal distribution, and the multiple noise input data and the multiple noise data correspond one by one. The noise input data is used to represent the time series data with the corresponding noise data added;

[0169] According to the multiple noise input data and the time series data, generate multiple robustness values. The multiple robustness values and the multiple noise input data correspond one by one. The robustness value is the ratio of the output norm to the input norm of the corresponding noise input data. The output norm is used to represent the norm of the difference between the noise mapping value and the non-noise mapping value. The noise mapping value is the mapping value of the corresponding noise input data in the autonomous driving system, and the non-noise mapping is the mapping value of the time series data in the autonomous driving system; The input norm is the norm of the time series data;

[0170] Calculate the probability distribution characteristics of the multiple robust values, and form the first test information based on the probability distribution characteristics, where the probability distribution characteristics include an expected value, and the greater the expected value, the higher the sensitivity of the autonomous driving system to the external interference.

[0171] In one embodiment, the fault test module 203 is specifically configured to:

[0172] Determine a fault factor based on the fault information, where the fault information includes: the performance degradation probability distribution corresponding to each vehicle-mounted sensor among multiple vehicle-mounted sensors; the fault factor is used to represent: the fault probability of each vehicle-mounted sensor among multiple vehicle-mounted sensors under the fault problem;

[0173] Based on the determination time factor, effectiveness factor, and redundancy factor of the autonomous driving system, where the time factor is used to represent: the response time of the backup mechanism of the autonomous driving system under the corresponding fault problem; the effectiveness factor is used to represent: the effectiveness degree of the backup mechanism of the autonomous driving system under the corresponding fault problem; the redundancy factor is used to represent: the redundancy degree corresponding to the multiple vehicle-mounted sensors;

[0174] Analyze the fault factor, the time factor, the effectiveness factor, and the redundancy factor to obtain the second test information.

[0175] In one embodiment, the adversarial test module 204 includes:

[0176] An initial generation unit, configured to generate initial information based on the timing data and the deep neural network model included in the autonomous driving system, where the non-adversarial information includes: the output corresponding to the timing data in the deep neural network model, model parameters;

[0177] An adversarial generation unit, configured to fuse the initial information and the adversarial information based on the fast gradient sign method to obtain adversarial data;

[0178] An adversarial unit, configured to process the adversarial data with the deep neural network model to obtain an adversarial output;

[0179] An adversarial analysis unit, configured to analyze the difference between the adversarial output and the output corresponding to the timing data in the deep neural network model to obtain the third test information.

[0180] In one embodiment, the adversarial analysis unit is specifically configured to:

[0181] Using the output corresponding to the timing data in the deep neural network model as the reference object and the adversarial output as the object to be processed, calculate the misclassification rate and the execution failure rate, where the misclassification rate is used to represent: the proportion of the part in the object to be processed that is different from the reference object; the execution failure rate is used to represent: the proportion of the part in the object to be processed that matches the execution exception situation;

[0182] Perform a weighted calculation on the misclassification rate and the execution failure rate to obtain the third test information.

[0183] In one embodiment, the security evaluation module 205 is specifically configured to:

[0184] Perform hierarchical analysis on the first test information, the second test information, and the third test information to obtain the evaluation information.

[0185] The safety comprehensive evaluation device 200 of the autonomous driving system provided by the embodiments of the present invention can implement each process in the above method embodiments. To avoid repetition, it will not be elaborated here.

[0186] According to the embodiments of the present invention, the present invention also provides an electronic device and a readable storage medium.

[0187] Figure 3 FIG. shows a schematic block diagram of an exemplary electronic device 300 that can be used to implement the embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processing, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0188] As Figure 3 shown, the device 300 includes a computing unit 301, which can execute various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 302 or the computer program loaded from the storage unit 308 into the random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the device 300 can also be stored. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other through a bus 304. The input / output (I / O) interface 305 is also connected to the bus 304.

[0189] Multiple components in device 300 are connected to I / O interface 305, including: input unit 306, such as a keyboard, mouse, etc.; output unit 307, such as various types of displays, speakers, etc.; storage unit 308, such as a disk, optical disc, etc.; and communication unit 309, such as a network card, modem, wireless communication transceiver, etc. Communication unit 309 allows device 300 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0190] Computing unit 301 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing unit 301 include but are not limited to a central processing unit (CPU), a graphic process unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Computing unit 301 executes the various methods and processes described above, such as the comprehensive safety assessment method for an autonomous driving system. For example, in some embodiments, the comprehensive safety assessment method for an autonomous driving system can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by computing unit 301, one or more steps of the comprehensive safety assessment method for the autonomous driving system described above can be executed. Alternatively, in other embodiments, computing unit 301 can be configured to execute the comprehensive safety assessment method for the autonomous driving system in any other suitable way (e.g., by means of firmware).

[0191] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on chip (SOC) systems, complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0192] The program code for implementing the methods of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0193] In the context of the present invention, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0194] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0195] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0196] A computer system can include a client and a server. The client and the server are generally far from each other and usually interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

[0197] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present invention can be achieved, and no limitation is imposed herein.

[0198] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A comprehensive safety assessment method for an autonomous driving system, characterized in that, The method includes: Obtaining time-series data, where each data point in the time-series data is obtained by fusing multiple sensor data corresponding to a corresponding time point, and the multiple sensor data correspond one-to-one to multiple vehicle-mounted sensors associated with the autonomous driving system; Using the noise information and the time-series data as system inputs to perform an interference test on the autonomous driving system to obtain first test information, where the noise perturbation data is used to represent: the external interference corresponding to the autonomous driving system; the first test information is used to represent: the sensitivity of the autonomous driving system to the external interference; Using the fault information as a system input to perform a fault recovery test on the autonomous driving system to obtain second test information, where the fault information is used to represent: the fault problem corresponding to the autonomous driving system; the second test information is used to represent: the recovery probability of the autonomous driving system after encountering the fault problem; Using the adversarial information and the time-series data as system inputs to perform an adversarial attack test on the autonomous driving system to obtain third test information, where the adversarial information is used to represent: the malicious attack corresponding to the autonomous driving system; the third test information is used to represent: the defense performance of the autonomous driving system against the malicious attack; Analyzing the first test information, the second test information, and the third test information to obtain evaluation information, where the evaluation information includes: a safety score, and the higher the safety score, the higher the safety of the autonomous driving system.

2. The comprehensive security assessment method according to claim 1, wherein The obtaining of the time-series data corresponding to consecutive time includes: Obtaining multiple groups of original data, where the multiple groups of original data correspond one-to-one to multiple time points of the time-series data, and each group of original data includes multiple sensor data corresponding to a corresponding time point, the sensor data is data obtained based on the corresponding vehicle-mounted sensor, and / or, the sensor data is data obtained based on the Lagrange interpolation algorithm; In the multiple groups of original data, respectively performing normalization processing on the multiple sensor data included in each group of original data to obtain multiple groups of normalized data, where the multiple groups of normalized data correspond one-to-one to the multiple groups of original data, the normalized data includes multiple normalized values, the multiple normalized values correspond one-to-one to the multiple sensor data of the corresponding original data, and different normalized values included in the same normalized data correspond to the same numerical interval; In the multiple groups of normalized data, performing weighted calculation on the multiple normalized values included in each group of normalized data to obtain the time-series data.

3. The comprehensive security assessment method according to claim 2, characterized in that, In the multiple groups of normalized data, performing weighted calculation on the multiple normalized values included in each group of normalized data to obtain the time-series data, including: In the multiple groups of normalized data, respectively performing standardization processing on the multiple normalized values included in each group of normalized data to obtain multiple groups of standard data, where the multiple groups of standard data correspond one-to-one to the multiple groups of normalized data, the standard data includes multiple standard values, the multiple standard values correspond one-to-one to the multiple normalized values of the corresponding normalized data, and the standard value is obtained by performing Z-Score standardization processing on the corresponding normalized value; Among the multiple sets of standard data, weighted calculation is performed on the multiple standard values included in each set of standard data to obtain the timing data. The weight coefficient of the standard value is determined according to the corresponding vehicle-mounted sensor. The smaller the data fluctuation of the vehicle-mounted sensor in the historical period, the higher the corresponding weight coefficient.

4. The comprehensive security assessment method according to claim 1, wherein Using the noise information and the timing data as system inputs, an interference test is performed on the autonomous driving system to obtain first test information, including: Based on the multiple noise data included in the noise information and the timing data, multiple noise input data are generated. The multiple noise data follow a normal distribution. The multiple noise input data correspond one-to-one with the multiple noise data. The noise input data is used to represent the timing data with the corresponding noise data added. According to the multiple noise input data and the timing data, multiple robustness values are generated. The multiple robustness values correspond one-to-one with the multiple noise input data. The robustness value is the ratio of the output norm to the input norm of the corresponding noise input data. The output norm is used to represent the norm of the difference between the noise mapping value and the non-noise mapping value. The noise mapping value is the mapping value of the corresponding noise input data in the autonomous driving system. The non-noise mapping is the mapping value of the timing data in the autonomous driving system. The input norm is the norm of the timing data. Calculate the probability distribution characteristics of the multiple robustness values, and form the first test information based on the probability distribution characteristics. Among them, the probability distribution characteristics include an expected value. The larger the expected value, the higher the sensitivity of the autonomous driving system to the external interference.

5. The comprehensive security assessment method according to claim 1, wherein Using the fault information as a system input, a fault recovery test is performed on the autonomous driving system to obtain second test information, including: Determine a fault factor based on the fault information. Among them, the fault information includes the performance degradation probability distribution corresponding to each vehicle-mounted sensor among the multiple vehicle-mounted sensors. The fault factor is used to represent the fault probability of each vehicle-mounted sensor among the multiple vehicle-mounted sensors under the fault problem. Determine a time factor, an effectiveness factor, and a redundancy factor based on the autonomous driving system. Among them, the time factor is used to represent the response time of the backup mechanism of the autonomous driving system under the corresponding fault problem. The effectiveness factor is used to represent the effectiveness degree of the backup mechanism of the autonomous driving system under the corresponding fault problem. The redundancy factor is used to represent the redundancy degree corresponding to the multiple vehicle-mounted sensors. Analyze the fault factor, the time factor, the effectiveness factor, and the redundancy factor to obtain the second test information.

6. The method according to claim 1, wherein Using the adversarial information and the timing data as system inputs, an adversarial attack test is performed on the autonomous driving system to obtain third test information, including: Based on the timing data and the deep neural network model included in the autonomous driving system, initial information is generated. Among them, the non-adversarial information includes the output corresponding to the timing data in the deep neural network model and the model parameters. Fuse the initial information and the adversarial information based on the Fast Gradient Sign Method to obtain adversarial data; Process the adversarial data with the deep neural network model to obtain an adversarial output; Analyze the difference between the adversarial output and the output corresponding to the time series data in the deep neural network model to obtain the third test information.

7. The method according to claim 6, wherein The analyzing the difference between the adversarial output and the output corresponding to the time series data in the deep neural network model to obtain the third test information includes: Using the output corresponding to the time series data in the deep neural network model as the reference object and the adversarial output as the object to be processed, calculate the misclassification rate and the execution failure rate, where the misclassification rate is used to represent: the proportion of the part of the object to be processed that is different from the reference object; the execution failure rate is used to represent: the proportion of the part of the object to be processed that matches the abnormal execution situation; Perform a weighted calculation on the misclassification rate and the execution failure rate to obtain the third test information.

8. The method according to claim 1, characterized in that, Analyze the first test information, the second test information, and the third test information to obtain evaluation information, including: Perform hierarchical analysis on the first test information, the second test information, and the third test information to obtain the evaluation information.

9. An integrated safety assessment device for an autonomous driving system, characterized in that, The device includes: A data acquisition module for acquiring time series data, where each data point in the time series data is obtained by fusing multiple sensor data corresponding to a corresponding time point, and the multiple sensor data correspond one-to-one to multiple vehicle-mounted sensors associated with the autonomous driving system; An interference test module for using noise information and the time series data as system inputs to perform an interference test on the autonomous driving system to obtain first test information, where the noise perturbation data is used to represent: the external interference corresponding to the autonomous driving system; the first test information is used to represent: the sensitivity of the autonomous driving system to the external interference; A fault test module for using fault information as a system input to perform a fault recovery test on the autonomous driving system to obtain second test information, where the fault information is used to represent: the fault problem corresponding to the autonomous driving system; the second test information is used to represent: the recovery probability of the autonomous driving system after encountering the fault problem; An adversarial test module for using adversarial information and the time series data as system inputs to perform an adversarial attack test on the autonomous driving system to obtain third test information, where the adversarial information is used to represent: the malicious attack corresponding to the autonomous driving system; the third test information is used to represent: the defense performance of the autonomous driving system against the malicious attack; A security evaluation module for analyzing the first test information, the second test information, and the third test information to obtain evaluation information, where the evaluation information includes: a security score, and the higher the security score, the higher the security of the autonomous driving system.

10. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 8.

11. A readable storage medium, characterized in that, A computer program is stored on the readable storage medium. When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images