Test apparatus, method, storage medium and electronic device

Abstract

Embodiments of the present disclosure disclose a test device, a method, a storage medium and an electronic device. The test device comprises a processor configured to: obtain a test result; the test result is obtained based on a test case performing a test on an intelligent driving system; based on the test result, determine a software test score and a software event sequence; obtain sensor data corresponding to the test case and a hardware test score; based on the software event sequence and the sensor data, determine an event correlation score; and based on the software test score, the hardware test score and the event correlation score, determine a test score of the intelligent driving system. The test device can improve the accuracy of the test result of the intelligent driving system, while reducing the fault omission rate.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent driving system testing technology, and in particular to a testing apparatus, method, storage medium and electronic device. Background Technology

[0002] With the development of technology, testing intelligent driving systems usually involves several fundamentally different test domains, such as software function test domain and sensor performance test domain.

[0003] Software functional testing primarily focuses on the correctness of the software algorithm logic, the reliability of communication between various modules of the intelligent driving system, and the timing consistency of the intelligent driving system. The test output of software functional testing is a discrete binary result of software test pass or software test failure, error messages, and execution trajectory. Sensor performance testing mainly focuses on the physical perception quality, hardware response performance, and environmental adaptability involved in the intelligent driving system. The test output of sensor performance testing is a continuous physical indicator such as detection confidence, tracking accuracy, end-to-end latency, and signal-to-noise ratio. In related technologies, when testing intelligent driving systems, different test domains are mostly tested independently, which can easily lead to missed faults and reduce the accuracy of the test results.

[0004] Therefore, improving the accuracy of intelligent driving system test results while reducing the rate of missed fault detection is an urgent problem to be solved. Summary of the Invention

[0005] This disclosure provides a testing apparatus, method, storage medium, and electronic device that can improve the accuracy of intelligent driving system test results while reducing the rate of missed fault detection.

[0006] In a first aspect, embodiments of this disclosure provide a testing apparatus, the apparatus comprising: a processor configured to: Obtain test results; the test results are obtained by performing tests on the intelligent driving system based on test cases; Based on the test results, the software test score and software event sequence are determined; Obtain the sensor data and hardware test scores corresponding to the test cases; Based on the software event sequence and the sensor data, an event association score is determined; The test score of the intelligent driving system is determined based on the software test score, the hardware test score, and the event association score.

[0007] Secondly, embodiments of this disclosure provide a testing method, including: Obtain test results; the test results are obtained by performing tests on the intelligent driving system based on test cases; Based on the test results, the software test score and software event sequence are determined; Obtain the sensor data and hardware test scores corresponding to the test cases; Based on the software event sequence and the sensor data, an event association score is determined; The test score of the intelligent driving system is determined based on the software test score, the hardware test score, and the event association score.

[0008] Thirdly, embodiments of this disclosure provide a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the test method as described in the second aspect.

[0009] Fourthly, embodiments of this disclosure provide a computer program product, including a computer program executed by a processor to implement the testing method as described in the second aspect.

[0010] Fifthly, embodiments of this disclosure provide an electronic device, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory, the processor executing the executable instructions to implement the test method as described in the second aspect.

[0011] The testing apparatus of this disclosure determines event correlation scores through software event sequences and sensor data. When determining the test score of the intelligent driving system, it considers not only the software test score and hardware test score, but also the event correlation score between the two. Compared to the existing method of independently testing the software and hardware domains, this approach offers several advantages. First, when the software function test passes but the sensor performance is on the verge of degradation, the event correlation score can capture the abnormal correlation between the software and hardware domains. This avoids directly marking the test case as "passed" and missing "software false pass" faults caused by sensor degradation, significantly reducing the cross-domain fault detection rate and improving the accuracy of the intelligent driving system test results. Second, when software and hardware anomalies are highly coupled in time, the event correlation score can perform correlation analysis between the two, avoiding misjudging independent hardware anomalies as intelligent driving system anomalies, which is beneficial for identifying cross-domain interaction faults. Attached Figure Description

[0012] Figure 1 This is a schematic flowchart illustrating the processor implementation testing process provided in one embodiment of this disclosure; Figure 2 This is a first flowchart illustrating the processor implementation of the event association score determination process according to an embodiment of this disclosure; Figure 3 This is a second flowchart illustrating the processor implementation of the event association score determination process according to an embodiment of this disclosure; Figure 4 This is a schematic flowchart illustrating the processor implementation of the time correlation determination process according to an embodiment of this disclosure; Figure 5 This is a schematic flowchart illustrating the process of determining software test scores implemented by a processor according to an embodiment of this disclosure; Figure 6 This is a schematic flowchart illustrating the processor implementation hardware test score determination process according to an embodiment of this disclosure; Figure 7 This is a first flowchart illustrating the processor implementing a fault analysis process according to an embodiment of the present disclosure; Figure 8 This is a second flowchart illustrating the processor implementing the fault analysis process according to an embodiment of this disclosure; Figure 9 This is a third flowchart illustrating the processor implementing the fault analysis process according to an embodiment of this disclosure; Figure 10 This is a flowchart illustrating a testing method provided in one embodiment of this disclosure; Figure 11 This is a structural diagram of an electronic device provided in one embodiment of the present disclosure. Detailed Implementation

[0013] The features and exemplary embodiments of various aspects of this disclosure will now be described in detail. To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description, in conjunction with the accompanying drawings and specific embodiments, will provide a further detailed description. It should be understood that the specific embodiments described herein are intended only to explain this disclosure and not to limit it. For those skilled in the art, this disclosure can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this disclosure by illustrating examples.

[0014] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0015] Application Overview The testing of intelligent driving systems typically involves several fundamentally different test domains, such as software functional testing and sensor performance testing. Software functional testing mainly focuses on the correctness of the software algorithm logic, the reliability of communication between various modules of the intelligent driving system, and the timing consistency of the intelligent driving system; sensor performance testing mainly focuses on the physical perception quality, hardware response performance, and environmental adaptability of the intelligent driving system.

[0016] In existing technologies, the two test domains mentioned above are typically executed independently by different teams using different toolchains at different testing phases, with test results output separately. Software functional testing results are usually discrete binary results of software test pass or software test failure; sensor performance testing results are usually continuous physical indicators such as detection confidence and signal-to-noise ratio meeting or failing standards. The lack of cross-domain data correlation analysis and fusion analysis of test results between the two leads to the following technical problems: First, when the software function test passes but the sensor performance is on the verge of degradation, since the sensor degradation does not cause an explicit failure at the software logic level, the test case is directly marked as "passed" in the existing technology, creating a hidden danger of "software false pass caused by sensor degradation", which leads to missed detection of cross-domain faults.

[0017] Second, anomalies in either the software function test domain or the sensor performance test domain alone will not lead to serious consequences. However, due to the lack of cross-domain data correlation analysis and fusion analysis of test results, when both anomalies occur simultaneously, system-level coupling risks may arise. Existing technologies cannot automatically identify such cross-domain interaction failures.

[0018] Exemplary device The testing device in this embodiment can be a processor, a chip, or a controller; this disclosure does not limit the specific device used. Figure 1As shown, the testing apparatus includes a processor, which is configured to: S101, obtain test results.

[0019] The test results are obtained by performing tests on the intelligent driving system based on test cases.

[0020] Test cases are pre-written test tasks used to test specific functions of an intelligent driving system. Test cases include preset input conditions and execution steps. During the execution of a test case, the system controls or drives the intelligent driving system based on the preset input conditions and execution steps, and obtains the actual output results generated by the intelligent driving system. The actual output results are compared with the expected output results to determine the test result of the test case.

[0021] The intelligent driving system is the test object, and the system contains software algorithm modules. Sensor hardware components are used to collect environmental perception data (sensor data) in the test scenario, and the hardware test results are obtained from the evaluation results of the sensor data.

[0022] In some embodiments, the processor can read the actual output results after the test case execution platform completes, compare the actual output results with the expected output results, and obtain the test results. It should be noted that the test results can come from data generated by synchronous joint test runs, or from data that has undergone time alignment processing after decoupled testing. The test results originate from the same test run, ensuring consistency in time reference between software events and sensor data in subsequent steps.

[0023] S102, Based on the test results, determine the software test score and software event sequence.

[0024] Software test scores characterize the probability of software failures in intelligent driving systems. For example, the score range can be 0-1. A lower score is output when no software failures occur during the overall software test case execution, and a higher score is output when serious software failures occur.

[0025] A software event sequence is a sequence of key behaviors that occur during software operation, arranged chronologically. Each software event corresponds to an internal execution behavior of the software, such as a state transition, function call, exception reporting, or function output. A state transition indicates that a software module switches between different operating modes, such as switching from standby mode to perception and computing mode. When this switch occurs, the software outputs a corresponding event record, which is the software event corresponding to the state transition. A function call indicates that a software module calls the function interface of other modules during execution; this call triggers an event record, forming the software event corresponding to the function call. An exception reporting behavior indicates that the software actively outputs an exception log event after detecting an internal operational anomaly. A function output behavior refers to the intelligent driving software completing perception, planning, and control algorithm calculations and outputting the algorithm results. This result output action generates a corresponding software event, and each software event carries its own corresponding occurrence time.

[0026] In some embodiments, the processor generates a software event sequence by sorting all software events in chronological order of their actual occurrence times.

[0027] The software test score can be determined based on the first score representing the importance of the module to which the test case belongs to the intelligent driving system, the second score representing the impact of the module to which the test case belongs on the scope of the intelligent driving system fault, and the third score representing the severity level of the intelligent driving system fault. The process of determining the software test score will be described in detail below and will not be elaborated here.

[0028] S103, obtain the sensor data and hardware test score corresponding to the test case.

[0029] Sensor data refers to the physical sampling data acquired by sensor components. Sensor data includes the acquisition time for each sampling action and the physical parameters of the sensing output. These physical parameters may include signal-to-noise ratio, detection confidence level, tracking accuracy, signal delay, etc.

[0030] The hardware test score characterizes the probability of a failure in the sensor's hardware. For example, the hardware test score can range from 0 to 1. It is calculated based on the deviation of the sensor's actual sampling data from a scene-appropriate adaptation threshold. A higher hardware test score indicates a higher probability of a hardware failure.

[0031] It should be noted that software testing and hardware testing can be performed simultaneously or decoupled. The processor extracts sensor data from the test results of the same test run to ensure the consistency and correlation between the sensor data and the software event sequence on the time base.

[0032] S104, based on software event sequences and sensor data, determines event association scores.

[0033] Event correlation score is used to characterize the degree of correlation between software events and sensor data; the event correlation score is determined based on the software time sequence corresponding to the software event sequence and the acquisition time sequence corresponding to the sensor data. For example, the value range of the event correlation score can be 0-1. The higher the event correlation score, the greater the temporal overlap between the key software behavior and the sensor sampling data, and the stronger the temporal coupling effect.

[0034] In some embodiments, the processor extracts the occurrence times corresponding to all software events from the software event sequence to form a software event time sequence. The processor extracts the acquisition time corresponding to each sample from the sensor data to form an acquisition time sequence. The processor sets a time window centered on the occurrence time of each software event, and retrieves sensor sampling data falling within the time window within the acquisition time sequence, generating a set of matching pairs between software events and sensor samples. For each matching pair, the processor calculates the time deviation between the software event occurrence time and the sensor sampling acquisition time. The processor combines the time deviation with the width of the time window to calculate the time correlation degree of a single matching pair. The processor calculates the average of the time correlation degrees of all matching pairs and uses this average as the event correlation score. The process of determining the event correlation score will be described in detail below and will not be elaborated here.

[0035] It's important to note that software test scores only assess the probability of software malfunction, while hardware test scores only assess the probability of sensor hardware malfunction. Neither type of score reflects the temporal coupling between software behavior and sensor sampling. In real-world testing scenarios, abnormal sensor data presents two possibilities. The first possibility is random sensor fluctuation, with no temporal correlation between the anomaly and software behavior; this type of anomaly should not be attributed to the fault cause of this test case. The second possibility is a high degree of overlap between the occurrence of critical software behavior and the occurrence of sensor data anomalies; these two types of anomalies are temporally coupled, and this type of situation represents a cross-domain fault that requires focused identification. Event correlation scores are used to quantify the strength of this temporal coupling. Introducing this dimension when calculating the test scores for intelligent driving systems can avoid including independent sensor fluctuations as a fault cause in test cases, while simultaneously identifying potential problems arising from software-hardware temporal coupling.

[0036] S105 determines the test score of the intelligent driving system based on software test scores, hardware test scores, and event correlation scores.

[0037] The test score of an intelligent driving system is a comprehensive score derived from the probability of software failure, the probability of sensor hardware failure, and the probability of hardware-software timing coupling failure. For example, the test score for an intelligent driving system can range from 0 to 1. The test score measures the overall probability of the intelligent driving system failing after the execution of this test case. A higher test score indicates a higher probability of overall system failure during this test run.

[0038] In some embodiments, the processor reads the software test score Sw, the hardware test score Ss, and the event correlation score Ccross. The processor retrieves the pre-configured weight coefficients α, β, and γ corresponding to each dimension. The processor multiplies the three scores by their respective weight coefficients and then performs a summation operation to calculate the test score F of the intelligent driving system.

[0039] F = α•Sw + β•Ss + γ•Ccross Where F represents the test score of the intelligent driving system, α is the weighting coefficient corresponding to the software test score, β is the weighting coefficient corresponding to the hardware test score, and γ is the weighting coefficient corresponding to the event correlation score, and α+β+γ=1. The weighting coefficients can be adjusted according to the degree of dependence of the test scenario on software, hardware, and the correlation between software and hardware. After obtaining the test score, this value can be input into the subsequent judgment process to distinguish whether the intelligent driving system has passed the test, failed the test, or entered the intermediate state of in-depth fault analysis.

[0040] It's important to note that the software test score, hardware test score, and event correlation score represent the probability of failure in three independent dimensions. The software test score reflects the probability of failure within the software itself, the hardware test score reflects the probability of failure within the sensors themselves, and the event correlation score reflects the probability of failure resulting from the temporal coupling between the software and hardware. Relying on any single score alone cannot fully measure the probability of failure of the entire intelligent driving system under this test case. By fusing the quantitative results from these three dimensions, a unified test score is output. This test score comprehensively reflects the probability of failure within the software itself, the probability of failure within the sensors themselves, and the probability of failure resulting from the temporal coupling of the two, which helps improve the accuracy of intelligent driving system test results.

[0041] The testing apparatus of this disclosure determines event correlation scores through software event sequences and sensor data. When determining the test score of the intelligent driving system, it considers not only the software test score and hardware test score, but also the event correlation score between the two. Compared to the existing method of independently testing the software and hardware domains, this approach offers several advantages. First, when the software function test passes but the sensor performance is on the verge of degradation, the event correlation score can capture the abnormal correlation between the software and hardware domains. This avoids directly marking the test case as "passed" and missing "software false pass" faults caused by sensor degradation, significantly reducing the cross-domain fault detection rate and improving the accuracy of the intelligent driving system test results. Second, when software and hardware anomalies are highly coupled in time, the event correlation score can perform correlation analysis between the two, avoiding misjudging independent hardware anomalies as intelligent driving system anomalies, which is beneficial for identifying cross-domain interaction faults.

[0042] In some embodiments of this disclosure, such as Figure 2 As shown, the processor determines the event correlation score based on the software event sequence and sensor data, specifically configured as follows: S1041, Based on the software event sequence, determine the software event time sequence corresponding to the software event sequence.

[0043] A software event time sequence is a time set formed by arranging the occurrence times of each software event in the sequence according to the chronological order of their occurrence.

[0044] In some embodiments, the processor iterates through all software events in the software event sequence. The processor reads the occurrence time (i.e., timestamp) carried by each software event. Following the original order of the software events within the sequence, the processor sequentially arranges all the read occurrence times to generate a software event time sequence. The processor stores the mapping relationship between software events and their corresponding times, ensuring that the original software event can be located later using the software event time.

[0045] S1042, Based on sensor data, determine the acquisition time sequence corresponding to the sensor data.

[0046] A data acquisition time sequence is a collection of time data formed by arranging the acquisition times corresponding to each sampling action in the sensor data in chronological order. Each time element within the data acquisition time sequence corresponds one-to-one with a single sampling record within the sensor data.

[0047] In some embodiments, the processor iterates through all sensor data, reads the acquisition time corresponding to each sensor sampling record, and arranges all acquisition times according to the actual order in which the sampling occurred, generating a sequence of acquisition times. The processor also stores the mapping relationship between acquisition times and the original sensor sampling records, ensuring that after matching the acquisition times later, the corresponding physical sampling content of the sensors, i.e., the sensor data, can be retrieved in reverse.

[0048] S1043, Based on the software event time sequence and the acquisition time sequence, determine the software event-sensor data matching pair.

[0049] Software incidents Sensor data matching pairs are pairs of software events and sensor sampling records that meet the time proximity condition. Each matching pair is associated with one original software event, one original sensor sampling record, and the time difference between the two pairs. Multiple matching pairs form a matching pair set, which comprehensively records all software and sensor pairings that meet the time proximity condition during this test.

[0050] As an example, a time window is determined centered on the moment of the software event, and sensor data within the time window is searched. The sensor data is then time-series bound to the software event to obtain at least one software event-sensor data matching pair.

[0051] In some embodiments, the processor pre-sets a time window ΔT, for example, ΔT = ±100ms. The processor sequentially retrieves each software event moment within the software event time sequence. The processor determines the time range of the time window with the current software event moment as the center. The processor searches for all acquisition moments within the time window range in the acquisition time sequence. Based on the time mapping relationship, the processor locates the sensor sampling records corresponding to the acquisition moments. The processor combines the software event record corresponding to the current software event with the sensor sampling records found within the time window in pairs to generate a software event. Sensor data matching pairs. After the processor traverses all elements of the software event time sequence, it collects all generated matching pairs to form a matching pair set.

[0052] S1044, determine the event association score based on software event-sensor data matching pairs.

[0053] In this embodiment of the disclosure, software events are constructed by extracting the time sequence of software events and the time sequence of sensor data acquisition, using the time dimension as the link. By matching sensor data pairs and then calculating event correlation scores based on these pairs, the timing of software domain runtime events and hardware domain sampled data can be aligned. This eliminates analytical bias caused by the inconsistency in the time bases of the two types of heterogeneous data, and improves the accuracy of identifying cross-domain coupling anomalies between software and hardware.

[0054] In some embodiments of this disclosure, such as Figure 3 As shown, the processor determines the event association score based on software event-sensor data matching pairs, specifically configured as follows: S10441, Obtain the temporal correlation between software events and sensor data in the software event-sensor data matching pair.

[0055] The temporal correlation degree w_ij is used to characterize a single set of software events. The degree of temporal alignment between software events and sensor data in a sensor data matching pair. The smaller the time deviation of a single matching pair, the higher the temporal correlation, indicating a better temporal alignment between the software events and sensor data in that matching pair.

[0056] S10442, Determine the mean of time correlation based on the number of software event-sensor data matching pairs and the time correlation.

[0057] The mean of time correlation is the sum of all software events. The statistical value is obtained by averaging the time correlation of the sensor data matching pairs.

[0058] In some embodiments, the processor counts software events. The total number of sensor data matching pairs. The processor summarizes the temporal correlation degree corresponding to all matching pairs. The processor performs a summation operation on all temporal correlation degrees. The processor divides the summation result Σw_ij by the total number of matching pairs N to obtain the mean temporal correlation degree. When the total number of matching pairs is zero, the processor directly sets the mean temporal correlation degree to a preset minimum value, indicating that there is no temporal matching relationship between the software event and the sensor sampling.

[0059] S10443, determine the event association score based on the mean of time association degree.

[0060] Event correlation score Ccross = (1 / N) • Σw_ij; Where Ccross is the event correlation score, w_ij is the time correlation degree, and N is the number of software events. The number of sensor data matching pairs.

[0061] In this embodiment of the disclosure, by combining software events The mean temporal correlation degree is calculated by the number of sensor data matching pairs and the temporal correlation degree corresponding to each matching pair. The event correlation score is then determined based on the mean temporal correlation degree. This can comprehensively consider the effective matching sample size and the temporal fit between pairs of data, avoiding interference with the overall evaluation results due to temporal anomalies of individual samples. The event correlation score can objectively reflect the temporal coupling level between software events and sensor data at the overall level, providing a stable and reliable data foundation for the evaluation of cross-domain fault causes.

[0062] In some embodiments of this disclosure, such as Figure 4 As shown, the processor acquires the temporal correlation between software events and sensor data in the software event-sensor data matching pair, specifically configured as follows: S104411, Based on the software event time and sensor data acquisition time of the software event in the software event-sensor data matching pair, determine the time deviation between the software event and the sensor data.

[0063] Time deviation Δt_ij represents the same set of software events. The time deviation is the absolute time difference between the software event moment and the sensor data acquisition moment in sensor data matching. The larger the value of the time deviation, the farther apart the software event and the sensor data are on the time axis.

[0064] In some embodiments, the processor reads the software event time t_i corresponding to the software event in the current software event-sensor data matching pair, and simultaneously reads the acquisition time t_j corresponding to the sensor data in the software event-sensor data matching pair. The processor calculates the difference between the two times and then takes the absolute value of the difference to obtain the time deviation Δt_ij.

[0065] S104412, calculates the ratio of time deviation to time window.

[0066] The time window ΔT is the time during which the processor generates software events. The time range parameter used in the sensor data matching phase. The time window limits the upper limit of the time distance between software events and sensor data that can form a valid matching pair. The ratio is the relative proportion Δt_ij / ΔT obtained by dividing the time deviation Δt_ij by the time window width ΔT, used to characterize the proportion of the time deviation to the entire valid matching time range.

[0067] S104413, Determine the degree of time correlation based on the ratio.

[0068] The temporal correlation degree w_ij is negatively correlated with the ratio Δt_ij / ΔT. The smaller the ratio, the higher the output of the temporal correlation degree, indicating a better time alignment between software events and sensor data.

[0069] When determining the degree of temporal correlation based on the ratio, the degree of temporal correlation w_ij can be obtained by subtracting the ratio Δt_ij / ΔT from the numerical value 1.

[0070] w_ij = 1 Δt_ij / ΔT Among them, the smaller the time deviation Δt_ij, the closer the time correlation degree w_ij is to 1.

[0071] In this embodiment of the disclosure, the time deviation is obtained by using the software event time and the sensor data acquisition time, and then the time correlation degree is calculated by the ratio of the time deviation to the time window. This can transform the degree of time offset between two types of heterogeneous data into a quantifiable numerical indicator, and realize a standardized measurement of the degree of time alignment.

[0072] In some embodiments of this disclosure, such as Figure 5 As shown, the processor determines the software test score, specifically configured as follows: S501, based on the test results, determine the first score representing the importance of the module to which the test case belongs to the intelligent driving system, the second score representing the impact range of the module to which the test case belongs on the intelligent driving system fault, and the third score representing the severity level of the fault of the module to which the test case belongs to the intelligent driving system fault.

[0073] The first score, M, characterizes the importance of the functional module to which the current test case belongs to the intelligent driving system. If the module malfunctions, the more important its function is to the safe operation of the entire vehicle, the higher the first score will be.

[0074] The second score, R, characterizes the functional scope that a software malfunction can affect. The more functions of the intelligent driving system are impacted by the malfunction of this module, the higher the value of the second score.

[0075] The third score, S, characterizes the severity of the software fault itself. The higher the degree of program abnormality caused by the fault, the higher the value of the third score.

[0076] It should be noted that the first score M, the second score R, and the third score S are used to represent the potential risks once the failure occurs.

[0077] In some embodiments, the processor reads the module identification information recorded in the test results to determine the system module corresponding to the current test case. The processor retrieves pre-stored system security architecture analysis data and determines a first score based on the importance of the module in the intelligent driving system, for example, core security links > communication base > non-critical modules > test infrastructure, and a mapping table between importance and score.

[0078] The processor identifies the remaining system modules and functional units that can be affected after a fault occurs. The processor calculates the degree of fault propagation in the intelligent driving system, for example, loss of vehicle function > multi-module cascading > local degradation > no impact on vehicle functions, and establishes a mapping table between the degree of propagation and the score to determine the second score.

[0079] The processor reads the fault performance information recorded in the test results and identifies the corresponding anomaly type. Fault types include different levels such as program crash, logic assertion failure, runtime timeout, numerical out-of-bounds, and occasional jitter. The processor determines a third score based on the severity of the fault, for example, program crash > logic assertion failure > runtime timeout > numerical out-of-bounds > occasional jitter, and a mapping table between severity and score.

[0080] S502 determines the software test score based on the first score, the second score, and the third score.

[0081] Software testing score is a quantitative indicator derived from the comprehensive factors of module importance, the scope of failure impact, and the severity of the failure. The software testing score characterizes the probability of a software failure occurring during the test run. A higher software testing score indicates a higher probability of a software failure. There is a positive correlation between the software testing score and the probability of software failure; a higher score corresponds to a higher probability of software failure.

[0082] The processor retrieves the pre-configured weight coefficients w_m, w_r, and w_s corresponding to the three dimensions. These weight coefficients adjust the contribution percentage of different dimensions in the final result. The sum of all weight coefficients is a fixed value, satisfying w_m + w_r + w_s = 1. The processor multiplies the first score, second score, and third score by their respective weight coefficients, then sums the results to obtain the software test score Sw.

[0083] Sw = w_m • M + w_r • R + w_s • S When the test results show that the software ran without any faults, the processor directly sets the software test score to the lowest value, such as 0.

[0084] In this embodiment of the disclosure, the software test score is calculated by integrating multi-dimensional evaluation based on the potential risk of failure of the module to which the test case belongs. Instead of relying solely on the failure phenomenon itself for scoring, it can take into account the weight of the module itself in the system, the scope of failure spread and the degree of harm caused by the failure, so that the software test score can fully reflect the actual probability of failure in the software domain.

[0085] In some embodiments of this disclosure, such as Figure 6 As shown, the processor determines the hardware test score, specifically configured as follows: S601, determine the deviation of sensor data from a preset threshold.

[0086] The preset threshold is determined based on the mean and standard deviation of historical sensor data for the target scenario; the target scenario is the environment in which the intelligent driving system is tested based on test cases. The target scenario is correlated with operating conditions that can affect sensor output, such as ambient lighting, weather, and test distance.

[0087] Historical sensor data consists of sensor sampling statistics accumulated from multiple normal test runs under this target scenario. The mean μ represents the average output level of the sensor data under this scenario, and the standard deviation σ represents the normal fluctuation range of the sensor indicators under this scenario. The preset threshold is adjusted according to changes in the target scenario to adapt to the normal performance boundaries of the sensor under different operating conditions.

[0088] The preset threshold T_dynamic is calculated from the mean μ and standard deviation σ of the historical sensor data corresponding to the target scene.

[0089] T_dynamic = μ + k•σ, where k is the confidence coefficient.

[0090] It should be noted that the normal output range of a sensor varies under different external environments and operating conditions. A fixed judgment threshold cannot adapt to the fluctuations in sensor output caused by changes in the scene, and is prone to misjudgment. This threshold can be dynamically adjusted to adapt to the natural degradation of sensor performance.

[0091] Deviation degree is used to characterize the degree to which sensor data deviates from a preset threshold. The farther the sensor data is from the preset threshold, the higher the deviation degree, indicating that the sensor hardware performance deviates more significantly from the normal operating level.

[0092] In some embodiments, the processor compares the sensor data with a preset threshold, calculates the degree of deviation of the sensor data relative to the preset threshold, and obtains the deviation degree.

[0093] For example, for a vision sensor, sensor data can be pixel brightness values ​​obtained from image acquisition. A preset threshold is determined based on the mean and standard deviation of pixel brightness values ​​from historical images in the target scene. The processor acquires the pixel brightness value of the currently acquired image, calculates the difference between the pixel brightness value and the preset threshold, and normalizes this difference to obtain the deviation corresponding to the vision sensor.

[0094] For LiDAR sensors, the sensor data can be the ranging values ​​of the LiDAR point cloud. The preset threshold is determined based on the mean and standard deviation of historical point cloud ranging data in the target scene. The processor obtains the current ranging value output by the LiDAR, calculates the difference between the ranging value and the preset threshold, and normalizes this difference to obtain the corresponding deviation of the LiDAR. The magnitude of the deviation represents the level of deviation between the actual output data of the sensor and the normal data range in the target scene.

[0095] S602 determines hardware test scores based on deviation.

[0096] The hardware test score characterizes the probability of a failure in the sensor hardware. A higher deviation indicates a more significant deviation of the sensor's performance from the normal performance level of the scenario, and consequently, a higher hardware test score.

[0097] In some embodiments, the processor converts the deviation into a hardware test score according to a preset mapping relationship. When the sensor data is within the normal range allowed by a preset threshold, the deviation is at a low level, and the resulting hardware test score corresponds to a low value. When the sensor data significantly exceeds the allowable range of the preset threshold, the deviation is at a high level, and the resulting hardware test score corresponds to a high value.

[0098] In some embodiments of this disclosure, the processor determines the test score of the intelligent driving system, specifically configured as follows: If the test score is less than the first threshold, the test is deemed passed. If the test score is greater than or equal to the second threshold, the test is determined to be failed. In response to a test score that is greater than or equal to a first threshold and less than a second threshold, a fault analysis step is triggered to determine the fault type. The first threshold is less than the second threshold.

[0099] The first threshold is a dividing value used to distinguish between the passing state of the intelligent driving system test and the state requiring in-depth analysis. The first threshold corresponds to the test score corresponding to the acceptable upper limit of failures. A test score lower than this value indicates that the overall probability of failure in this test run is within an acceptable range. For example, the first threshold could be 0.3.

[0100] A test score below the first threshold indicates that the probability of a failure during this test run is within a controllable range. The probabilities of software failure, sensor hardware failure, and hardware-software timing coupling failure are all below the level requiring in-depth investigation. When the test score is below the first threshold, the test is considered passed. This allows for rapid evaluation of test cases with no significant failure risk, eliminating the need for time-consuming fault analysis and improving overall test execution efficiency.

[0101] The second threshold is a dividing line used to distinguish between the state to be analyzed in depth and the state of test failure. The second threshold corresponds to the test score corresponding to the unacceptable lower limit of failure. A test score greater than or equal to this value indicates that the overall probability of failure in this test run exceeds the allowable range. For example, the second threshold could be 0.7.

[0102] When the test score is greater than or equal to the second threshold, regardless of whether the fault originates from the software itself, the sensor hardware itself, or the timing coupling of software and hardware, the overall probability level of the fault no longer meets the test release conditions. When the test score is greater than or equal to the second threshold, the test is directly marked as failed, achieving rapid interception of high-failure-risk test cases.

[0103] The fault analysis step is the subsequent processing procedure initiated when the test score falls within the middle range of the two threshold levels. This step is used to conduct in-depth analysis of software and hardware anomalies, locate the source of the fault, and output the corresponding fault type. The fault type is used to distinguish whether the fault originates from the software itself, the sensor hardware itself, or from the interaction and coupling between software and hardware.

[0104] It's important to note that a test score falling between the first and second thresholds indicates that the test run is neither directly deemed safe for release nor deems it a direct failure. This type of scenario often corresponds to situations where software testing is normal but sensor metrics are abnormal, or where there is a degree of timing coupling between software and hardware—a gray area. Directly declaring the test pass in this case would leave potential fault causes, while directly declaring it fail would lead to false interceptions. Therefore, the processor does not output a final pass or fail conclusion but instead initiates a fault analysis process to further pinpoint the root cause of the fault and output the fault type.

[0105] In some embodiments of this disclosure, such as Figure 7 As shown, in response to a test score greater than or equal to a first threshold and less than a second threshold, the processor triggers a fault analysis step to determine the fault type, which is specifically configured as follows: S701, in response to a fault type where the software test passes but the hardware test fails, performs time series analysis on the software event-sensor data matching pair to determine the coupling time interval between the software anomaly and the hardware anomaly.

[0106] The software test passed, but the hardware test failed, which falls under a fault type identified in the fault analysis process. This fault type indicates that the overall software assessment result is satisfactory, but the sensor hardware specifications do not meet the requirements.

[0107] The software event-sensor data matching pair is determined based on the software event time sequence and the acquisition time sequence.

[0108] Time series analysis refers to the analytical processing performed on a set of matching pairs arranged chronologically, used to filter out consecutive time segments where both software and hardware anomalies occur simultaneously.

[0109] The coupling time interval represents the continuous time range during the test run where the occurrence of abnormal software behavior and the occurrence of abnormal sensor hardware sampling overlap. The start and end times of the coupling time interval are jointly determined by the actual occurrence times of the software and hardware anomalies.

[0110] In some embodiments, after the processor identifies the current fault type as software test passed and hardware test failed, it retrieves previously generated software events. Sensor data matching pairs. The processor iterates through all software events. Sensor data matching pairs are selected, identifying those containing software anomaly events and those containing hardware anomaly sampling. The processor extracts the time information corresponding to the selected matching pairs, performs time series analysis on the time information, and identifies continuous time segments where software and hardware anomalies occur simultaneously. The processor records the start and end times of these continuous segments, forming coupled time intervals.

[0111] S702, within the coupling time interval, obtains the software configuration layer parameters corresponding to the software event.

[0112] The software configuration layer parameters include parameters that were not validated when obtaining software test scores.

[0113] Software configuration layer parameters are a type of parameter used to control the hardware operating state during the operation of intelligent driving system software. These parameters are not verified during the software testing score calculation phase. Anomalies in software configuration layer parameters do not directly cause software business logic failures, but they can alter the sensor's operating environment and mode, indirectly leading to abnormal sensor hardware performance.

[0114] In some embodiments, the processor uses the start and end times of the coupling time interval as filtering criteria to filter all software events whose occurrence time falls within the coupling time interval. The processor retrieves software runtime snapshot data retained during the test and reads the software configuration layer parameters loaded and effective at the time of occurrence of the filtered software events.

[0115] S703 determines the cause of the fault based on software configuration layer parameters.

[0116] The cause of the fault is the root cause of the fault, determined through analysis and judgment. The cause of the fault can point to abnormal parameters in the software configuration layer, or it can point to a fault in the sensor hardware itself.

[0117] In some embodiments, the processor compares the read software configuration layer parameters with preset parameters to obtain a test result indicating whether there is an anomaly in the software configuration layer. The processor then combines the test result indicating whether there is an anomaly in the software configuration layer within the coupling time interval, the software anomaly mode, and the hardware anomaly mode to perform correlation reasoning to obtain the cause of the current fault.

[0118] For example, within the coupling time interval, the identified hardware anomaly pattern was a persistently low brightness in the image output by the visual sensor, while the software anomaly pattern was a decrease in the target recognition confidence level output by the perception algorithm module. After comparing the software configuration layer parameters, it was found that the image exposure gain parameter was inconsistent with the preset parameter. Using domain causality coefficients for correlation reasoning, the cause of the fault was determined to be an incorrect configuration of the exposure gain parameter in the software configuration layer. This configuration anomaly further caused the sensor data to be incorrectly parsed by the algorithm, resulting in a hardware-related anomaly.

[0119] In another example, the hardware anomaly pattern identified within the coupling time interval was a jump in LiDAR point cloud ranging. After comparing the software configuration layer parameters, all parameters of the software configuration layer were consistent with the preset parameters. By combining the software anomaly pattern and the hardware anomaly pattern and performing correlation reasoning, the cause of the fault was determined to be a fault in the LiDAR hardware itself.

[0120] S704 generates repair information based on the cause of the fault.

[0121] Repair information is action guidance information generated based on the cause of the failure. It includes a description of the root cause of the failure and the corresponding action plan. Repair information can be output to the test reporting system for reference by developers or testers.

[0122] In some embodiments, the processor reads the determined cause of the fault and retrieves a preset fault. Repair the mapping relationship, match the corresponding handling content according to the cause of the fault, and generate repair information. If the cause of the fault is abnormal software configuration layer parameters, the repair information includes the parameter objects that need to be corrected and the target parameter values. If the cause of the fault is hardware failure, the repair information includes handling instructions related to hardware component replacement.

[0123] For example, when the fault is caused by an incorrect configuration of the exposure gain parameter in the software configuration layer, the generated repair information records the fault type as "software configuration anomaly," records the coupling time interval, and provides a suggestion to modify the exposure gain parameter to the preset parameter range. When the fault is caused by a fault in the LiDAR hardware itself, the generated repair information records the fault type as "hardware fault," records the coupling time interval, and provides a suggestion to replace the LiDAR hardware component.

[0124] In this embodiment of the disclosure, under a specific fault type where software testing passes but hardware testing fails, the software events are constructed based on the software event time sequence and the data acquisition time sequence. Sensor data matching is used to perform time series analysis to locate the coupled time intervals where software and hardware anomalies coexist. Within these intervals, unverified software configuration layer parameters from the software test score calculation process are extracted as the analysis object. This allows for the deduction of fault causes and the generation of repair information. This enables focusing on the actual time window of the fault, uncovering configuration-related anomalies missed in single-round software test score verification, and distinguishing between hardware indicator anomalies that are superficial faults induced by software configuration and actual hardware faults. This reduces the possibility of misjudging hardware anomalies and replacing faulty hardware.

[0125] In some embodiments of this disclosure, such as Figure 8 As shown, the processor determines the cause of the fault based on software configuration layer parameters, specifically configured as follows: S7031 determines the test results of the software configuration layer based on the software configuration layer parameters and preset parameters.

[0126] Preset parameters are standard values ​​for the software configuration layer that are predetermined during the design phase of an intelligent driving system. These preset parameters serve as a reference benchmark to ensure the normal operation of sensors and business functions.

[0127] The software configuration layer test results characterize the deviation of the actual effective software configuration layer parameters from the preset parameters within the coupling time interval. The software configuration layer test results record whether the parameters deviated, and also record the parameter identifiers and actual values ​​of the deviated parameters.

[0128] In some embodiments, the processor reads the software configuration layer parameters acquired within the coupling time interval. The processor retrieves the preset parameters corresponding to these parameters. The processor compares the actual software configuration layer parameters with the preset parameters item by item. The processor determines whether there are any deviations in the parameters and records the types of parameters that deviate, as well as the differences between the actual values ​​and the standard values. The processor organizes the above comparison data and generates the software configuration layer test results.

[0129] For example, if the software configuration layer parameter is the external parameter matrix Pitch (tilt angle) parameter, and the external parameter matrix Pitch parameter is inconsistent with the preset parameter, the software configuration layer is determined to be abnormal.

[0130] S7032 determines the cause of the fault based on the test results of the software configuration layer, the software anomaly mode and hardware anomaly mode within the coupling time interval, and the preset domain causality coefficient.

[0131] The domain causality coefficient is used to characterize the confidence level of a causal relationship between software anomalies and hardware anomalies. A higher domain causality coefficient indicates a higher degree of confidence that the corresponding software anomaly can induce the corresponding hardware anomaly.

[0132] A software anomaly pattern is a set of features obtained by aggregating multiple software anomaly events within a coupled time interval. The software anomaly pattern includes the type of the anomaly event, the time of occurrence of the anomaly event, and the associated intelligent driving system module. Hardware anomaly patterns are feature sets obtained by aggregating multiple hardware anomaly samples within a coupled time interval. A hardware anomaly pattern includes anomalous sensor data, the time of occurrence of the anomaly, and the sensor type. As an example, software anomaly patterns and sensor anomaly patterns are generated as follows: Extract software and hardware anomalous event subsets from software event-sensor data matching pairs; the software anomalous event subset includes software events indicating test failure; the hardware anomalous event subset includes sensor data exceeding a preset threshold; generate software anomalous patterns based on the software anomalous event subsets; generate sensor anomalous patterns based on the hardware anomalous event subsets.

[0133] When generating software exception patterns based on a subset of software exception events, aggregation processing can be performed on the subset of software exception events to generate software exception patterns. When the subset of software exception events includes multiple software exception events, the information of multiple software exception events can be merged through aggregation processing to obtain a software exception pattern representing the exception characteristics of the subset.

[0134] As an example, the processor performs aggregation processing on a subset of software exception events to generate software exception patterns, specifically configured as follows: Cluster multiple software exception events within a subset of software exception events according to time intervals; Extract the common event type and associated intelligent driving system module of multiple software abnormal events under the same cluster group, and determine the abnormal software event type and the associated intelligent driving system module; Based on the time boundaries of clustering, the occurrence time of abnormal software events corresponding to software anomaly patterns is determined.

[0135] When generating a hardware anomaly pattern based on a subset of hardware anomaly events, aggregation processing can be performed on the subset of hardware anomaly events to generate a hardware anomaly pattern. When the subset of hardware anomaly events contains multiple sensor anomaly data, the information of the multiple sensor anomaly data is merged through aggregation processing to obtain a hardware anomaly pattern representing the overall anomaly characteristics of the subset.

[0136] As an example, the processor performs aggregation processing on a subset of hardware exception events to generate a hardware exception pattern, specifically configured as follows: Cluster and group multiple sensor anomaly data within the hardware anomaly event subset according to continuous time windows; Extract common indicator features and sensor types from multiple abnormal sensor data under the same cluster group to determine the abnormal sensor data and sensor type; Based on the time boundaries of clustering, the occurrence time of abnormal sensors corresponding to hardware anomaly patterns is determined.

[0137] In this embodiment, standardized software configuration layer test results are first obtained by comparing software configuration layer parameters with preset parameters. Then, the cause of the fault is jointly determined by combining the software and hardware anomaly patterns within the coupling time interval with the domain causal coefficient representing cross-domain causal confidence. This overcomes the limitations of single parameter comparison and determination, and conducts causal reasoning by comprehensively considering parameter deviation characteristics, temporal anomaly characteristics, and domain prior knowledge. This can accurately distinguish between hardware manifestation anomalies induced by software configuration anomalies and original hardware faults, significantly reducing the probability of root cause misjudgment in software and hardware coupling fault scenarios, and effectively improving the accuracy of software and hardware cross-fault localization in intelligent driving systems.

[0138] In some embodiments of this disclosure, such as Figure 9 As shown, the processor determines the cause of the failure based on the software configuration layer test results, software anomaly modes within the coupling time interval, hardware anomaly modes, and preset domain causality coefficients. Specifically, it is configured as follows: S70321, based on the test results of the software configuration layer, the software anomaly mode and hardware anomaly mode within the coupling time interval, and the preset domain causality coefficient, construct at least one candidate causal chain.

[0139] At least one candidate causal chain includes: a software configuration layer anomaly causal chain, a hardware failure causal chain, and an environmental failure causal chain. A software configuration layer anomaly causal chain indicates that the fault originates from an abnormality in software configuration layer parameters, which propagates through the system chain and ultimately results in hardware manifestation anomalies. A hardware failure causal chain indicates that the fault originates from a failure in the sensor hardware itself, directly causing hardware manifestation anomalies. An environmental failure causal chain indicates that the fault originates from external environmental factors, which act on the sensor and then cause hardware manifestation anomalies.

[0140] Candidate causal chains comprise one or more anomalous propagation paths from a root cause event to a hardware manifestation anomaly. Each candidate causal chain includes at least two anomalous event nodes arranged chronologically. An anomalous event node is the fundamental building block of the candidate causal chain. An anomalous event node includes at least one of software or hardware anomalous event nodes; software anomalous event nodes correspond to anomalous behavior occurring at the software level, while hardware anomalous event nodes correspond to anomalous phenomena occurring in sensor data. A causal direction exists between anomalous event nodes, which characterizes the logical relationship of anomalous propagation between two adjacent anomalous event nodes. The causal direction is determined based on the chronological order of the anomalous event nodes on the timeline.

[0141] In some embodiments, the processor reads the test results of the software configuration layer, software anomaly patterns and hardware anomaly patterns within the coupling time interval, as well as preset domain causality coefficients. The processor identifies all plausible root causes in the current scenario and constructs at least one candidate causal chain accordingly.

[0142] Each candidate causal chain has at least two anomalous event nodes arranged in chronological order. The processor determines the causal direction between adjacent nodes based on the order in which the anomalous event nodes appear on the timeline, forming a propagation path from the root cause event to the hardware manifestation anomaly.

[0143] S70322, obtain the temporal correlation between each abnormal event in the candidate causal chain.

[0144] The temporal correlation between anomalous events represents the degree of temporal proximity between two anomalous event nodes that have a potential causal transmission relationship within the same candidate causal chain. The higher the temporal correlation, the closer the occurrence times of the two anomalous event nodes are, and the more sufficient the temporal conditions for the anomalous event to propagate along that path.

[0145] In some embodiments, the processor selects a single candidate causal chain and traverses each pair of adjacent anomalous event nodes within that candidate causal chain that have a potential causal transmission relationship. The processor reads the occurrence time corresponding to the preceding anomalous event node and the occurrence time corresponding to the following anomalous event node. The processor calculates the time deviation between the two sets of occurrence times. The processor reads the system's preset time window width, calculates the ratio of the time deviation to the time window width, and calculates the temporal correlation degree corresponding to the pair of adjacent anomalous event nodes based on the difference between 1 and this ratio.

[0146] S70323, determine the score of each candidate causal chain based on the domain causality coefficient and temporal correlation degree corresponding to the candidate causal chain.

[0147] The score of a candidate causal chain is a quantitative value calculated by combining the domain causality coefficient and the temporal correlation between nodes within the chain. The higher the score of a candidate causal chain, the higher the initial credibility of the anomaly propagation path.

[0148] In some embodiments, the processor retrieves the domain causality coefficient corresponding to each candidate causal chain, and simultaneously retrieves the temporal correlation between each anomalous event node within the candidate causal chain. The processor then performs a fusion calculation on the domain causality coefficient and the average temporal correlation between each anomalous event node within the candidate causal chain according to a preset calculation rule, to obtain the score for the corresponding candidate causal chain.

[0149] For example, when determining the score of each candidate causal chain based on the domain causality coefficient and temporal correlation degree, the preset calculation rule can specifically be a weighted sum of the domain causality coefficient and the temporal correlation degree. For example, score = λ•(domain causality coefficient) + ( λ)•(average temporal correlation of nodes in the chain), where λ is the weight and λ∈[0,1].

[0150] S70324, perform causal relationship verification on the candidate causal chains to be verified that meet the preset conditions, and obtain the verification results.

[0151] The preset conditions are pre-defined score filtering rules used to select candidate causal chains that need to be verified from all candidate causal chains. The preset conditions can be that the score is greater than a set score threshold, or that a specified number of candidate causal chains with the highest scores are selected.

[0152] Candidate causal chains to be verified are those that require actual verification operations to meet the preset conditions for a score.

[0153] In some embodiments, the processor compares the score of each candidate causal chain with preset conditions to filter and obtain candidate causal chains to be verified. The processor performs causal relationship verification operations on the candidate causal chains to be verified. The verification operation selects the corresponding processing method according to the type of the candidate causal chain to be verified. For a software configuration layer abnormal causal chain, the verification operation is to correct the software configuration parameters and then re-acquire sensor data. For a hardware failure causal chain, the verification operation is to replace the hardware component and then re-acquire sensor data. For an environmental failure causal chain, the verification operation is to adjust the environmental conditions and then re-acquire sensor data. The processor compares the changes in the observed indicators before and after verification to obtain the verification result corresponding to the candidate causal chain to be verified.

[0154] S70325, Based on the verification results, the cause of the fault was determined.

[0155] The cause of the failure is the root source of the failure that has been verified in practice. The cause of the failure corresponds to the root cause event of the candidate causal chain that is actually established.

[0156] In some embodiments, the processor reads the verification results corresponding to the candidate causal chain to be verified. If the verification result shows that the propagation path of the current candidate causal chain to be verified is valid, the processor determines the root cause event corresponding to the candidate causal chain as the cause of the failure. If the verification result shows that the current candidate causal chain to be verified is invalid, the processor selects candidate causal chains whose scores meet the preset conditions and performs verification in descending order of scores.

[0157] For example, the test scenario is the end-of-line (EOL) test of a front-view camera. Within the coupling time interval, the software anomaly mode is a decrease in the number of targets detected by the sensing module, and the hardware anomaly mode is an overall darker image output by the camera. The software configuration layer test results show that the image exposure gain parameter deviates from the preset parameter. Based on the above information and the preset domain causality coefficient, two candidate causal chains are constructed. The first candidate causal chain is the software configuration layer anomaly causal chain, with the event nodes being, in sequence, an abnormal exposure gain parameter configuration, a decrease in the number of targets detected by the sensing module, and an abnormal darker image appearance by the camera, with a corresponding domain causality coefficient of 0.88. The second candidate causal chain is the hardware failure causal chain, with the event nodes being, in sequence, hardware damage to the camera's photosensitive device, a darker camera image, and a decrease in the number of targets detected by the sensing module, with a corresponding domain causality coefficient of 0.4.

[0158] The temporal correlation between anomalous events within the two candidate causal chains was obtained separately. The average temporal correlation of the first candidate causal chain was 0.81, and the average temporal correlation of the second candidate causal chain was 0.31. The domain causality coefficient and the temporal correlation were combined for calculation, and the score was calculated as follows: Score = λ•(domain causality coefficient) + ( λ)•(average temporal correlation of nodes within the chain), taking λ as 0.5 as an example, the score of the first candidate causal chain is 0.845, and the score of the second candidate causal chain is 0.355. The preset condition is that the score of the candidate causal chain is greater than or equal to 0.7, and the first candidate causal chain is selected as the candidate causal chain to be verified.

[0159] Causal relationship verification is performed on the candidate causal chain to be verified. Based on the verification results, the cause of the failure is determined. For example, the exposure gain parameter is corrected to the preset parameter and the test is re-executed. The brightness of the re-acquired camera image returns to the normal range, and the verification result determines that the causal relationship of the candidate causal chain is valid. Ultimately, the cause of the failure is determined to be an incorrect setting of the exposure gain parameter in the software configuration layer.

[0160] In another example, the software configuration layer test results within the coupling time interval showed that all software configuration layer parameters met the preset parameters. The software anomaly mode was intermittent loss of target recognition results, and the hardware anomaly mode was random frame loss of camera output images. Two candidate causal chains were constructed based on the anomaly information and the domain causality coefficient. The first candidate causal chain was the hardware failure causal chain, with a comprehensive score of 0.87. The second candidate causal chain was the software timing scheduling anomaly causal chain, with a comprehensive score of 0.38. The hardware failure causal chain was selected as the candidate causal chain to be verified. Verification was performed, and based on the verification results, the cause of the failure was determined. For example, after replacing the camera hardware components and retesting, the random frame loss phenomenon disappeared. The verification results determined that the causal relationship of the candidate causal chain was valid, and the cause of the failure was determined to be a camera hardware failure.

[0161] In this embodiment, multiple candidate causal chains covering different fault propagation paths are constructed by integrating software configuration layer test results, software and hardware anomaly patterns, and domain causality coefficients. The causal chains are scored and screened by combining temporal correlation in the time series dimension with domain prior causality coefficients. Entity causality verification is then performed on high-scoring candidate causal chains to ultimately determine the cause of the fault. This hierarchical reasoning mechanism, combining multi-path hypotheses, quantitative scoring, and experimental verification, effectively solves the problems of fuzzy causal relationships and misjudgment due to accidental temporal overlap in software and hardware coupling anomalies. It can accurately distinguish different root causes such as software configuration anomalies, hardware failures, and environmental anomalies, significantly improving the accuracy of cross-domain coupling fault localization in intelligent driving systems.

[0162] In some embodiments of this disclosure, the processor performs causal relationship verification on candidate causal chains to be verified that meet preset conditions, and obtains verification results; based on the verification results, the cause of the fault is determined, specifically configured as follows: A1, obtain the abnormal software configuration layer parameters corresponding to the software abnormal events in the candidate causal chain to be verified.

[0163] Software exception events are software exception event nodes that serve as the root cause starting point in the candidate causal chain to be verified.

[0164] Abnormal software configuration layer parameters are software configuration layer parameters whose actual running values ​​deviate from preset parameters as pointed to by the software abnormal event node.

[0165] In some embodiments, the processor reads the internal information of the candidate causal chain to be verified and locates the software anomaly event node that serves as the root cause. The processor reads the parameter identification information recorded by the software anomaly event node and locates the abnormal software configuration layer parameters based on the parameter identification information.

[0166] A2 performs a correction operation on the abnormal software configuration layer parameters.

[0167] The correction operation is used to adjust the abnormal software configuration layer parameters to the preset parameters.

[0168] In some embodiments, the processor retrieves preset parameters corresponding to the abnormal software configuration layer parameters. The processor rewrites the abnormal software configuration layer parameters in effect in the runtime environment to ensure that their values ​​are consistent with the preset parameters. At the same time, the processor ensures that all other conditions in the test environment remain unchanged, modifying only this target parameter to eliminate interference from other variables on the verification results.

[0169] A3, re-perform the tests on the software and hardware to obtain the corresponding sensor data within the coupling time interval.

[0170] After correcting the abnormal software configuration layer parameters, it is necessary to reproduce the original test scenario and collect sensor data for the corresponding time period. Only by obtaining the sensor data after parameter correction can the changes in hardware indicators before and after the parameter changes be compared, thereby determining whether the software configuration layer parameters are the root cause of the fault.

[0171] In some embodiments, the processor maintains the external environment, test case inputs, and hardware state consistent with the original test, and restarts the software and hardware to re-execute the test cases. The processor acquires the sensor sampling data stream of the complete test process, filters the sampling records whose time falls within the coupling time interval, obtains the corresponding sensor data within the coupling time interval, and caches this part of the data for subsequent threshold comparison.

[0172] A4, in response to the reacquired sensor data meeting the preset threshold range, determines that the verification result is that the causal relationship of the candidate causal chain to be verified is established.

[0173] The preset threshold range is the normal range of sensor data values ​​allowed in the target scenario.

[0174] The verification result is an objective judgment obtained after retesting, indicating whether the abnormal propagation path corresponding to the candidate causal chain to be verified is true.

[0175] The establishment of a causal relationship indicates that the propagation path from software configuration anomalies to hardware manifestation anomalies described by the candidate causal chain to be verified has been experimentally confirmed.

[0176] In some embodiments, the processor compares the reacquired sensor data with a preset threshold range. When all the sensor data falls within the preset threshold range, the processor marks the verification result as indicating that the causal relationship of the candidate causal chain to be verified is valid.

[0177] A5, in response to the verification result that the causal relationship of the candidate causal chain to be verified is established, determines the fault type corresponding to the candidate causal chain to be verified as the fault cause.

[0178] It should be noted that once the causal relationship is experimentally verified, there is no need to continue verifying other candidate causal chains. The fault type corresponding to the current candidate causal chain to be verified is directly identified as the cause of the fault, thus completing root cause localization.

[0179] A6, in response to the reacquired sensor data not conforming to the preset threshold range, the verification result is determined to be that the causal relationship of the candidate causal chain to be verified is not valid.

[0180] The causal relationship is not established, meaning that the experimental results cannot confirm the abnormal propagation path described by the current candidate causal chain to be verified. The hardware anomaly is not caused by the root cause event recorded in this chain.

[0181] In some embodiments, the processor compares the reacquired sensor data with a preset threshold range. If the sensor data exceeds the preset threshold range, the processor marks the verification result as indicating that the causal relationship of the candidate causal chain to be verified is invalid. Simultaneously, the processor saves the verification result and marks the candidate causal chain as having been verified and will not be processed again.

[0182] A7. In response to the verification result that the causal relationship of the candidate causal chain to be verified is not established, the causal relationship verification is performed on the candidate causal chains that meet the preset conditions in descending order of scores, until the cause of the failure is determined based on the verification results.

[0183] It should be noted that the failure of a single candidate causal chain does not mean that all other candidate causal chains are invalid. The remaining candidate causal chains are verified in descending order of their scores, representing different levels of initial credibility. This allows for prioritizing the investigation of propagation paths with higher credibility until a confirmed cause of the failure is found, thus avoiding the omission of the true root cause.

[0184] In some embodiments, the processor filters all candidate causal chains, removes those that have already been verified, and obtains a set of remaining candidate causal chains. The processor sorts the remaining candidate causal chains within this set according to their scores from highest to lowest. The processor selects the top-ranked candidate causal chain as the new candidate causal chain to be verified and re-executes the entire causal relationship verification process. The processor iteratively executes the steps of selecting candidate causal chains to be verified, locating abnormal parameters, correcting parameters, retesting, and comparing thresholds. The loop continues until a candidate causal chain whose verification result confirms a causal relationship is established, at which point the processor determines the cause of the failure and terminates the loop.

[0185] It should be noted that if the causal relationship verification of all candidate causal chains fails, the processor outputs an indication that the cause of the fault cannot be determined, triggers the manual intervention process, and stores the fault analysis record in the historical fault database.

[0186] For example, in the EOL (End-of-Life) test scenario for a forward-looking camera in intelligent driving, the software anomaly mode within the coupling time interval is a decrease in the number of targets detected by the perception module, while the hardware anomaly mode is an overall darker image output by the camera. The software configuration layer test results determine that the image exposure gain parameter deviates from the preset parameter. The processor combines the above information with preset domain causality coefficients to construct two candidate causal chains. The first is the software configuration layer anomaly causal chain, with a comprehensive score of first score, for example, 0.845. The second is the camera hardware failure causal chain, with a comprehensive score of second score, for example, 0.355. The preset condition is that the candidate causal chain score is greater than or equal to 0.7, and the first candidate causal chain is selected as the candidate causal chain to be verified.

[0187] The processor acquires the abnormal software configuration layer parameters corresponding to the software anomaly event within the candidate causal chain to be verified, namely the image exposure gain parameter. The processor performs a correction operation, adjusting the exposure gain parameter to a preset value. After parameter correction, the software and hardware joint test is re-executed, acquiring image data output from the camera within the coupling time interval. If the brightness of the re-acquired image falls within a preset threshold range, the verification result confirms that the causal relationship of the candidate causal chain to be verified is valid, and the software configuration layer anomaly corresponding to the candidate causal chain is identified as the cause of the fault.

[0188] In another example, two candidate causal chains are constructed. The first candidate causal chain has a comprehensive score of third place, for example, 0.74, which meets the preset conditions and is selected as the candidate causal chain to be verified. The processor extracts the corresponding software configuration parameters and corrects them, then performs the joint test again. The reacquired sensor image data still exceeds the preset threshold range, indicating that the causal relationship is not established in this verification. The processor continues to select the candidate causal chain with the highest score among the remaining candidate causal chains as the new candidate causal chain to be verified, and repeats the causal relationship verification process. If the new candidate causal chain to be verified passes the verification, the corresponding fault type is determined as the fault cause. This process is repeated until a verification result is obtained and the fault cause is determined.

[0189] In this embodiment, for the candidate causal chain to be verified, the abnormal software configuration layer parameters are accurately located and corrected to the standard parameter state. Under the premise of controlling variables, the test is re-executed to collect sensor data in the corresponding coupling time interval. The validity of the causal chain is determined based on whether the retest data returns to normal. After the current chain verification fails, the remaining candidate causal chains are automatically traversed according to the score priority for continuous investigation. The physical verification method of parameter correction and retesting replaces pure theoretical temporal reasoning, effectively eliminating the misjudgment problem caused by false temporal coupling. At the same time, the step-by-step selection verification mechanism ensures the comprehensiveness and orderliness of the root cause investigation, greatly improving the accuracy of root cause location in software and hardware coupling failure scenarios.

[0190] In some embodiments of this disclosure, in response to the verification result indicating that the causal relationship of the candidate causal chain to be verified is established, the processor determines the fault type corresponding to the candidate causal chain to be verified as the fault cause, specifically configured as follows: B1, in response to the verification result that the causal relationship of the candidate causal chain to be verified is established and the candidate causal chain to be verified is a software configuration layer abnormal causal chain, the cause of the failure is determined to be a software configuration layer abnormality. The causes of failure that pass the software test but fail the hardware test include hardware appearance abnormalities caused by software configuration layer abnormalities but not hardware failures.

[0191] The software configuration layer anomaly causal chain is one type of candidate causal chain. The root cause of this type of causal chain originates from an anomaly in the software configuration layer parameters, which, after propagation through internal system links, results in anomalies in hardware metrics.

[0192] An anomaly in the software configuration layer indicates that the root cause of the fault is that the parameters in the software configuration layer deviate from the preset standard.

[0193] Hardware anomalies refer to situations where the sensor output parameters do not meet requirements. These phenomena are caused by software-related factors, and there is no physical damage to the sensor hardware itself.

[0194] It should be noted that the causal chain of the software configuration layer anomaly has been verified and confirmed to be valid, indicating that the anomalies observed on the hardware side are caused by the transmission of abnormal parameters in the software configuration layer. At this time, the sensor hardware itself has not suffered physical damage; the hardware anomalies are merely a manifestation caused by external factors.

[0195] In some embodiments, the processor reads the type identifier of the candidate causal chain to be verified, and simultaneously reads the verification result already obtained. The processor determines that the verification result confirms the causal relationship, and that the candidate causal chain to be verified belongs to a software configuration layer anomaly causal chain. The processor records the fault cause as a software configuration layer anomaly. The processor simultaneously records that the phenomenon of the software test passing but the hardware test failing is a hardware manifestation anomaly caused by a software configuration layer anomaly, and that the sensor hardware itself is not faulty. The processor then passes this fault cause information to the step of generating repair information.

[0196] B2, in response to the verification result that the causal relationship of the candidate causal chain to be verified is valid and the candidate causal chain to be verified is a hardware fault causal chain, the cause of the fault is determined to be a hardware fault.

[0197] Hardware failure causal chains are one type of candidate causal chains. The root cause of this type of causal chain originates from a physical failure of the sensor hardware itself, directly resulting in abnormal hardware performance.

[0198] Hardware failure indicates that the root cause of the failure is a physical failure of the sensor hardware itself.

[0199] It should be noted that the hardware failure causal chain has been verified and confirmed to be valid, indicating that the abnormal hardware performance stems from a failure of the sensor hardware itself. There are no correctable parameters at the software level that can eliminate this anomaly; therefore, hardware component repair is necessary to resolve the fault.

[0200] In some embodiments, the processor reads the type identifier of the candidate causal chain to be verified, and also reads the verification result already obtained. The processor determines that the verification result indicates a causal relationship is established, and that the candidate causal chain to be verified belongs to a hardware fault causal chain. The processor records the fault cause as a hardware fault. The processor then passes this fault cause information to the step of generating repair information.

[0201] For example, in a scenario where a smart driving camera is offline for testing, if the causal chain verification for a software configuration layer anomaly is successful, the cause of the fault is a software configuration layer anomaly, and the low hardware signal-to-noise ratio is a hardware-related anomaly. The corresponding repair information points to modifying the software configuration parameters, and there is no need to replace the camera hardware. If the causal chain verification for a hardware fault is successful, the cause of the fault is a hardware fault, and the corresponding repair information points to replacing the camera hardware components.

[0202] In this embodiment, the final cause of the failure is determined based on the differences in the types of verified candidate causal chains. This accurately distinguishes between hardware anomalies caused by software configuration layer anomalies and actual hardware failures. In coupled failure scenarios where software tests pass but hardware tests fail, it can effectively identify false hardware failures and true hardware failures, avoiding misjudging hardware indicator deviations caused by abnormal software configuration layer parameters as hardware damage. This not only improves the precision and accuracy of failure root cause classification but also provides a basis for generating accurate repair strategies, reducing unnecessary hardware replacement costs and maintenance losses.

[0203] In this embodiment of the disclosure, the processor, in response to the verification result indicating that the causal relationship of the candidate causal chain to be verified is valid and that the candidate causal chain to be verified is a hardware fault causal chain, determines that the cause of the fault is a hardware fault, and is specifically configured as follows: B21, in response to the candidate causal chain to be verified being a hardware failure causal chain, replace the abnormal hardware component corresponding to the abnormal hardware event in the candidate causal chain to be verified.

[0204] Abnormal hardware events are the root cause nodes in the causal chain of hardware failures, recording relevant information about the occurrence of hardware abnormalities.

[0205] An abnormal hardware component is a sensor hardware part corresponding to an abnormal hardware event; it is a hardware object suspected of having undergone physical failure.

[0206] In some embodiments, the processor identifies the current candidate causal chain to be verified as a hardware failure causal chain. The processor reads the component identifier carried by the abnormal hardware event within the candidate causal chain to locate the abnormal hardware component. In the test execution environment, the abnormal hardware component is removed and replaced with a functional hardware component of the same model. The processor ensures that the software configuration, external test environment, and test case inputs remain in their original states, and only performs the hardware component replacement operation.

[0207] B22, re-perform tests on the software and hardware to obtain the corresponding sensor data within the coupling time interval.

[0208] It should be noted that after replacing the hardware components, the original test procedure needs to be reproduced, and sensor output data must be collected within the time range corresponding to the fault. Only by obtaining the sensor data after the hardware replacement can the sensor indicators be compared with those under the original fault condition to determine whether the hardware component replacement has eliminated the hardware anomaly.

[0209] In some embodiments, the processor maintains the software configuration and external test environment unchanged, and restarts the software and hardware to re-execute the original test cases. The processor collects all sensor sampling data streams generated during the complete test process. The processor filters sampling records whose time falls within the coupling time interval to obtain the corresponding sensor data within the coupling time interval, and caches this data for subsequent threshold comparison.

[0210] B23, in response to the reacquired sensor data conforming to a preset threshold range, determines that the causal relationship of the hardware failure causal chain is established.

[0211] In some embodiments, the processor compares the sensor data obtained after hardware replacement with a preset threshold range. When all the newly acquired sensor data falls within the preset threshold range, the processor determines that the causal relationship of the hardware failure causal chain is valid and saves the corresponding verification results.

[0212] It should be noted that if the sensor readings return to the acceptable normal range after replacing the suspected hardware component, it means that the hardware replacement eliminated the hardware anomaly. This experimental phenomenon confirms that the hardware failure itself is the real cause of the abnormal hardware readings, and the anomaly propagation path described by the hardware failure causal chain is valid.

[0213] B24, in response to the establishment of the causal relationship in the hardware failure causal chain, determines that the cause of the failure is a hardware failure.

[0214] The causal chain of hardware failure was confirmed after hardware replacement and retesting, indicating that the root cause of the failure was physical damage to the hardware component itself.

[0215] In some embodiments, the processor reads the verification result corresponding to the causal chain of the hardware failure. When the verification result indicates that the causal relationship is true, the processor records the cause of the failure as a hardware failure. The processor outputs this cause of failure and passes it to the subsequent step of generating repair information.

[0216] In yet another embodiment of this disclosure, the processor is specifically configured as follows: C1, in response to a fault type where the software test passes but the hardware test fails, performs time series analysis on the software event-sensor data matching pair to determine the coupling time interval between the software anomaly and the hardware anomaly.

[0217] C2, based on a pre-set fault knowledge graph, performs causal reasoning on software and hardware anomalies within the coupling time interval to obtain the causal relationship of the anomalies.

[0218] Fault knowledge graphs are used to characterize the causal relationships between software anomalies, hardware anomalies, and software-hardware interaction anomalies.

[0219] The fault knowledge graph is a pre-built structured knowledge base. It contains software anomaly entities, hardware anomaly entities, and software-hardware interaction anomaly entities. It also stores the association edges between different entities that represent causal transmission, and is used to record the objectively existing domain causal relationships between various anomalies.

[0220] Software anomalies refer to abnormal behaviors or states that occur at the software level within the coupling time interval.

[0221] Hardware malfunction refers to abnormal phenomena exhibited by the sensor hardware output acquired within the coupling time interval.

[0222] Abnormal causal relationships are the causal transmission relationships between entities obtained through knowledge graph reasoning, representing the correspondence between the causes and manifestations of observed software and hardware anomalies.

[0223] In some embodiments, the processor reads all software and hardware anomaly features within the coupling time interval. The processor retrieves a preset fault knowledge graph and performs entity matching between the observed software and hardware anomalies and the entity nodes within the fault knowledge graph. The processor performs traversal reasoning along the causal relationship edges within the graph to find upstream anomaly entities that can explain the current hardware anomaly. The processor organizes the propagation logic between the inferred entities, generates anomaly causal relationships, and outputs these relationships to determine the cause of the fault.

[0224] C3, based on the abnormal causal relationship, determine the cause of the failure.

[0225] In some embodiments, the processor reads the causal relationships of anomalies and traverses the causal transmission links obtained through reasoning. The processor locates the root cause entity in the causal transmission link where there is no upstream inducing entity, and determines the anomaly type corresponding to the root cause entity as the cause of the failure. The processor saves the cause of the failure for subsequent generation of repair information.

[0226] C4 generates repair information based on the cause of the fault.

[0227] In some embodiments, the processor reads the determined fault cause. The processor retrieves a preset mapping relationship between the fault cause and repair information. The processor matches the corresponding handling content according to the fault cause and generates repair information. If the fault cause is a software anomaly, the repair information includes software configuration items to be adjusted; if the fault cause is a hardware anomaly, the repair information includes hardware component replacement instructions; if the fault cause is a software / hardware interaction anomaly, the repair information includes software / hardware interaction interface parameter adjustment instructions. The processor stores the repair information, along with the coupling time interval and the causal relationship of the anomaly, in the test record.

[0228] In yet another embodiment of this disclosure, after the processor determines the test score of the intelligent driving system, it is configured to: D1, obtain the historical fault distribution table of the intelligent driving system.

[0229] The historical fault distribution table is a pre-stored statistical data table. It compiles historical test samples from multiple intelligent driving systems of the same model, recording software event characteristics, sensor data characteristics, historical test scores, and corresponding historical fault results. The historical fault distribution table can be continuously updated during the equipment's offline testing phase.

[0230] D2, based on software events and sensor data, retrieves fault results that match the intelligent driving system from the historical fault distribution table.

[0231] In some embodiments, the processor acquires software events and sensor data collected during the current test. The processor compares the features of the current software events and sensor data with the features of each historical sample in the historical fault distribution table. The processor selects historical samples whose feature similarity meets a preset similarity threshold. The processor extracts the fault results corresponding to the selected historical samples as the fault results matched with the current intelligent driving system.

[0232] D3 uses the fault results to verify the test score.

[0233] In some embodiments, the processor reads the matched fault result and parses the historical anomaly level and historical score range corresponding to the fault result. The processor then compares the current test score with the historical score range.

[0234] If the current test score falls within the historical score range corresponding to the historical fault results, the verification result is that the test score is reasonable, and the current test score is retained to continue the subsequent fault analysis process.

[0235] If the current test score exceeds the historical score range corresponding to historical fault results, the verification result indicates a suspicious bias in the test score. The processor outputs a suspicious score warning, and you can choose to re-collect software events and sensor data, recalculate the test score, and then proceed with the subsequent fault analysis process.

[0236] It should be noted that the current test score reflects the degree of system anomaly obtained in this test. The matched historical fault results reflect the actual fault situations corresponding to historical samples with similar characteristics. Comparing the two can verify whether the current test score conforms to historical statistical patterns. When the degree of anomaly reflected in the test score deviates significantly from that in the historical fault results, an anomaly in the current test score can be identified, eliminating erroneous scores caused by transient noise or sampling anomalies.

[0237] In another embodiment of this disclosure, the processor employs a multi-layered progressive fusion judgment logic to determine the test results. This judgment process is divided into three levels: L1 rapid screening of the software domain, L2 dynamic threshold determination of the sensor domain, and L3 in-depth cross-domain correlation analysis. Scenarios with obvious anomalies in the software domain can directly output the judgment result at the first level, without executing subsequent levels of calculation and analysis, thus saving processor computing resources. Only when the current level of judgment fails to identify a fault will the next level be used to perform more complex cross-domain correlation calculations.

[0238] After determining the software test score, hardware test score, and event correlation score, the processor is configured as follows: E1 indicates that the software test has failed because the software test score exceeds a preset software failure threshold.

[0239] A preset software fault threshold is defined as a pre-defined threshold for judging software domains. If the software test score exceeds this threshold, it indicates that the software domain has reached a fault level.

[0240] L1 is the software domain rapid screening level, where the software test score characterizes the overall anomaly level of the software domain. In some embodiments, the processor obtains the software test score and compares it with a preset software fault threshold. When the software test score exceeds the preset software fault threshold, the processor directly determines that the software test has failed, ending this multi-layer progressive fusion judgment process and no longer executing subsequent hardware and cross-domain related judgment steps.

[0241] E2, in response to the software test score not exceeding the preset software fault threshold, determines whether the hardware test score exceeds the preset hardware fault threshold.

[0242] If no faults are detected in the software domain screening, it indicates that the overall software operation is within an acceptable range, and the judgment process proceeds to the L2 sensor domain dynamic threshold judgment level. At this point, it is necessary to further evaluate the anomaly level of the sensor hardware domain to provide a basis for the hardware test result judgment.

[0243] The preset hardware fault threshold is a pre-defined hardware domain judgment threshold used to measure whether the sensor hardware has reached a fault level.

[0244] In some embodiments, when the software test score does not exceed a preset software fault threshold, the processor reads the hardware test score. The processor compares the hardware test score with the preset hardware fault threshold to obtain the hardware test result.

[0245] E3 indicates that the hardware test has failed because the hardware test score exceeds a preset hardware failure threshold.

[0246] The L2 level completes the hardware domain determination. If the hardware test score exceeds the preset hardware fault threshold, provided that no fault is detected in the software domain, it indicates that there is a significant abnormality in the sensor hardware domain. The hardware test fails directly, and the determination process is terminated.

[0247] E4, in response to the hardware test score not exceeding the preset hardware failure threshold, determines whether the event association score exceeds the preset cross-domain risk increment threshold.

[0248] A preset cross-domain risk increment threshold is a pre-set cross-domain coupling risk judgment standard used to identify scenarios where single-domain indicators are normal but cross-domain interactions pose potential risks.

[0249] E5 indicates that the software test has failed because the event correlation score exceeds the preset cross-domain risk increment threshold.

[0250] At level L3, a deep cross-domain correlation analysis was completed. Neither the software domain independent score nor the hardware domain independent score triggered the fault criteria. However, abnormal coupling existed at the software-hardware interaction level, and the root cause of this anomaly typically originated from the interaction behavior on the software side. Therefore, the software test was deemed a failure, triggering the subsequent fault cause analysis process. If the event correlation score also did not exceed the preset cross-domain risk increment threshold, both the software and hardware tests were deemed successful, and there was no need to proceed to the subsequent fault cause analysis steps.

[0251] Exemplary application scenarios This example uses an EOL (End-of-Life) test scenario for a forward-looking camera used in intelligent driving. The test case task is to identify a pedestrian target ahead, and the test process simultaneously collects software event sequences and camera sensor data.

[0252] The software test score was low, but the overall software performance was deemed satisfactory. However, the signal-to-noise ratio (SNR) of the camera's pedestrian region of interest deteriorated, resulting in a higher hardware test score. The overall test score fell between the first and second thresholds, prompting the processor to enter the fault analysis phase.

[0253] First, the processor receives software events. Sensor data matching pairs are used to extract subsets of software and hardware abnormal events.

[0254] The software exception event subset contains two software events. One software event occurs at approximately 0.080 seconds and its content is that the actual loaded value of the external parameter Pitch configuration parameter deviates from the standard value. The other software event occurs at approximately 0.125 seconds and its content is that the AE (Auto Exposure) metering ROI (Region of Interest) has shifted (triggered by the Pitch parameter loading deviation).

[0255] The hardware anomaly event subset contains multiple camera sampling records, with sampling times ranging from 0.112 seconds to 0.312 seconds. The signal-to-noise ratio of the pedestrian's region of interest is consistently lower than the preset threshold range corresponding to the target scene.

[0256] The processor then aggregates the subset of software exception events to generate a software exception pattern. The first exception event type corresponding to the software exception pattern is configuration parameter loading exception, which occurs at 0.080 seconds and is associated with the camera perception configuration module; the second exception event type corresponding to the software exception pattern is AE metering ROI offset, which occurs at 0.125 seconds.

[0257] The processor aggregates a subset of hardware anomaly events to generate hardware anomaly patterns. The corresponding anomaly sensor data for these patterns is characterized by a low signal-to-noise ratio and an anomaly occurring within a time interval of 0.112 seconds. 0.312 seconds, sensor type is forward-facing camera.

[0258] Time series analysis determined the coupling time interval to be 0.080 seconds. 0.312 seconds.

[0259] Next, the processor reads the software configuration layer parameters that are effective within the coupling time interval, compares the actual parameters with the preset parameters item by item, and obtains the software configuration layer test results. The software configuration layer test results show that the actual value of the external parameter Pitch deviates from the preset parameter.

[0260] The processor combines software configuration layer test results, software anomaly modes, hardware anomaly modes, and preset domain causality coefficients to construct multiple candidate causal chains.

[0261] The first chain is a causal chain of software configuration layer anomalies: the root cause is an abnormal external parameter (Pitch) configuration, which triggers an AE metering ROI shift, further causing a hardware anomaly resulting in a decrease in the signal-to-noise ratio (SNR) of the pedestrian's region of interest. Nodes within the chain are arranged chronologically: node 1 is the software configuration parameter anomaly event node, node 2 is the AE metering ROI shift software anomaly event node, and node 3 is the low camera SNR hardware anomaly event node. The causal direction between nodes is determined based on their chronological order. The processor calculates the temporal correlation between adjacent anomaly event nodes within the chain. The time deviation between the parameter loading event and the AE metering shift event is 0.045 seconds, and the time deviation between the AE metering shift event and the SNR anomaly sampling time is 0.032 seconds. Two sets of temporal correlations are calculated for each.

[0262] The second is a hardware failure causal chain: the root cause event is damage to the camera's photosensitive hardware, which directly outputs a hardware anomaly with a low signal-to-noise ratio. The chain only contains hardware anomaly event nodes.

[0263] The third is the environmental failure causal chain: the root cause event is the abnormal illumination of the supplementary lighting at the test station, which causes a decrease in the signal-to-noise ratio of the camera.

[0264] The processor retrieves the domain causality coefficients corresponding to each candidate causal chain, and calculates the score for each candidate causal chain by combining the temporal correlation of each node within the chain. For example, in the case of a half-width time window ΔT = 0.2s, based on w_ij = 1... From Δt_ij / ΔT, we know that the temporal correlation between node 1 and node 2 is w_1 = 1 - 0.045 / 0.2 = 0.775, and the temporal correlation between node 2 and node 3 is w_2 = 1 - 0.032 / 0.2 = 0.84. The average temporal correlation between nodes in the chain is (w_1 + w_2) / 2 = (0.775 + 0.84) / 2 = 0.8075 ≈ 0.81.

[0265] For example, the domain causality coefficient of the software configuration layer anomaly causal chain is 0.88, and the average temporal correlation degree of the nodes within the chain is 0.81. Based on the preset calculation rules, the score = λ•(domain causality coefficient) + ( λ)•(Average temporal correlation of nodes within the chain), taking a value of λ of 0.5 as an example, the calculated chain score is 0.845. Similarly, the hardware failure causal chain score is 0.42, and the environmental failure causal chain score is 0.25.

[0266] The default condition in this example is to select the candidate causal chain with the highest score as the candidate causal chain to be verified. Therefore, the abnormal causal chain of the software configuration layer is selected as the candidate causal chain to be verified.

[0267] After that, the processor enters the causal relationship verification process.

[0268] The first step is to obtain the abnormal software configuration layer parameters corresponding to the software abnormal events in the candidate causal chain to be verified, which are the external parameters Pitch parameters that have deviated.

[0269] The second step is to perform a correction operation on the abnormal software configuration layer parameters, adjusting the external parameter Pitch to the preset parameters.

[0270] Third, keeping all other test conditions unchanged, re-execute the test cases on the software and hardware to obtain the coupling time interval of 0.080 seconds. The camera sensor data corresponding to 0.312 seconds.

[0271] The fourth step is to compare the sensor signal-to-noise ratio obtained from the retest with the preset threshold range. After the retest, the signal-to-noise ratio of the pedestrian region of interest returns to the acceptable range, meeting the preset threshold range requirements.

[0272] Based on this, the processor determines that the causal relationship of the candidate causal chain to be verified is valid. The processor identifies the abnormal software configuration layer parameters as the cause of the fault. The abnormal hardware indicators manifested by the fault are symptoms caused by abnormal software parameters, and not damage to the camera hardware itself.

[0273] The processor ceases verification of other candidate causal chains. Based on the identified fault cause, the processor generates repair information. The repair information specifies that the root cause is an abnormal external parameter (Pitch) configuration parameter, and the corresponding remedial action is to update the Pitch parameter to a preset standard value; replacement of the camera hardware is not required. The repair information, along with the coupling time interval, software configuration layer test results, and candidate causal chain information, is written into the test log and output to the test report system.

[0274] If, after correcting the external parameter Pitch, the camera signal-to-noise ratio remains below the preset threshold within the retested coupling time interval, the processor determines that the causal chain of the current software configuration layer is invalid.

[0275] The processor excludes the already verified candidate causal chain and selects the hardware failure causal chain with the highest score from the remaining candidate causal chains as the new candidate causal chain to be verified, initiating the next round of verification. For the hardware failure causal chain, a hardware component replacement verification operation is performed, and the test is executed again after replacing the camera hardware. If the sensor indicators return to normal after replacing the hardware, the cause of the failure is determined to be a fault in the camera hardware itself, and corresponding repair information for the replaced hardware is generated.

[0276] Exemplary methods Figure 10 This is a schematic flowchart illustrating a testing method provided in an exemplary embodiment of this disclosure. This embodiment can be applied to a processor, a chip, or a domain controller, such as... Figure 10 As shown, it includes the following steps: S1001, Obtain test results; The test results are obtained by performing tests on the intelligent driving system based on test cases; S1002, Based on the test results, determine the software test score and software event sequence; S1003, obtain the sensor data and hardware test score corresponding to the test case; S1004, Based on software event sequences and sensor data, determine the event association score; S1005, based on software test scores, hardware test scores, and event correlation scores, determines the test score of the intelligent driving system.

[0277] In some possible implementations, event correlation scores are determined based on software event sequences and sensor data, including: Based on the software event sequence, determine the software event time sequence corresponding to the software event sequence; Based on sensor data, determine the acquisition time sequence corresponding to the sensor data; Based on the software event time sequence and the acquisition time sequence, determine the software event-sensor data matching pair; The event association score is determined based on the matching pairs of software events and sensor data.

[0278] In some possible implementations, event association scores are determined based on software event-sensor data matching pairs, including: Obtain the temporal correlation between software events and sensor data in software event-sensor data matching pairs; The mean of the time correlation is determined based on the number of software event-sensor data matching pairs and the time correlation. The event association score is determined based on the mean of time correlation.

[0279] In some possible implementations, the temporal correlation between software events and sensor data in a software event-sensor data matching pair is obtained, including: Based on the software event time and sensor data acquisition time of the software event in the software event-sensor data matching pair, the time deviation between the software event and the sensor data is determined. Calculate the ratio of time deviation to time window; The degree of temporal correlation is determined based on the ratio.

[0280] In some possible implementations, software test scores are determined in the following ways: Based on the test results, a first score representing the importance of the module to which the test case belongs to the intelligent driving system, a second score representing the impact of the module to which the test case belongs on the scope of the intelligent driving system fault, and a third score representing the severity level of the intelligent driving system fault of the module to which the test case belongs; The software test score is determined based on the first score, the second score, and the third score.

[0281] In some possible implementations, hardware test scores are determined as follows: Determine the deviation between the sensor data and the preset threshold; The hardware test score is determined based on the deviation.

[0282] In some possible implementations, the preset threshold is determined based on the mean and standard deviation of historical sensor data of the target scenario; the target scenario is the scenario in which the intelligent driving system is tested based on test cases.

[0283] In some possible implementations, the test is determined to pass if the test score is less than a first threshold. If the test score is greater than or equal to the second threshold, the test is determined to be failed. In response to a test score that is greater than or equal to a first threshold and less than a second threshold, a fault analysis step is triggered to determine the fault type. The first threshold is less than the second threshold.

[0284] In some possible implementations, in response to a fault type where the software test passes but the hardware test fails, time series analysis is performed on the software event-sensor data matching pair to determine the coupling time interval between the software anomaly and the hardware anomaly; the software event-sensor data matching pair is determined based on the software event time sequence and the acquisition time sequence. Within the coupling time interval, obtain the software configuration layer parameters corresponding to the software events; the software configuration layer parameters include parameters that were not validated when obtaining the software test scores; Determine the cause of the fault based on software configuration layer parameters; Based on the cause of the fault, repair information is generated.

[0285] In some possible implementations, the cause of the failure is determined based on software configuration layer parameters, including: The test results of the software configuration layer are determined based on the software configuration layer parameters and preset parameters; Based on the test results of the software configuration layer, the software anomaly patterns and hardware anomaly patterns within the coupling time interval, and the preset domain causality coefficient, the cause of the failure is determined; the domain causality coefficient is used to characterize the confidence level of the causal relationship between software anomalies and hardware anomalies.

[0286] In some possible implementations, the cause of the failure is determined based on the software configuration layer test results, software anomaly patterns within the coupling time interval, hardware anomaly patterns, and preset domain causality coefficients, including: Based on the test results of the software configuration layer, the software anomaly patterns and hardware anomaly patterns within the coupling time interval, and the preset domain causality coefficients, at least one candidate causal chain is constructed. The score of each candidate causal chain is determined based on the domain causality coefficient and temporal correlation degree corresponding to the candidate causal chain. Perform causal relationship verification on candidate causal chains that meet the preset conditions to obtain the verification results; Based on the verification results, the cause of the failure was determined.

[0287] In some possible implementations, causal relationship verification is performed on candidate causal chains that meet preset conditions to obtain verification results; based on the verification results, the causes of failures are determined, including: Obtain the abnormal software configuration layer parameters corresponding to the software abnormal events in the candidate causal chain to be verified; Perform a correction operation on the abnormal software configuration layer parameters; the correction operation is used to adjust the abnormal software configuration layer parameters to the preset parameters; Re-perform tests on the software and hardware to obtain the corresponding sensor data within the coupling time interval; If the reacquired sensor data meets the preset threshold range, the verification result is determined to be that the causal relationship of the candidate causal chain to be verified is established; In response to the verification result indicating that the causal relationship of the candidate causal chain to be verified is established, the fault type corresponding to the candidate causal chain to be verified is determined as the fault cause; If the reacquired sensor data does not meet the preset threshold range, the verification result is determined to be that the causal relationship of the candidate causal chain to be verified is not valid. In response to the verification result indicating that the causal relationship of the candidate causal chain to be verified is not established, causal relationship verification is performed sequentially for candidate causal chains that meet the preset conditions, in descending order of score, until the cause of the failure is determined based on the verification result.

[0288] In some possible implementations, in response to the verification result indicating that the causal relationship of the candidate causal chain to be verified is established, the fault type corresponding to the candidate causal chain to be verified is determined as the fault cause, including: In response to the verification result that the causal relationship of the candidate causal chain to be verified is established and the candidate causal chain to be verified is a software configuration layer abnormal causal chain, the cause of the failure is determined to be a software configuration layer abnormality. The causes of failure that pass the software test but fail the hardware test include hardware appearance abnormalities caused by software configuration layer abnormalities but not hardware failures. If the verification result indicates that the causal relationship of the candidate causal chain to be verified is valid and the candidate causal chain to be verified is a hardware fault causal chain, the cause of the fault is determined to be a hardware fault.

[0289] Exemplary electronic devices Figure 11 A structural diagram of an electronic device provided in an embodiment of this disclosure includes at least one processor 11 and a memory 12.

[0290] The processor 11 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0291] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute one or more computer program instructions to implement the test methods of the various embodiments of this disclosure described above, and / or other desired functions.

[0292] In one example, the electronic device 10 may also include an input device 13 and an output device 14, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0293] The input device 13 may also include, for example, a keyboard, a mouse, etc.

[0294] The output device 14 can output various information to the outside, including, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0295] Of course, for the sake of simplicity, Figure 11Only some of the components of the electronic device 10 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 10 may include any other suitable components depending on the specific application.

[0296] Exemplary computer program products and computer-readable storage media In addition to the methods and apparatus described above, embodiments of this disclosure may also provide a computer program product, including computer program instructions that, when executed by a processor, cause the processor to perform the steps of the test methods of the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0297] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0298] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps of the test methods in the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0299] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0300] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0301] Various modifications and variations can be made to this disclosure without departing from its spirit and scope. Therefore, this disclosure is also intended to include such modifications and variations if they fall within the scope of the claims of this disclosure and their equivalents.

Claims

1. A testing apparatus, comprising a processor, the processor being configured to: Obtain test results; the test results are obtained by performing tests on the intelligent driving system based on test cases; Based on the test results, the software test score and software event sequence are determined; Obtain the sensor data and hardware test scores corresponding to the test cases; Based on the software event sequence and the sensor data, an event association score is determined; The test score of the intelligent driving system is determined based on the software test score, the hardware test score, and the event association score.

2. The testing apparatus according to claim 1, wherein, The processor is specifically configured as follows: Based on the software event sequence, determine the software event time sequence corresponding to the software event sequence; Based on the sensor data, determine the acquisition time sequence corresponding to the sensor data; Based on the software event time sequence and the acquisition time sequence, determine the software event-sensor data matching pair; The event association score is determined based on the matching pairs of software events and sensor data.

3. The testing apparatus according to claim 2, wherein, The processor is specifically configured as follows: Obtain the temporal correlation between the software event and the sensor data in the software event-sensor data matching pair; The mean of the time correlation degree is determined based on the number of software event-sensor data matching pairs and the time correlation degree. The event association score is determined based on the average time correlation score.

4. The testing apparatus according to claim 3, wherein, The processor is specifically configured as follows: Based on the software event time of the software event and the acquisition time of the sensor data in the software event-sensor data matching pair, the time deviation between the software event and the sensor data is determined. Calculate the ratio of the time deviation to the time window; Based on the ratio, the time correlation degree is determined.

5. The testing apparatus according to claim 1, wherein, The processor is specifically configured as follows: Based on the test results, a first score representing the importance of the module to which the test case belongs to the intelligent driving system, a second score representing the impact range of the intelligent driving system fault on the module to which the test case belongs, and a third score representing the severity level of the intelligent driving system fault on the module to which the test case belongs; The software test score is determined based on the first score, the second score, and the third score.

6. The testing apparatus according to claim 1, wherein, The processor is specifically configured as follows: Determine the deviation of the sensor data from a preset threshold; The hardware test score is determined based on the deviation.

7. The testing apparatus according to claim 6, wherein, The preset threshold is determined based on the mean and standard deviation of historical sensor data of the target scenario; the target scenario is the scenario in which the intelligent driving system is tested based on the test cases.

8. The testing apparatus according to any one of claims 1-7, wherein, The processor is specifically configured as follows: The test is deemed passed if the test score is less than a first threshold. If the test score is greater than or equal to the second threshold, the test is determined to be failed. In response to the test score being greater than or equal to the first threshold and less than the second threshold, a fault analysis step is triggered to obtain the fault type. Wherein, the first threshold is less than the second threshold.

9. The testing apparatus according to claim 8, wherein, The processor is specifically configured as follows: In response to a fault type where software testing passes but hardware testing fails, time series analysis is performed on the software event-sensor data matching pair to determine the coupling time interval between the software anomaly and the hardware anomaly; the software event-sensor data matching pair is determined based on the software event time sequence and the acquisition time sequence. Within the coupling time interval, software configuration layer parameters corresponding to software events are obtained; the software configuration layer parameters include parameters that were not validated when obtaining the software test score. Based on the software configuration layer parameters, the cause of the fault is determined; Based on the cause of the fault, repair information is generated.

10. The testing apparatus according to claim 9, wherein, The processor is specifically configured as follows: Based on the software configuration layer parameters and preset parameters, the software configuration layer test results are determined; Based on the test results of the software configuration layer, the software anomaly mode and hardware anomaly mode within the coupling time interval, and the preset domain causality coefficient, the cause of the failure is determined. The domain causality coefficient is used to characterize the confidence level of a causal relationship between software anomalies and hardware anomalies.

11. The testing apparatus according to claim 10, wherein, The processor is specifically configured as follows: Based on the test results of the software configuration layer, the software anomaly mode and hardware anomaly mode within the coupling time interval, and the preset domain causality coefficient, at least one candidate causal chain is constructed. Based on the domain causality coefficient and temporal correlation degree corresponding to the candidate causal chains, the score of each candidate causal chain is determined; Perform causal relationship verification on candidate causal chains that meet the preset conditions to obtain the verification results; Based on the verification results, the cause of the failure was determined.

12. The testing apparatus according to claim 11, wherein, The processor is specifically configured as follows: Obtain the abnormal software configuration layer parameters corresponding to the software abnormal events in the candidate causal chain to be verified; A correction operation is performed on the abnormal software configuration layer parameters; the correction operation is used to adjust the abnormal software configuration layer parameters to the preset parameters; Re-perform the tests on the software and hardware to obtain the corresponding sensor data within the coupling time interval; In response to the reacquired sensor data conforming to a preset threshold range, the verification result is determined to be that the causal relationship of the candidate causal chain to be verified is established; In response to the verification result indicating that the causal relationship of the candidate causal chain to be verified is established, the fault type corresponding to the candidate causal chain to be verified is determined as the fault cause; If the reacquired sensor data does not meet the preset threshold range, the verification result is determined to be that the causal relationship of the candidate causal chain to be verified is not established; In response to the verification result indicating that the causal relationship of the candidate causal chain to be verified is not established, causal relationship verification is performed sequentially on the candidate causal chains whose scores meet the preset conditions, in descending order of scores, until the cause of the failure is determined based on the verification result.

13. The testing apparatus according to claim 12, wherein, The processor is specifically configured as follows: In response to the verification result that the causal relationship of the candidate causal chain to be verified is established and the candidate causal chain to be verified is a software configuration layer abnormal causal chain, the cause of the failure is determined to be a software configuration layer abnormality. The cause of the failure where the software test passes but the hardware test fails includes hardware appearance abnormalities caused by software configuration layer abnormalities but not hardware failures. In response to the verification result that the causal relationship of the candidate causal chain to be verified is established and the candidate causal chain to be verified is a hardware failure causal chain, the cause of the failure is determined to be a hardware failure.

14. A testing method, comprising: Obtain test results; The test results are obtained by performing tests on the intelligent driving system based on test cases. Based on the test results, the software test score and software event sequence are determined; Obtain the sensor data and hardware test scores corresponding to the test cases; Based on the software event sequence and the sensor data, an event association score is determined; The test score of the intelligent driving system is determined based on the software test score, the hardware test score, and the event association score.

15. A computer-readable storage medium storing a computer program that is executed by a processor to perform the test method of claim 14.

16. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory, and execute the executable instructions to implement the test method of claim 14.

Images