End-to-end large model bi-directional failure boundary scan and hysteresis effect evaluation method and apparatus

Abstract

The application relates to the technical field of automatic driving car simulation, in particular to a kind of end-to-end big model two-way failure boundary scanning and hysteresis effect evaluation method and equipment.The method comprises the following steps: different intensity failure factors in limit failure scene factor library are generalized and combined to obtain different limit failure scenes;end-to-end intelligent driving big model is tested;hysteresis coefficient is calculated according to failure threshold and recovery threshold;the bottom strategy of end-to-end intelligent driving big model after failure is determined according to the size of hysteresis coefficient.The application calculates hysteresis coefficient to quantify system recovery capability, which helps to promote the algorithm progress of driving big model.

Description

Technical Field

[0001] This application relates to the field of autonomous vehicle simulation technology, and more specifically, to a method and device for end-to-end large-scale bidirectional failure boundary scanning and hysteresis effect evaluation. Background Technology

[0002] End-to-end intelligent driving big model (hereinafter referred to as end-to-end big model) refers to a machine learning system that uses a single deep neural network architecture (usually based on Transformer or diffusion model) to directly map raw multimodal sensor data (such as surround view camera images, LiDAR point clouds, millimeter-wave radar signals) into vehicle low-level control commands (such as steering wheel angle, acceleration / brake pedal opening, gear position).

[0003] Currently, the application of end-to-end intelligent driving large-scale models in real-world road environments faces severe safety verification challenges. Existing technologies for evaluating the safety of autonomous driving systems mainly suffer from the following shortcomings: 1. Existing security boundary testing methods have the limitation of being one-way.

[0004] Existing safety boundary patent technologies generally employ a one-way testing strategy: starting from a safety scenario, the difficulty of the scenario is gradually increased until the system fails for the first time, thereby determining the system's failure threshold. This method sets the test endpoint at the critical point of system failure, only answering the single question of "under what conditions will the system fail." This type of method has been widely used in industry for boundary testing of sensor perception capabilities, and its testing logic is typically: within the safety boundary, the system operates autonomously; outside the boundary, manual intervention is triggered. This binary evaluation framework simplifies the system state to two discrete states: "normal" and "failure," ignoring the possible recovery process that may occur after a system failure.

[0005] 2. Existing technologies lack the ability to assess system recovery capabilities.

[0006] In real-world road environments, autonomous driving systems face dynamically changing operating conditions. For example, the dirtiness of a camera may improve as windshield wipers operate, and traffic congestion may ease over time. Under these dynamic conditions, the system's behavior in recovering from a failed state to a normal state—including the conditions required for recovery, the speed of recovery, and the presence of recovery lag—has a significant impact on vehicle safety. However, existing testing methods terminate testing after the system's first failure, failing to capture the system's recovery performance after a failure, let alone quantify its recovery capabilities.

[0007] 3. Existing security boundary assessments focus on the sensor perception dimension.

[0008] Existing research on safety boundary patents largely focuses on boundary testing of sensor perception capabilities, such as sensor-level failure scenarios like camera dirt, radar obstruction, and changes in lighting. However, failures in end-to-end intelligent driving models not only stem from sensor perception failures but also exist more broadly in system-level integrated operating conditions: when a vehicle is driving through a complex intersection and encounters a cluster of polyhedral obstacles, the system's computing power may approach its limit; when realistic images of people or vehicles appear on highway billboards or bus bodies, the algorithm may make cognitive misjudgments. These multi-source coupled failure scenarios involving computing power limits and algorithmic cognitive defects exceed the coverage of traditional sensor safety boundary testing, and existing technologies have not yet formed a systematic evaluation method.

[0009] 4. Existing fallback strategies lack scenario-specific design.

[0010] Based on the results of one-way boundary testing, the fallback strategy adopted by existing technologies is usually a uniform "manual takeover when the boundary is exceeded" model. This strategy does not consider the differences in system recovery capabilities after failure, nor does it differentiate between different failure types, which may lead to two situations: first, it is too conservative, frequently requesting manual intervention in scenarios where unnecessary takeover is required, affecting user experience; second, it is too aggressive, failing to take sufficient security measures in a timely manner in scenarios with poor system recovery capabilities, posing security risks.

[0011] In view of the above, this application is hereby submitted. Summary of the Invention

[0012] The purpose of this application is to provide an end-to-end large-scale model bidirectional failure boundary scanning and hysteresis effect evaluation method and device. By actively creating dangerous scenario accidents, the extreme conditions of the driving large-scale model are tested, the hysteresis effect is introduced, the system failure boundary and recovery boundary are explored, and the hysteresis coefficient is calculated to quantify the system recovery capability, which helps to promote the algorithmic progress of driving large-scale models.

[0013] To achieve the above objectives, this application adopts the following technical solution: Firstly, this application provides an end-to-end large-scale model bidirectional failure boundary scanning and hysteresis effect assessment method, including: Construct a library of extreme failure scenario factors, including: sensor interference factor, computing power limit factor, and perception trap factor; Different extreme failure scenarios are obtained by generalizing and combining failure factors of different intensities in the extreme failure scenario factor library. The end-to-end intelligent driving model was tested using different extreme failure scenarios. During the test, the intensity of the failure factor was controlled from weak to strong, and the intensity of the failure factor when the end-to-end intelligent driving model first failed was recorded as the failure threshold. During the test, the intensity of the failure factor was controlled from strong to weak, and the intensity of the failure factor when the end-to-end intelligent driving model first recovered was recorded as the recovery threshold. Calculate the hysteresis coefficient based on the failure threshold and recovery threshold; Based on the magnitude of the hysteresis coefficient, a fallback strategy is determined after the failure of the end-to-end intelligent driving big model.

[0014] Secondly, this application provides an electronic device, comprising: At least one processor, and a memory communicatively connected to at least one of the processors; The memory stores instructions that can be executed by at least one of the processors, which enable the at least one processor to perform an end-to-end large model bidirectional failure boundary scanning and hysteresis effect assessment method.

[0015] Compared with the prior art, this application has the following beneficial effects: This application introduces a bidirectional boundary scanning mechanism to obtain both the failure threshold and recovery threshold of the large model, thereby achieving a joint evaluation of the "failure conditions" and "recovery conditions" of the large model. This solves the problem that existing testing methods only perform unidirectional boundary searches and neglect the evaluation of system recovery capabilities.

[0016] This application expands the evaluation scope from the sensor level to system-level comprehensive failure scenarios by constructing a library of extreme failure scenario factors covering multiple factors such as sensor interference, computing power limits, and perception traps, thus solving the problem that existing security boundary testing is limited to the sensor perception dimension.

[0017] Based on the hysteresis coefficient quantification results, this application matches differentiated fallback strategies for different systems, avoiding the imbalance between efficiency and security caused by a uniform "takeover when the boundary is exceeded" mode, and solving the problem of the "one-size-fits-all" fallback strategy in the existing system. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1This is a flowchart illustrating an end-to-end large model bidirectional failure boundary scanning and hysteresis effect assessment method provided in an embodiment of this application. Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0020] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0021] Figure 1 This is a flowchart illustrating a bidirectional failure boundary scanning and hysteresis effect assessment method for an end-to-end large-scale model, as provided in this embodiment. This embodiment is applicable to simulation testing of an end-to-end intelligent driving large-scale model. This method can be executed by an electronic device.

[0022] See Figure 1 The method provided in this embodiment includes: S110. Construct a library of extreme failure scenario factors, including: sensor interference factor, computing power limit factor, and perception trap factor.

[0023] Table 1 Factor Description Table

[0024] Table 1 lists sensor interference factors including: visual contamination, optical glare, and radar interference; computational power limitation factors including: irregularly shaped obstacles, extremely high complexity, and different types of road shoulders; and perception trap factors including: misleading 2D images, adversarial textures, and semantically ambiguous scenes. Table 1 also provides a detailed description of each failure factor.

[0025] Sensor interference factors reproduce real-world interference by precisely superimposing various noise, occlusion, or attack signals at the sensor model level. For example, visual dirt can be simulated by dynamically synthesizing textures such as stains, mud spots, and raindrops in a camera sensor model; optical glare can be simulated by using a physical rendering engine to simulate the complex light path reflection, scattering, and lens flare of strong light sources (such as the sun or oncoming headlights) within the lens, and by adjusting the position, intensity, and duration of the glare; precise modulated false signals can be injected into a radar sensor model to create "false targets" that do not exist in reality to simulate radar interference; or, point cloud anomalies caused by multipath reflection and electromagnetic interference in a millimeter-wave radar model can be simulated.

[0026] The computational power limit factor introduces scenarios with far greater complexity than conventional algorithms into simulations, forcing large models to fail under conditions of limited computing resources. For example, AI-enhanced scene generation techniques (such as generative adversarial networks) are used to create long-tailed samples that rarely appear in the training set, such as bicycles hanging on the back of a campervan or tarpaulins not properly secured on a truck, to simulate irregular obstacles; high-density scenes containing a large number of dynamic elements are constructed, such as intersections where more than 100 traffic participants are simultaneously competing, or extreme scenarios with adversarial, rare, and ambiguous elements are added to simulate extremely high complexity; and different types of road shoulders, such as road shoulders, guardrails, and water-filled barriers, are constructed in the scenes to simulate different types of road shoulders.

[0027] Perceptual traps primarily generate physically realistic traps that can deceive algorithms through adversarial attacks and content editing. For example, AI technology can be used to implant realistic human and vehicle images onto billboards and building surfaces in simulated scenarios to simulate misleading 2D images. Specially designed adversarial patterns, such as misleading images, can be applied to simulated objects (e.g., stop signs, pedestrians) to simulate adversarial textures; scenarios where traffic signs contradict real-world road conditions can be constructed to simulate semantically ambiguous situations.

[0028] The intensity of each of the aforementioned failure factors can be defined based on the degree of interference, complexity, and trap induction. For example, for "optical glare," the greater the glare effect, the stronger the intensity of "optical glare," which can be represented by a value from 0 to 10. For "visual dirt," the larger the area of ​​dirt covering the camera lens surface, the stronger the intensity of "visual dirt." For "two-dimensional image misleading," the larger the area of ​​misleading images added to the vehicle surface, the stronger the intensity of "two-dimensional image misleading."

[0029] S120. Generalize and combine failure factors of different intensities in the extreme failure scenario factor library to obtain different extreme failure scenarios.

[0030] Because the end-to-end intelligent driving big model is a black box algorithm, it can only control all the input data of the big model, and comprehensively judge whether it can drive normally under such extreme failure scenarios through the final intelligent driving decision output.

[0031] Therefore, this embodiment integrates various failure factors based on Table 1 to construct a comprehensive extreme failure scenario. An extreme failure scenario includes all failure factors and required scene elements (such as pedestrians, oncoming vehicles, road markings, and traffic lights), and each failure factor carries an intensity attribute. For example, in an extreme failure scenario, the intensity of visual dirt is 0 (representing no visual dirt), the intensity of optical glare is 3 (representing a small amount of optical glare), and the intensity of radar interference is 5 (representing moderate radar interference). The intensities of other failure factors are not detailed here.

[0032] S130. Test the end-to-end intelligent driving model using different extreme failure scenarios; during the test, control the intensity of the failure factor from weak to strong, and record the intensity of the failure factor when the end-to-end intelligent driving model fails for the first time as the failure threshold; during the test, control the intensity of the failure factor from strong to weak, and record the intensity of the failure factor when the end-to-end intelligent driving model recovers for the first time as the recovery threshold.

[0033] Different extreme failure scenarios are simulated in the simulation environment. For example, CARLA is used to simulate urban environments, traffic flow, and weather; aiSim provides high-fidelity sensor simulation. The scenario factors in each extreme failure scenario are provided to the intelligent driving large model for simulation testing, realizing the testing of the intelligent driving large model in the simulation scenario.

[0034] This embodiment categorizes the output of the end-to-end intelligent driving model into normal output and abnormal output. Normal output refers to the model directly outputting continuous vehicle control signals (steering, acceleration, braking) to achieve autonomous driving in accordance with traffic rules when the perception-decision-planning chain is healthy, environmental complexity is within limits, and system redundancy is sufficient. Abnormal output refers to the output with a clear safety retreat strategy when the model detects its own capability limits, system failure, or environmental risks exceeding the tolerance threshold. This includes degraded execution, requesting takeover, and safe stopping. Degraded execution includes actively restricting some high-level functions and switching to a lower-level assisted driving mode, but the vehicle can still maintain basic longitudinal (speed / distance) or lateral (lane keeping) control without immediate driver intervention. Requesting takeover includes the large model determining that it can no longer safely handle the current or upcoming scenario and issuing a clear warning to the driver that immediate manual intervention is required. After issuing the request, the vehicle will maintain control briefly but will gradually reduce the level of intervention, waiting for the driver's response. When the request for takeover expires or the system determines that immediate manual takeover is no longer feasible (e.g., the driver faints) and safe driving cannot continue, the intelligent driving system will autonomously execute the Minimum Risk Maneuver (MRM) to bring the vehicle to a safe stop, i.e., a safe stop.

[0035] The testing process for the large-scale model from normal intelligent driving to failure includes: During testing, the intensity of failure factors is controlled from weak to strong according to a set intensity step size (e.g., 1), and extreme failure scenarios of different intensities are constructed. To simplify processing, a controlled variable method is adopted, focusing only on the impact of the intensity change of one failure factor on the test results in a set of tests, while the intensities of other failure factors remain constant (can be any value from 0 to 10). Therefore, extreme failure scenarios of single failure factors with different intensities are constructed in a set of tests. Following an order from weak to strong (e.g., intensity 0, 1, 2, 3, ..., 10), the end-to-end intelligent driving large-scale model is placed in extreme failure scenarios of different intensities for testing, and whether the end-to-end intelligent driving large-scale model operates normally is recorded. The intensity of the failure factor when the end-to-end intelligent driving large-scale model first transitions from normal intelligent driving to abnormal intelligent driving is recorded as the failure threshold. For example, when the intensity of visual contamination is 5, the end-to-end intelligent driving model is normal intelligent driving; when the intensity of visual contamination is 6, the end-to-end intelligent driving model is abnormal intelligent driving, so the failure threshold for visual contamination is 6.

[0036] The testing process for the large-scale intelligent driving model from intelligent driving failure to recovery includes: During the testing process, the intensity of the failure factor is controlled from strong to weak according to a set intensity step size (e.g., 1), and extreme failure scenarios of different intensities are constructed. To simplify the processing, the approach of controlling variables is adopted. In a set of tests, only the influence of the intensity change of one failure factor on the test results is considered, while the intensities of other failure factors remain constant (which can be any value from 0 to 10). Therefore, extreme failure scenarios of single failure factors of different intensities are constructed in a set of tests. In order from strong to weak (e.g., intensity 10, 9, 8, 7, ..., 0), the end-to-end intelligent driving large-scale model is placed in extreme failure scenarios of different intensities for testing, and the normal intelligent driving status of the end-to-end intelligent driving large-scale model is recorded; the intensity of the failure factor when the end-to-end intelligent driving large-scale model first switches from abnormal intelligent driving to normal intelligent driving is recorded as the recovery threshold. For example, when the intensity of visual contamination gradually decreases from 10 to 5, the end-to-end intelligent driving model is still in abnormal intelligent driving mode; when the intensity of visual contamination continues to decrease to 4, the end-to-end intelligent driving model returns to normal intelligent driving mode, so the recovery threshold for visual contamination is 4.

[0037] It should be noted that after completing one set of tests, the intensity values ​​of other failure factors are changed, and the tests are conducted again. Once the intensity values ​​of all other failure factors have been tested, another failure factor with variable intensity is introduced, and the testing continues using the controlled variable method, until all combinations of failure factor intensities have been tested.

[0038] It is evident that the recovery threshold and failure threshold of the current end-to-end intelligent driving model do not coincide, with an intermediate "hysteresis range." This hysteresis range helps prevent misjudgments and reduces the frequency of switching between normal and failure states. However, a wider "hysteresis range" can also lead to premature withdrawal and delayed recovery of intelligent driving functions, degrading the user experience.

[0039] S140. Calculate the hysteresis coefficient based on the failure threshold and recovery threshold.

[0040] S150. Determine the fallback strategy after the end-to-end intelligent driving big model fails, based on the magnitude of the hysteresis coefficient.

[0041] This application uses a hysteresis coefficient to describe the size of the "hysteresis interval", as shown in the following formula: H = (F_fail - F_recover) / F_fail; Where H is the hysteresis coefficient, F_fail is the failure threshold, and F_recover is the recovery threshold.

[0042] Note that, according to the description in S130, a set of failure thresholds and recovery thresholds can be obtained for different extreme failure scenarios, corresponding to single failure factors of different intensities. Therefore, it is necessary to calculate the hysteresis coefficient for the same set of failure thresholds and recovery thresholds to obtain the hysteresis coefficient for that extreme failure scenario.

[0043] If the hysteresis coefficient (in an extreme failure scenario) is greater than the set value, it indicates that the hysteresis range is wide and the intelligent driving recovery is slow. The more conservative fallback strategy after the end-to-end intelligent driving model (in this extreme failure scenario) fails includes: requesting takeover and safe stopping. If the hysteresis coefficient (in an extreme failure scenario) is less than or equal to the set value, it indicates that the hysteresis range is narrow and the intelligent driving recovery is fast. The more aggressive fallback strategy after the end-to-end intelligent driving model (in this extreme failure scenario) fails includes: degraded execution (such as speed limit, increasing safety distance).

[0044] Compared with the prior art, this application has the following beneficial effects: This application introduces a bidirectional boundary scanning mechanism to obtain both the failure threshold and recovery threshold of the large model, thereby achieving a joint evaluation of the "failure conditions" and "recovery conditions" of the large model. This solves the problem that existing testing methods only perform unidirectional boundary searches and neglect the evaluation of system recovery capabilities.

[0045] This application expands the evaluation scope from the sensor level to system-level comprehensive failure scenarios by constructing a library of extreme failure scenario factors covering multiple factors such as sensor interference, computing power limits, and perception traps, thus solving the problem that existing security boundary testing is limited to the sensor perception dimension.

[0046] Based on the hysteresis coefficient quantification results, this application matches differentiated fallback strategies for different systems, avoiding the imbalance between efficiency and security caused by a uniform "takeover when the boundary is exceeded" mode, and solving the problem of the "one-size-fits-all" fallback strategy in the existing system.

[0047] In one alternative implementation, if the hysteresis coefficient is large, the model parameters can be updated by training an end-to-end intelligent driving large model to improve the model's ability to maintain intelligent driving in the face of high-intensity failure factors. Specifically, training samples are constructed based on failure thresholds and recovery thresholds, and the end-to-end intelligent driving large model is trained using these training samples.

[0048] Specifically, the intensity of failure factors is reduced based on the failure threshold to construct the first positive approximation sample. For example, if the failure threshold for irregularly shaped obstacles is 6, and the intensity of other failure factors is set to x (x belongs to 0~10), then, setting the intensity of other failure factors to x and the intensity of irregularly shaped obstacles to 4 or 5, a limit failure scenario is constructed, and the label (i.e., the ideal output of the end-to-end intelligent driving model) is: end-to-end intelligent driving model output failure warning. This embodiment can train the large model to provide early warning capabilities. The end-to-end intelligent driving model is trained based on the first positive approximation sample so that the end-to-end intelligent driving model can provide early failure warnings. For example, a loss function is constructed based on the difference between the quantized output value and the quantized label value of the end-to-end intelligent driving model. The loss function is minimized by iterating the parameters of the end-to-end intelligent driving model. The quantization process of the large model output and label includes: using one-hot encoding to map the large model output and label to values, for example, quantizing the failure warning to 10, and quantizing all non-failure warnings in the large model output to 1.

[0049] Specifically, the intensity of failure factors is increased based on the failure threshold to construct a second positive approximation sample. For example, if the failure threshold for irregularly shaped obstacles is 6, and the intensity of other failure factors is set to x (x belongs to 0~10), then, setting the intensity of other failure factors to x and the intensity of irregularly shaped obstacles to 7 or 8, a limit failure scenario is constructed. The label (i.e., the ideal output of the end-to-end intelligent driving model) is: continuous vehicle control signals (steering, acceleration, braking). This embodiment can increase the failure threshold, allowing the intelligent driving function to exit later. The end-to-end intelligent driving model is trained based on the second positive approximation sample to ensure that the end-to-end intelligent driving model maintains normal intelligent driving. For example, a loss function is constructed based on the difference between the output quantized value and the label quantized value of the end-to-end intelligent driving model. The loss function is minimized by iterating the parameters of the end-to-end intelligent driving model. The difference between the output quantized value and the label quantized value is the Euclidean distance between multi-dimensional values, requiring the model output and label to be quantized into data of the same dimension for distance calculation. If the large model outputs vehicle control signals, the vehicle control signals and labels are normalized to obtain quantized values; if the large model outputs non-intelligent driving information (such as parking), it is quantized into a 0 vector.

[0050] Specifically, the strength of failure factors is increased based on the recovery threshold to construct reverse recovery samples. For example, if the recovery threshold for irregular obstacles is 3, and the strength of other failure factors is set to x (x belongs to 0~10), then, setting the strength of other failure factors to x and the strength of irregular obstacles to 4 or 5, a limit failure scenario is constructed. The label (i.e., the ideal output of the end-to-end intelligent driving model) is: continuous vehicle control signals (steering, acceleration, braking). This embodiment can increase the recovery threshold, enabling intelligent driving functions to intervene more quickly and reducing the hysteresis coefficient. The end-to-end intelligent driving model is trained based on the reverse recovery samples to enable the end-to-end intelligent driving model to recover normal intelligent driving. For example, a loss function is constructed based on the difference between the output quantized value and the label quantized value of the end-to-end intelligent driving model. The loss function is minimized by iterating the parameters of the end-to-end intelligent driving model. The difference between the output quantized value and the label quantized value is the Euclidean distance between multi-dimensional values, requiring the model output and label to be quantized into data of the same dimension for distance calculation. If the large model outputs vehicle control signals, the vehicle control signals and labels are normalized to obtain quantized values; if the large model outputs non-intelligent driving information (such as parking), it is quantized into a 0 vector.

[0051] After training, the end-to-end intelligent driving large model is retested using different extreme failure scenarios; and new failure thresholds and recovery thresholds are obtained, as detailed in S130. Then, based on the new failure thresholds and recovery thresholds, a new hysteresis coefficient is calculated; based on the magnitude of the new hysteresis coefficient, a fallback strategy is determined after the end-to-end intelligent driving large model fails, as detailed in S140 and S150.

[0052] This application also provides an electronic device, see [link to relevant documentation] Figure 2 It includes at least one processor 301 and a memory 302 communicatively connected to at least one of the processors 301.

[0053] The memory 302 stores instructions that can be executed by at least one of the processors 301, such that at least one of the processors 301 can perform the above-described end-to-end large model bidirectional failure boundary scanning and hysteresis effect assessment method, thus having at least the same advantages as the above-described method.

[0054] Optionally, the electronic device also includes interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The components are interconnected using different buses and can be mounted on a common motherboard or otherwise installed as needed. The processor can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI (Graphical User Interface) on an external input / output device (such as a display device coupled to the interface). In other embodiments, multiple processors can be used with multiple memories, and / or multiple buses can be used with multiple memories, if desired. Similarly, multiple electronic devices (e.g., as a server array, a group of blade servers, or a multiprocessor system) can be connected, each providing some of the necessary operations.

[0055] The memory 302, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the end-to-end large model bidirectional failure boundary scanning and hysteresis effect evaluation method in this embodiment. The processor 301 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 302, thereby realizing the aforementioned end-to-end large model bidirectional failure boundary scanning and hysteresis effect evaluation method.

[0056] The memory 302 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on terminal usage. Furthermore, the memory 302 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory, or other non-volatile solid-state storage device. In some instances, the memory 302 may further include memory remotely configured relative to the processor, which can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0057] The electronic device may also include an input device 303 and an output device 304. The processor 301, memory 302, input device 303, and output device 304 may be connected via a bus or other means.

[0058] Input device 303 can receive input digital or character information, and output device 304 may include a display device, an auxiliary lighting device (e.g., an LED), and a haptic feedback device (e.g., a vibration motor). The display device may include, but is not limited to, a liquid crystal display (LCD), a light-emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touchscreen.

[0059] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0060] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for bidirectional failure boundary scanning and hysteresis effect assessment in an end-to-end large model, characterized in that, include: Construct a library of extreme failure scenario factors, including: sensor interference factor, computing power limit factor, and perception trap factor; Different extreme failure scenarios are obtained by generalizing and combining failure factors of different intensities in the extreme failure scenario factor library. The end-to-end intelligent driving model was tested using different extreme failure scenarios. During the test, the intensity of the failure factor was controlled from weak to strong, and the intensity of the failure factor when the end-to-end intelligent driving model first failed was recorded as the failure threshold. During the test, the intensity of the failure factor was controlled from strong to weak, and the intensity of the failure factor when the end-to-end intelligent driving model first recovered was recorded as the recovery threshold. Calculate the hysteresis coefficient based on the failure threshold and recovery threshold; Based on the magnitude of the hysteresis coefficient, a fallback strategy is determined after the failure of the end-to-end intelligent driving big model.

2. The method according to claim 1, characterized in that, Sensor interference factors include: visual contamination, optical glare, and radar interference; computing power limit factors include: irregularly shaped obstacles, ultra-high complexity, and different types of road shoulders; perception trap factors include: misleading two-dimensional images, adversarial textures, and semantically ambiguous scenes.

3. The method according to claim 2, characterized in that, During the testing process, the intensity of the failure factor was controlled from weak to strong, and the intensity of the failure factor at the time of the first failure of the end-to-end intelligent driving model was recorded as the failure threshold, including: During the test, the intensity of the failure factor was controlled from weak to strong according to the set intensity step size, and extreme failure scenarios of different intensities were constructed respectively. The end-to-end intelligent driving model was placed in extreme failure scenarios of different intensities in order from weakest to strongest, and the results were recorded to determine whether the end-to-end intelligent driving model could drive normally. The intensity of the failure factor when the end-to-end intelligent driving model first transitions from normal intelligent driving to abnormal intelligent driving is recorded as the failure threshold.

4. The method according to claim 3, characterized in that, During the test, the intensity of the failure factor was controlled from strong to weak, and the intensity of the failure factor at the first recovery of the end-to-end intelligent driving model was recorded as the recovery threshold, including: During the test, the intensity of the failure factor was controlled from strong to weak according to the set intensity step size, and extreme failure scenarios of different intensities were constructed respectively. The end-to-end intelligent driving model was placed in extreme failure scenarios of different intensities in order from strong to weak, and the results were recorded to determine whether the end-to-end intelligent driving model could drive normally. The intensity of the failure factor when the end-to-end intelligent driving model first transitions from abnormal intelligent driving to normal intelligent driving is recorded as the recovery threshold.

5. The method according to claim 4, characterized in that, The output of the end-to-end intelligent driving model during abnormal intelligent driving is: degraded execution, request for takeover, and safe stopping.

6. The method according to claim 5, characterized in that, After recording the intensity of the failure factor during the first recovery of the end-to-end intelligent driving big model as a recovery threshold, the method further includes: Training samples are constructed based on the failure threshold and recovery threshold, and the end-to-end intelligent driving big model is trained using the training samples. After training, the end-to-end intelligent driving model was retested using different extreme failure scenarios, and new failure thresholds and recovery thresholds were obtained.

7. The method according to claim 6, characterized in that, Training samples are constructed based on the failure threshold and recovery threshold, and the end-to-end intelligent driving big model is trained using the training samples, including: Based on the failure threshold, the intensity of the failure factor is reduced to construct the first positive approximation sample; The end-to-end intelligent driving big model is trained based on the first positive approximation sample so that the end-to-end intelligent driving big model can provide early failure warnings. Based on the failure threshold, the strength of the failure factor is increased to construct a second positive approximation sample; The end-to-end intelligent driving big model is trained based on the second positive approximation sample so that the end-to-end intelligent driving big model can maintain normal intelligent driving.

8. The method according to claim 7, characterized in that, Training samples are constructed based on the failure threshold and recovery threshold, and the end-to-end intelligent driving big model is trained using the training samples, including: Based on the recovery threshold, the intensity of the failure factor is increased to construct a reverse recovery sample; The end-to-end intelligent driving big model is trained based on the reverse recovery samples so that the end-to-end intelligent driving big model can restore normal intelligent driving.

9. The method according to claim 8, characterized in that, Based on the magnitude of the hysteresis coefficient, a fallback strategy is determined after the failure of the end-to-end intelligent driving big model, including: If the hysteresis coefficient is greater than the set value, the fallback strategy after the failure of the end-to-end intelligent driving big model includes: requesting takeover and safe parking; If the hysteresis coefficient is less than or equal to the set value, the fallback strategy after the failure of the end-to-end intelligent driving big model includes: degraded execution.

10. An electronic device, characterized in that, include: At least one processor, and a memory communicatively connected to at least one of the processors; The memory stores instructions executable by at least one of the processors, which are executed to enable the at least one processor to perform the end-to-end large model bidirectional failure boundary scanning and hysteresis effect evaluation method as described in any one of claims 1-9.

Images