Intelligent driving HMI visual automatic test system and method based on signal simulation

Abstract

The invention discloses an intelligent driving HMI visual automatic testing system and method based on signal simulation, and relates to the technical field of intelligent driving testing, and the system comprises a signal generalization scene library which is used for storing a virtual vehicle signal sequence generated based on a parameterized scene, and the virtual vehicle signal sequence simulates the driving or parking working condition of an intelligent driving system; the signal injection engine is in communication connection with the signal generalization scene library and is used for acquiring a virtual vehicle signal sequence from the signal generalization scene library and injecting the virtual vehicle signal sequence into a vehicle machine HMI sandbox in a non-intrusive mode; the vehicle machine HMI sandbox is in communication connection with the signal injection engine and is used for operating the tested vehicle machine HMI system, receiving and responding to the virtual vehicle signal sequence and outputting a corresponding visual interface and an audio stream; and the automated analysis engine is used for obtaining an expected HMI response standard corresponding to the current virtual vehicle signal sequence from the signal generalization scene library.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving test, and particularly relates to a signal simulation-based intelligent driving HMI visualized automated test system and method. BACKGROUND

[0002] The rapid development of intelligent driving technology puts forward higher requirements for the test and verification of vehicle-mounted human-machine interface. At present, two test schemes, real vehicle road test and hardware-in-the-loop simulation, are mainly used in this field. The real vehicle test is carried out in a real road environment, and the hardware-in-the-loop simulation is completed by building a laboratory environment containing real controllers and sensors.

[0003] In the prior art, the long-tail scene reproduction capability is seriously insufficient, and the real vehicle test is difficult to safely and effectively reproduce low-probability high-risk scenes such as ghost probes and abnormal traffic signal lights, resulting in a significant gap in the coverage range of test cases, and the reliability and safety of the HMI system under extreme working conditions cannot be fully verified. SUMMARY

[0004] The purpose of the present application is to provide a signal simulation-based intelligent driving HMI visualized automated test system, which at least solves one of the technical problems that the real vehicle test is difficult to safely and effectively reproduce low-probability high-risk scenes, resulting in a significant gap in the coverage range of test cases, and the reliability and safety of the HMI system under extreme working conditions cannot be fully verified.

[0005] The present application provides the following solutions:

[0006] According to one aspect of the present application, a signal simulation-based intelligent driving HMI visualized automated test system is provided, comprising:

[0007] A signal generalization scenario library for storing virtual vehicle signal sequences generated based on parameterized scenarios, the virtual vehicle signal sequences simulating driving or parking working conditions of an intelligent driving system;

[0008] A signal injection engine in communication connection with the signal generalization scenario library, for obtaining virtual vehicle signal sequences from the signal generalization scenario library and injecting them into a vehicle machine HMI sandbox in a non-intrusive manner;

[0009] A vehicle machine HMI sandbox in communication connection with the signal injection engine, for running a tested vehicle machine HMI system, receiving and responding to virtual vehicle signal sequences, and outputting corresponding visualized interfaces and audio streams;

[0010] An automated analysis engine, in communication with the signal generalization scenario library and the car-machine HMI sandbox, is configured to obtain expected HMI response criteria corresponding to a current virtual vehicle signal sequence from the signal generalization scenario library; capture visual interface and audio stream from the car-machine HMI sandbox; and automatically analyze the visual interface using computer vision techniques and the audio stream using speech recognition techniques based on the expected HMI response criteria to quantitatively evaluate rendering accuracy and prompt content compliance of the measured car-machine HMI system.

[0011] Further, the virtual vehicle signal sequence is generated by:

[0012] receiving scenario parameter inputs, including obstacle position, vehicle speed, and fault type;

[0013] based on the scenario parameters, generating corresponding CAN ID and data frame through parameterized scripts to form a virtual signal sequence simulating driving or parking conditions.

[0014] Further, the non-invasive injection into the car-machine HMI sandbox includes:

[0015] simulating ECU nodes through software Socket interface to non-invasively send virtual vehicle signal sequences to the car-machine HMI sandbox;

[0016] During the injection process, normal signals are sent first, and then signal faults including signal packet loss, signal delay, or data frame error are injected.

[0017] Further, the car-machine HMI sandbox is configured to perform the following operations:

[0018] obtain screen frame buffer through ADB instruction or GPU DirectX interface to capture visual interface;

[0019] When the measured car-machine HMI system is running, record the resource occupation rate of CPU and GPU in real time, and trigger an alarm when the resource occupation rate exceeds a preset threshold.

[0020] Further, the expected HMI response criteria include:

[0021] visual interface features include expected position and color of AR elements and expected text content;

[0022] audio prompt content includes expected keywords that need to be verified.

[0023] Further, the automated analysis using computer vision techniques includes:

[0024] Using OpenCV library, AR elements are located in the captured visual interface through template matching and feature point detection algorithm, and the position deviation is calculated;

[0025] Text areas in the interface are recognized by OCR technology, and the recognition result is compared with the expected text content.

[0026] Further, the automatic analysis using speech recognition technology includes:

[0027] Transcribe the audio stream through automatic speech recognition technology;

[0028] In the transcribed text, it is checked whether the expected key words and their occurrence times exist.

[0029] Further, the quantitative evaluation includes performing timing compliance detection:

[0030] Record the timestamp of the virtual vehicle signal sequence injection;

[0031] Detect the timestamp of the expected feature appearing in the visual interface or the expected key word starting to play in the audio stream;

[0032] Calculate the time difference between the two as the response delay, and compare it with the predefined delay threshold.

[0033] Further, the system is configured to perform conflict scenario testing, the process of which is:

[0034] Obtain logically conflicting virtual vehicle signal sequences from the signal generalization scenario library;

[0035] Inject the conflicting signal sequences into the car machine HMI sandbox at the same time or almost at the same time through the signal injection engine;

[0036] Verify whether the measured car machine HMI system is rendered and prompted according to the preset priority strategy or degradation strategy through the automatic analysis engine.

[0037] According to the two aspects of the present application, a kind of intelligent driving HMI visual automatic test method based on signal simulation is provided, comprising:

[0038] Obtain virtual vehicle signal sequences generated based on parameterized scenarios from the signal generalization scenario library, the virtual vehicle signal sequences simulate the driving or parking working condition of intelligent driving system;

[0039] Through the signal injection engine, the virtual vehicle signal sequence is injected into the car machine HMI sandbox running the measured car machine HMI system in a non-invasive manner;

[0040] The virtual vehicle signal sequence is received and responded by the car machine HMI sandbox, and the corresponding visual interface and audio stream are output.

[0041] acquire, by an automated analysis engine, expected HMI response criteria corresponding to the current virtual vehicle signal sequence from the signal generalization scenario library; capture visual interfaces and audio streams from the car machine HMI sandbox; based on the expected HMI response criteria, and using computer vision techniques for the visual interfaces and speech recognition techniques for the audio streams, perform automated analysis to quantitatively evaluate the rendering accuracy and prompt content compliance of the measured car machine HMI system.

[0042] Through the above scheme, the following beneficial technical effects are obtained:

[0043] The application can automatically generate diversified test cases including extreme working conditions and long-tail scenarios through the parameterized signal generalization scenario library; the generation efficiency of test cases is improved, so that low-probability high-risk scenarios that are difficult to reproduce in traditional real vehicle testing can be safely and efficiently reproduced in a laboratory environment; the fully automated testing pipeline achieves an order of magnitude improvement in testing efficiency, significantly shortening the testing cycle compared to traditional real vehicle testing methods.

[0044] The application through a pure software simulation scheme, gets rid of the dependence on special hardware sensors and expensive simulation equipment; the test system only needs general computing equipment to run, significantly reducing the hardware investment cost; at the same time, various resource consumptions required by real vehicle testing are avoided, effectively controlling the comprehensive cost in the research and development verification stage.

[0045] The application achieves positioning accuracy that manual visual inspection cannot achieve by using computer vision technology to realize fine AR element positioning detection; by using optical character recognition and speech recognition technology, automatic verification of text content and voice prompts is realized, objective and quantitative HMI rendering accuracy evaluation criteria are established, and subjective differences of manual interpretation are eliminated.

[0046] The application can systematically verify the response logic of the HMI system under abnormal signal conditions through the conflict signal injection mechanism; this method can effectively identify potential interaction logic defects and priority strategy problems, and improve the functional safety level of the system; timing compliance detection ensures that the response delay of critical safety warnings meets the real-time requirements of functional safety standards.

[0047] The application realizes independent testing capability of the car machine software team by constructing a test environment decoupled from the intelligent driving system, breaking through the development limitations in the traditional testing process; based on the scenario generation method and standardized interface design of the parameterized script, good test case extension capability is provided, which can adapt to the testing needs of different vehicle platforms and software versions. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1An architecture diagram of a signal simulation-based intelligent driving HMI visualized automated test system provided by one or more embodiments of the present application.

[0049] Figure 2 A signal generalization scenario library schematic diagram of a signal simulation-based intelligent driving HMI visualized automated test system provided by one or more embodiments of the present application.

[0050] Figure 3 A signal injection engine schematic diagram of a signal simulation-based intelligent driving HMI visualized automated test system provided by one or more embodiments of the present application.

[0051] Figure 4 A car machine HMI sandbox schematic diagram of a signal simulation-based intelligent driving HMI visualized automated test system provided by one or more embodiments of the present application.

[0052] Figure 5 An automated analysis engine schematic diagram of a signal simulation-based intelligent driving HMI visualized automated test system provided by one or more embodiments of the present application.

[0053] Figure 6 A flowchart of a signal simulation-based intelligent driving HMI visualized automated test method provided by one or more embodiments of the present application. DETAILED DESCRIPTION

[0054] The technical solutions of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0055] Figure 1 An architecture diagram of a signal simulation-based intelligent driving HMI visualized automated test system provided by one or more embodiments of the present application.

[0056] Figure 2 A signal generalization scenario library schematic diagram of a signal simulation-based intelligent driving HMI visualized automated test system provided by one or more embodiments of the present application.

[0057] Figure 3 A signal injection engine schematic diagram of a signal simulation-based intelligent driving HMI visualized automated test system provided by one or more embodiments of the present application.

[0058] Figure 4 A car machine HMI sandbox schematic diagram of a signal simulation-based intelligent driving HMI visualized automated test system provided by one or more embodiments of the present application.

[0059] Figure 5 This is a schematic diagram of the automated analysis engine of an intelligent driving HMI visualization automated testing system based on signal simulation, provided by one or more embodiments of the present invention.

[0060] like Figures 1-5 The intelligent driving HMI visualization automated testing system shown includes:

[0061] The signal generalization scenario library is used to store virtual vehicle signal sequences generated based on parameterized scenarios. The virtual vehicle signal sequences simulate the driving or parking conditions of the intelligent driving system.

[0062] Furthermore, generating the virtual vehicle signal sequence includes:

[0063] Receive scene parameter input, which includes obstacle position, vehicle speed and fault type;

[0064] Based on scenario parameters, corresponding CAN IDs and data frames are generated through parameterized scripts to form a virtual signal sequence simulating driving or parking conditions.

[0065] Specifically, the signal generalization scenario library generates virtual vehicle signal sequences through a parameterized scenario construction method. The construction of this scenario library first requires receiving externally input scenario parameters, which include, but are not limited to, the three-dimensional position coordinates of obstacles in the vehicle coordinate system, the real-time vehicle speed value, and the sensor fault type identification code. Based on these parameterized inputs, the system executes the signal sequence generation logic through a pre-built parameterized script engine, where the parameterized script can be implemented using either Python scripts or JSON configuration files.

[0066] During operation, the parameterized script dynamically calculates the corresponding sensing signal features based on the input obstacle position parameters, generates the vehicle kinematic model output based on the vehicle speed parameters, and configures the corresponding abnormal signal mode based on the fault type parameters. The script engine parses these parameter relationships and generates standardized CAN communication identifiers and corresponding data frame content according to the preset communication protocol specifications. The CAN ID is used to identify the functional module from which the signal originates, while the data frame carries the specific signal value information.

[0067] In the signal sequence construction logic, the parameterized script simulates real driving or parking conditions by defining the timing relationship of signals. For example, when generating a lane change warning scenario, the script organizes the CAN message sequence of steering wheel angle signal, blind spot warning signal and vehicle speed signal in time order. When generating a parallel parking scenario, the script constructs a time-sequential message stream containing gear position signal and obstacle distance signal.

[0068] One possible parameter mapping relationship is achieved through the following function:

[0069]

[0070] in Represents the obstacle position parameter vector. This indicates the vehicle's speed parameter. The function represents the fault type parameter. This represents the signal generation rules defined in the parameterization script;

[0071] During application, the system executes the signal sequence generation task by calling the interpreter of the parameterized script. After reading the input scene parameters, the interpreter calculates the corresponding CAN ID list and data frame array according to the logical rules defined in the script, and finally assembles them into a virtual vehicle signal sequence that meets the simulation test requirements. This signal sequence is indexed by timestamps and organized into a continuous scene data stream according to the preset sampling frequency, providing a standardized data source for subsequent signal injection.

[0072] Through this parameterized generation method, the system can flexibly construct test scenarios covering both normal and extreme working conditions based on a unified scenario description framework. At the same time, due to the use of script-based implementation, the expansion and modification of test scenarios do not require recompiling the core modules of the system, thus improving the adaptability and maintainability of the test system.

[0073] The signal injection engine communicates with the signal generalization scenario library to obtain virtual vehicle signal sequences from the signal generalization scenario library and inject them into the vehicle HMI sandbox in a non-intrusive manner.

[0074] Furthermore, non-invasive injection into the vehicle's HMI sandbox includes:

[0075] By simulating ECU nodes through a software Socket interface, virtual vehicle signal sequences are sent to the vehicle's HMI sandbox in a non-intrusive manner.

[0076] During the injection process, a normal signal is sent first, followed by the injection of signal faults, including signal packet loss, signal delay, or data frame errors.

[0077] Specifically, the signal injection engine obtains pre-generated virtual vehicle signal sequences from the signal generalization scenario library through the established communication link; the engine adopts a software-defined communication architecture and simulates electronic control unit nodes in the actual vehicle network by creating a Socket network interface based on the transmission control protocol.

[0078] In terms of operation mechanism, when the signal injection engine is initialized, it first establishes a Socket connection with the vehicle HMI sandbox. This connection adopts a client-server architecture, in which the signal injection engine acts as the client and the simulation environment running in the vehicle HMI sandbox acts as the server. After the connection is established, the engine encapsulates the virtual vehicle signal sequence into standard network data packets for transmission according to the preset timing requirements.

[0079] During the specific injection process, the engine follows the principle of injecting normal faults first and then abnormal ones. First, it sends a complete sequence of normal signals according to the scenario definition to ensure that the vehicle HMI system enters a stable working state. Then, according to the test case requirements, it selectively injects three types of signal faults: signal packet loss is achieved by randomly dropping a specific proportion of data packets, signal delay is achieved by introducing a configurable delay time at the transport layer, and data frame errors are achieved by modifying the check bit of the data packet or filling in erroneous data.

[0080] A possible fault injection model can be described by the following mathematical expression:

[0081]

[0082] in Indicates at time Injected signal data packets, This represents a normal signal sequence. Represents a fault signal sequence. Indicates the start time point of fault injection;

[0083] At the technical implementation level, signal packet loss faults are controlled by a pseudo-random number generator to prevent packet loss events from occurring. The decision-making strategy can be expressed as follows:

[0084]

[0085] in This represents the preset packet loss probability threshold. Generate uniformly distributed random numbers in the range [0,1].

[0086] Signal delay faults are addressed by introducing a delay queue mechanism, with the specific delay time... satisfy:

[0087]

[0088] in and These represent the minimum and maximum latency time boundaries, respectively.

[0089] Data frame errors are implemented using bit flipping or field replacement; error injection location. Determined by the error mode mask:

[0090]

[0091] in Represents the original data frame. Indicates the error mode mask. Indicates a bitwise XOR operation;

[0092] In application, this non-intrusive injection method allows the test system to complete signal simulation without physical access to the vehicle network bus; through the software-defined fault injection mechanism, the robustness and error handling capability of the vehicle HMI system under abnormal signal conditions can be systematically verified; the entire injection process maintains timing accuracy, ensuring the repeatability and comparability of test results.

[0093] The vehicle HMI sandbox communicates with the signal injection engine and is used to run the vehicle HMI system under test. It receives and responds to virtual vehicle signal sequences and outputs corresponding visual interfaces and audio streams.

[0094] Furthermore, the vehicle HMI sandbox is configured to perform the following operations:

[0095] Obtain the screen frame buffer using ADB commands or the GPU DirectX interface to capture the visual interface;

[0096] When the vehicle's HMI system under test is running, the resource utilization of CPU and GPU is recorded in real time, and an alarm is triggered when the resource utilization exceeds a preset threshold.

[0097] Specifically, the vehicle HMI sandbox receives a sequence of virtual vehicle signals from the signal injection engine through an established communication connection. The sandbox constructs an isolated software execution environment for running the vehicle HMI system under test and simulating the hardware interaction interface of the real vehicle electronic system. During the initialization of the sandbox environment, the software image of the system under test is loaded, and virtual sensor input channels and display output channels are configured to ensure that the system runs stably in a controlled environment.

[0098] In terms of operation mechanism, the vehicle HMI sandbox captures the visual interface output through two optional technical paths: one is to send a specific screen capture command through the Android debug bridge protocol to directly obtain the frame buffer data of the display device; the other is to access the output of the rendering pipeline through the DirectX application programming interface provided by the graphics processor and extract uncompressed bitmap data. Both methods can avoid the frame loss problem that may be caused by traditional screenshot methods and ensure the integrity and timing accuracy of the interface capture.

[0099] In terms of resource monitoring, the sandbox integrates a system performance counter, which collects real-time resource usage data of the central processing unit and graphics processing unit through the performance monitoring interface provided by the operating system kernel. The monitoring data is recorded at a fixed sampling frequency, and the sampling interval can be configured according to the test requirements. When the resource usage rate is detected to exceed the preset threshold, the sandbox triggers an alarm mechanism. This threshold can be dynamically adjusted through the configuration file to adapt to test scenarios with different performance requirements.

[0100] A possible resource utilization rate determination model can be described by the following mathematical expression:

[0101]

[0102] in This indicates the CPU utilization rate collected in real time. This indicates the graphics processor utilization rate during real-time data acquisition. This indicates the CPU utilization threshold. This indicates the graphics processor utilization threshold. Indicates the alarm trigger status;

[0103] At the technical implementation level, the screen frame capture process follows a strict timing synchronization mechanism. For ADB capture, the system transmits display data by establishing a stable USB or network connection. For DirectX capture, the system intercepts rendering output by creating a virtual display device interface. Both capture methods maintain timing consistency with signal injection to ensure that the subsequent analysis engine can accurately correlate input signals with output responses.

[0104] The resource monitoring module uses a polling method to periodically query the system performance status, and its data collection cycle is... It meets real-time requirements and is configurable; the collected performance data includes, but is not limited to, indicators such as processor core utilization, video memory usage, and rendering frame rate; these data are stored together with the corresponding test timestamps to form a complete performance baseline record;

[0105] During application, the vehicle-mounted HMI sandbox achieves non-intrusive monitoring of the system under test through this design; it ensures the complete execution of the original system functions and provides comprehensive output capture and performance monitoring capabilities; the entire sandbox operation process maintains collaborative work with upstream and downstream modules, providing a reliable execution environment foundation for automated testing processes.

[0106] An automated analysis engine communicates with a signal generalization scenario library and an in-vehicle HMI sandbox. It is used to obtain the expected HMI response standard corresponding to the current virtual vehicle signal sequence from the signal generalization scenario library; capture the visual interface and audio stream from the in-vehicle HMI sandbox; and perform automated analysis of the visual interface based on the expected HMI response standard, using computer vision technology and speech recognition technology to analyze the audio stream, in order to quantitatively evaluate the rendering accuracy and compliance of the prompts of the tested in-vehicle HMI system.

[0107] Furthermore, the expected HMI response criteria include:

[0108] Visual interface features include the expected position and color of AR elements, as well as the expected text content;

[0109] The audio prompts include expected keywords that need to be verified.

[0110] Furthermore, automated analysis using computer vision technology includes:

[0111] Using the OpenCV library, AR elements are located in the captured visualization interface through template matching and feature point detection algorithms, and their positional deviations are calculated.

[0112] The text region in the interface is identified using OCR technology, and the recognition result is compared with the expected text content.

[0113] Furthermore, automated analysis using speech recognition technology includes:

[0114] Audio streams are transcribed using automatic speech recognition technology;

[0115] In the transcribed text, the existence of the expected keywords and their frequency of occurrence are counted and verified.

[0116] Furthermore, the quantitative assessment includes performance timeline compliance checks:

[0117] Record the timestamp of the virtual vehicle signal sequence injection;

[0118] Detect the timestamp when expected features appear in the visualization interface or when expected keywords start playing in the audio stream;

[0119] Calculate the time difference between the two as the response latency and compare it with a predefined latency threshold.

[0120] Specifically, the automated analysis engine synchronously obtains the expected HMI response standard corresponding to the current test scenario from the signal generalization scenario library through a pre-established communication link, while continuously capturing the visual interface frame sequence and audio stream data from the vehicle HMI sandbox; the engine adopts a multimodal analysis architecture, integrating computer vision processing and speech recognition modules to realize automated verification and quantitative evaluation of the output response of the system under test.

[0121] During the expected standard acquisition phase, the engine parses the standardized response description files stored in the signal generalization scenario library and extracts key verification parameters. These parameters include the expected position coordinates, expected color value range, and expected text string of the augmented reality elements in the visualization interface, as well as the expected keyword list to be detected in the audio output. The standard data is stored in a structured format, such as using JSON format to define interface feature constraints and audio verification rules.

[0122] In the visualization interface analysis phase, the computer vision processing module implements an image processing pipeline based on the OpenCV library. For augmented reality element localization, a hybrid method combining template matching and feature point detection algorithms is adopted. Template matching initially locates elements by calculating the similarity between the target region and a preset template image, while feature point detection precisely locates the element contours by extracting ORB or SIFT feature descriptors. The positional deviation is calculated using the Euclidean distance formula to measure the offset between the actual and expected positions.

[0123]

[0124] in and This represents the x and y coordinates of the center of the actually detected element. and Indicates the x and y coordinates of the center of the expected element. This indicates the pixel value representing the positional deviation.

[0125] In the text content verification stage, the optical character recognition engine extracts text from a specified area of ​​the interface; the recognition process includes steps such as image preprocessing, text region detection, character segmentation, and recognition; the recognition result is compared with the expected text content, and the edit distance algorithm is used to quantify the text differences.

[0126]

[0127] in This indicates the actual text string being recognized. This indicates the expected text string. This function calculates the edit distance between two strings. Indicates the text error rate;

[0128] During the audio stream analysis phase, the automatic speech recognition module converts the continuous audio stream into text transcription results; the speech recognition engine can adopt an end-to-end recognition architecture based on deep learning to map audio feature sequences into text sequences; after transcription, the system searches for expected keywords in the output text and counts the frequency of each word; keyword matching adopts a strategy that combines exact matching and fuzzy matching, with fuzzy matching allowing a certain degree of pronunciation variation;

[0129] During the timing compliance check, the system records two key timestamps: the moment when the virtual vehicle signal sequence injection is completed. The moment when the expected feature first appears in the visualization interface or when the expected keyword begins playing in the audio stream. Response delay is calculated using the time difference:

[0130]

[0131] Calculated delay value With predefined delay threshold Compare and determine the timing compliance;

[0132] At the technical implementation level, each analysis module adopts a parallel processing architecture, with visual analysis and voice analysis running independently and synchronizing results through an event bus; the system integrates quantitative indicators such as positional deviation, text error, keyword matching rate, and response latency into a comprehensive evaluation result by defining a unified quality assessment index system; the entire analysis process follows strict temporal logic to ensure accurate correspondence between test cases and verification standards;

[0133] In application, this multimodal automated analysis method can systematically verify the compliance of the vehicle HMI system across multiple output dimensions; it provides objective test basis through quantitative evaluation indicators, avoiding errors introduced by subjective human judgment; and the configurability of the analysis engine allows for adjustment of verification parameters and threshold standards according to different test requirements, adapting to diverse test scenario requirements.

[0134] In this embodiment, the system is configured to perform conflict scenario testing, and the process is as follows:

[0135] Obtain logically conflicting virtual vehicle signal sequences from the signal generalization scenario library;

[0136] The signal injection engine injects conflicting signal sequences simultaneously or nearly simultaneously into the vehicle's HMI sandbox.

[0137] The automated analysis engine verifies whether the vehicle's HMI system under test renders and provides prompts according to the preset priority or degradation strategy.

[0138] Specifically, the system verifies the compliance of the behavior of the vehicle's HMI system under signal conflict conditions through a predefined conflict scenario test process. The test process first obtains logically conflicting virtual vehicle signal sequences from the signal generalization scenario library. These sequences are generated by parameterized scripts and contain contradictory or competing control commands, such as a combination of automatic emergency braking signal and adaptive cruise control activation signal, or the parallel triggering of sensor failure indication and lane keeping assist request.

[0139] During the signal sequence acquisition phase, the definition of conflict scenarios is based on the anomaly handling specifications in functional safety standards, and the logical conflict relationships between signals are set through script configuration tools; a possible conflict signal combination model can be described by the following mathematical expression:

[0140]

[0141] in Represents a set of conflict signal sequences. and Representing different signal elements, This function determines the conflict relationship between signals and returns true when the signals logically contradict each other.

[0142] During the signal injection phase, the signal injection engine employs a time synchronization mechanism to send conflicting signal sequences to the vehicle's HMI sandbox simultaneously or nearly simultaneously. The injection process is controlled by a high-precision timer to ensure that the injection time deviation of multiple conflicting signals is less than a configurable threshold, for example, by using time window constraints.

[0143]

[0144] in and Indicates the injection timestamps for different conflict signals. Indicates the maximum permissible time deviation;

[0145] During the verification and analysis phase, the automated analysis engine monitors the output response of the vehicle's HMI sandbox, including the rendered content of the visual interface and the audio prompt stream. The engine verifies whether the system behavior meets expectations by comparing the actual output with a preset priority strategy or degradation strategy rule base. The priority strategy may be defined as a ranking function of signal importance.

[0146]

[0147] in Indicates the signal type identifier. Indicates the current system context state, function Map signals to priority values;

[0148] For visual interface verification, the analysis engine detects the appearance order, display duration, and overlap processing method of rendered elements; for audio prompt verification, the engine identifies the content sequence and interruption behavior of the voice output; the verification process is based on a state machine model, tracks the state transition path of the system after the conflict signal is triggered, and checks whether it follows the predefined exception handling process.

[0149] In terms of technical rationality, this conflict scenario testing method systematically exposes potential design flaws by simulating signal competition conditions that are rare in the real world; the repeatability and automation of the testing process ensure the consistency and objectivity of the evaluation results; the entire implementation scheme is based on modular design, allowing for flexible expansion of new conflict modes without modifying the core testing architecture.

[0150] Figure 6 This is a flowchart of a visual automated testing method for intelligent driving HMI based on signal simulation, provided by one or more embodiments of the present invention.

[0151] like Figure 6 The intelligent driving HMI visualization automated testing method shown includes: obtaining virtual vehicle signal sequences generated based on parameterized scenarios from the signal generalization scenario library, and simulating the driving or parking conditions of the intelligent driving system.

[0152] The signal injection engine injects virtual vehicle signal sequences into the vehicle HMI sandbox running the tested vehicle HMI system in a non-intrusive manner.

[0153] The vehicle's HMI sandbox receives and responds to virtual vehicle signal sequences, and outputs corresponding visual interfaces and audio streams.

[0154] The automated analysis engine retrieves the expected HMI response standard corresponding to the current virtual vehicle signal sequence from the signal generalization scenario library; captures the visual interface and audio stream from the vehicle HMI sandbox; and based on the expected HMI response standard, it uses computer vision technology to automatically analyze the visual interface and speech recognition technology to automatically analyze the audio stream, so as to quantitatively evaluate the rendering accuracy and compliance of the prompt content of the vehicle HMI system under test.

[0155] Specifically, a comprehensive test of the intelligent driving HMI system is achieved by constructing a complete closed loop of signal simulation and automated verification. The method begins with the configuration phase of the test scenario, extracting pre-parameterized virtual vehicle signal sequences from the signal generalization scenario library. These sequences accurately simulate the typical driving and parking conditions of the intelligent driving system based on mathematical modeling.

[0156] During the signal transmission phase, the method establishes a communication link with the vehicle HMI sandbox through a signal injection engine, and uses a non-intrusive signal injection mechanism to transmit the virtual vehicle signal sequence to the sandbox environment running the system under test; the injection process maintains the accuracy of signal timing to ensure the authenticity of the test scenario;

[0157] During the system response phase, the system under test in the vehicle HMI sandbox receives and processes the input signal sequence, generating the corresponding visual interface output and audio prompt stream; during this process, the sandbox environment captures complete output data through the underlying interface, including screen frame sequence and audio sampling data;

[0158] During the analysis and verification phase, the automated analysis engine simultaneously performs multimodal data analysis. The engine first obtains the expected HMI response standards corresponding to the current test scenario from the signal generalization scenario library. These standards clearly define the spatial location characteristics, color specifications, text content, and audio keyword list of the visualization elements.

[0159] For the visualization interface analysis, an OpenCV-based computer vision processing workflow is adopted. An algorithm combining template matching and feature point detection is used to locate AR elements, and an OCR engine is used to recognize the interface text. The spatial deviation between the actual detection results and the expected standard is measured using Euclidean distance.

[0160]

[0161] in Indicates the coordinates of the detected element. Indicates the expected coordinates;

[0162] For audio stream analysis, audio signals are converted into text sequences using automatic speech recognition technology, and the frequency of expected keywords is counted in the transcription results; the matching degree between the recognition results and the expected content is evaluated using a term frequency statistical model.

[0163] In the timing compliance verification phase, the method assesses system response latency through precise timestamp recording and comparison; it defines the time from the completion of signal injection. Until the system generates a valid response The time interval is the response delay:

[0164]

[0165] The delay value is related to the preset threshold. Compare the data to determine whether the system's real-time performance meets the design requirements;

[0166] At the technical implementation level, this method ensures the repeatability of the testing process and the comparability of the results by establishing a standardized test execution process; data transfer between each processing stage is encapsulated in a unified format to ensure the collaborative efficiency between the various modules of the system; the entire method reduces manual intervention and improves testing efficiency through automated script scheduling and execution.

[0167] At the application level, this method is applicable to multiple stages of the development cycle of intelligent driving HMI systems, including unit testing, integration testing, and system verification testing. By parametrically adjusting test scenarios and verification standards, it can adapt to the testing requirements of different vehicle platforms and system versions. The modular design of the method allows test cases and verification logic to be maintained and extended independently, demonstrating good engineering applicability.

[0168] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A signal simulation based automated test system for intelligent driving HMI visualization, characterized in that, The intelligent driving HMI visualization automated testing system based on signal simulation comprises: a signal generalization scenario library, configured to store virtual vehicle signal sequences generated based on parameterized scenarios, the virtual vehicle signal sequences simulating driving or parking working conditions of an intelligent driving system; a signal injection engine, in communication connection with the signal generalization scenario library, configured to obtain virtual vehicle signal sequences from the signal generalization scenario library and inject the virtual vehicle signal sequences into a car machine HMI sandbox in a non-invasive manner; a car machine HMI sandbox, in communication connection with the signal injection engine, configured to run a measured car machine HMI system, receive and respond to the virtual vehicle signal sequences, and output corresponding visual interfaces and audio streams; an automated analysis engine, in communication connection with the signal generalization scenario library and the car machine HMI sandbox, configured to obtain, from the signal generalization scenario library, expected HMI response standards corresponding to a current virtual vehicle signal sequence, capture visual interfaces and audio streams from the car machine HMI sandbox, and perform automated analysis on the visual interfaces by using computer vision technology and on the audio streams by using speech recognition technology, so as to quantitatively evaluate rendering accuracy and prompt content compliance of the measured car machine HMI system.

2. The signal simulation based automated test system for intelligent driver HMI visualization of claim 1, wherein, The generation of the virtual vehicle signal sequences comprises: receiving scenario parameter inputs, the scenario parameters including obstacle positions, vehicle speeds, and fault types; generating corresponding CAN IDs and data frames based on the scenario parameters by using a parameterized script to form virtual signal sequences simulating driving or parking working conditions.

3. The signal simulation based automated test system for intelligent driver HMI visualization of claim 1, wherein, The injection into the car machine HMI sandbox in a non-invasive manner comprises: simulating ECU nodes by using a software Socket interface to send the virtual vehicle signal sequences to the car machine HMI sandbox in a non-invasive manner; in the injection process, normal signals are first sent, and then signal faults including signal packet loss, signal delay, or data frame errors are injected.

4. The signal simulation based automated test system for intelligent driver HMI visualization of claim 1, wherein, The car machine HMI sandbox is configured to perform the following operations: obtaining screen frame buffers by using ADB instructions or GPU DirectX interfaces to capture visual interfaces; recording resource occupation rates of CPUs and GPUs in real time when the measured car machine HMI system is running, and triggering an alarm when the resource occupation rates exceed preset thresholds.

5. The signal simulation based automated test system for intelligent driver HMI visualization of claim 1, wherein, The expected HMI response standards comprise: expected positions and colors of AR elements in the visual interfaces, and expected text contents; expected key words in audio prompt contents that need to be checked.

6. The signal simulation based automated test system for intelligent driver HMI visualization of claim 1, wherein, The automated analysis by using computer vision technology comprises: locating AR elements in the captured visual interfaces by using an OpenCV library through template matching and feature point detection algorithms, and calculating position deviations of the AR elements; identifying text regions in the interfaces by using OCR technology, and comparing identification results with the expected text contents.

7. The signal simulation based automated test system for intelligent driver HMI visualization of claim 1, wherein, The automated analysis by using speech recognition technology comprises: transcribing the audio streams by using automatic speech recognition technology; counting and checking whether the expected key words and their occurrence times exist in the transcribed texts.

8. The signal simulation based automated test system for intelligent driver HMI visualization of claim 1, wherein, The quantitative evaluation comprises performing timing compliance detection: recording timestamps of injection of the virtual vehicle signal sequences; detecting a timestamp of an expected feature appearing in the visual interface or an expected keyword starting playing in the audio stream; calculating the time difference as a response delay and comparing it with a predefined delay threshold.

9. The signal simulation based automated test system for intelligent driver HMI visualization of claim 1, wherein, The system is configured to perform a conflict scenario test, the process of which is: obtaining logically conflicting virtual vehicle signal sequences from a signal generalization scenario library; injecting the conflicting signal sequences into a car machine HMI sandbox simultaneously through a signal injection engine; verifying whether the tested car machine HMI system renders and prompts according to a preset priority strategy or degradation strategy through an automated analysis engine.

10. A method for automated testing of intelligent driving HMI visualization based on signal simulation, characterized in that, The intelligent driving HMI visualization automated test method based on signal simulation includes: obtaining virtual vehicle signal sequences generated based on parameterized scenarios from a signal generalization scenario library, the virtual vehicle signal sequences simulating driving or parking conditions of an intelligent driving system; injecting the virtual vehicle signal sequences into a car machine HMI sandbox running a tested car machine HMI system in a non-intrusive manner through a signal injection engine; receiving and responding to the virtual vehicle signal sequences through the car machine HMI sandbox, and outputting corresponding visual interfaces and audio streams; obtaining expected HMI response standards corresponding to the current virtual vehicle signal sequences from the signal generalization scenario library through an automated analysis engine, capturing visual interfaces and audio streams from the car machine HMI sandbox, and performing automated analysis on the visual interfaces based on the expected HMI response standards and using computer vision technology, and on the audio streams using speech recognition technology, to quantitatively evaluate the rendering accuracy and prompt content compliance of the tested car machine HMI system.

Images