Optimization methods and systems for automotive infotainment systems and voice interaction testing based on reinforcement learning

By using a reinforcement learning-based voice interaction testing method, a comprehensive reward is calculated using multi-source data to drive the optimization of test cases and vehicle-mounted voice models. This solves the problem of the separation between evaluation and optimization in existing technologies and achieves efficient and accurate optimization of voice interaction testing.

CN123072373APending Publication Date: 2026-11-10CHANGSHA YIFENG AUTOMOBILE TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611418337.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-14
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing automotive infotainment voice interaction testing methods are inefficient, with evaluation and optimization being disconnected. Test cases rely on manual design and static pre-setting, and the evaluation dimensions are limited, making it impossible to comprehensively quantify the semantic consistency between the response content and the user's intent.

Method used

A reinforcement learning-based voice interaction testing method is adopted. By collecting multi-source interaction data, a dual-tower semantic evaluation model is used to calculate semantic similarity and interface state matching degree. Combined with bus signal matching degree and response latency, a comprehensive reward is generated, which drives the reinforcement learning strategy network to optimize test cases and vehicle voice model.

Benefits of technology

It achieves a closed loop for quantifying and optimizing the evaluation results of in-vehicle voice interaction, autonomously expands the coverage of test cases, and enables the evaluation results to comprehensively characterize the interaction quality, thereby improving the testing efficiency and accuracy of voice interaction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123072373A_ABST
    Figure CN123072373A_ABST
Patent Text Reader

Abstract

This invention discloses a voice interaction testing optimization method, system, and in-vehicle infotainment system based on reinforcement learning. The method collects multi-source voice interaction data from the in-vehicle infotainment system. A dual-tower semantic evaluation model with shared parameters calculates the semantic similarity between the user's voice input and the concatenated text of the instructions and responses output by the in-vehicle infotainment system, using this as a semantic reward. A comprehensive reward is obtained by combining interface state matching degree, bus signal matching degree, and response latency. The reinforcement learning policy network selects actions from an action space that includes generating test cases, adjusting thresholds, and choosing optimization strategies, based on historical reward distribution and other states. When generating test cases, intent categories with clustered low-reward samples are identified as weak points in the evaluation. A large language model generates adversarial test commands and issues them to the in-vehicle infotainment system for execution. The policy network parameters are updated based on trajectory data. A training sample set is constructed according to reward levels to generate the optimized product of the in-vehicle infotainment system's voice model, which is then pushed to the in-vehicle application. Testing and optimization form a continuous closed loop.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of automotive technology testing, specifically relating to an automotive infotainment system and a method and system for optimizing voice interaction testing based on reinforcement learning. Background Technology

[0002] With the development of intelligent connected vehicles, voice interaction has become one of the most important human-machine interaction methods in vehicles. Users control vehicle functions such as air conditioning, windows, and navigation and obtain information services through natural language. The voice interaction chain involves multiple links such as voice recognition, semantic understanding, command execution, and interface presentation. Moreover, users' expressions are characterized by colloquialism, multiple intentions, and diverse expressions, which makes the testing and continuous optimization of the quality of in-vehicle voice interaction an unavoidable part of the development of in-vehicle systems.

[0003] Currently, testing of in-vehicle voice interaction mainly relies on manual testing in real vehicles and rule-based automated testing. Manual testing involves testers interacting with the voice assistant in a real vehicle and judging response quality based on personal experience. This method is costly and time-consuming to prepare and execute, making it difficult to keep pace with the rapid iteration of in-vehicle software. Test conclusions also rely on subjective judgment, with different people potentially offering significantly different evaluations of the same response, lacking a unified quantitative evaluation standard. Furthermore, the number of interaction scenarios that can be covered on real roads is limited, making it difficult to exhaustively cover all the expressions used by users in actual use. Rule-based automated testing, on the other hand, simulates preset voice command inputs using scripts and judges the correctness of the response text using string matching or regular expressions. Test cases for this type of solution need to be written manually in advance. Once the test case set is formed, it is static and fixed, making it difficult to expand it in a targeted manner as problems are discovered during testing. The test objects are concentrated on the preset correct path, and there is a lack of consideration for diverse user expressions, compound commands, and abnormal situations such as commands not being executed or being executed incompletely. The judgment method is limited to whether the text is consistent, which cannot reflect semantic deviations, and it is also difficult to combine the effects of command execution links such as interface changes and the actual execution of vehicle functions to comprehensively evaluate the interaction quality.

[0004] Besides the methods mentioned above, other solutions involve collecting user interaction logs with the voice assistant, having them manually annotated and scored, or using hardware such as simulation robots to reproduce acoustic scenarios inside the vehicle, and then using an evaluation model to conduct a multi-dimensional assessment of the vehicle's response. The former has a long cycle from data collection to feedback of evaluation results, and the scoring results are usually only used to generate analysis reports, without being further transformed into input for improving the vehicle's voice model; the latter's scenario arrangement is limited by hardware conditions, and its optimization is limited to the iteration of parameters in the evaluation stage, similarly without extending to the vehicle's voice model under test.

[0005] Existing technologies suffer from the following problems: evaluation and optimization are disconnected; evaluation results cannot serve as signals to drive improvements in the vehicle's voice model, resulting in a situation of inspection without modification; and the improvement of interaction quality still relies on manual analysis and offline development. Test cases depend on manual design and static pre-setting; the testing system itself does not have the ability to autonomously generate targeted test cases based on test performance, and the test coverage is difficult to expand autonomously as the testing process progresses. Evaluation dimensions and judgment criteria are singular and rigid, making it impossible to comprehensively quantify the semantic consistency between response content and user intent, as well as the actual effect of command execution. The evaluation results are difficult to accurately depict the full picture of interaction quality. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to overcome the shortcomings of the existing automotive infotainment system voice testing methods in terms of efficiency, thereby providing an optimized method and system for automotive infotainment system and voice interaction testing based on reinforcement learning.

[0007] A reinforcement learning-based method for optimizing voice interaction testing includes the following steps: Step S1: Collect multi-source interactive data generated by the vehicle's voice interaction. The multi-source interactive data includes user voice input text, vehicle output command text and voice response text, interface state change data, bus execution signals and response delay. Step S2: Input the user's voice input text into the first tower of the dual-tower semantic evaluation model, and input the command text and voice response text concatenated into the second tower that shares parameters with the first tower. Extract semantic vectors respectively and calculate the semantic similarity between the two semantic vectors. Determine the semantic similarity as the semantic reward. Step S3: Determine the interface state matching degree based on the interface state change data, determine the bus signal matching degree based on the bus execution signal, and sum the semantic reward, the interface state matching degree, and the bus signal matching degree by weight and deduct the penalty item corresponding to the response delay to obtain the comprehensive reward; Step S4: Input the current state into the reinforcement learning policy network, and the reinforcement learning policy network outputs the probability distribution of each candidate action in the action space. Then, select an action to execute based on the probability distribution. The current state includes at least the historical reward distribution formed by the comprehensive reward of each round of voice interaction. The action space includes the optimization strategies for generating test cases, adjusting the matching threshold, and selecting the vehicle voice model. Step S5: When the selected action is to generate test cases, the intent category of the interaction samples whose comprehensive reward is lower than the preset reward threshold is identified as the evaluation weakness. The evaluation weakness is input into the large language model to generate adversarial test instruction text for the evaluation weakness. Test speech is synthesized based on the adversarial test instruction text and sent to the vehicle system for execution. The process returns to step S1 to collect multi-source interaction data generated when the vehicle system executes the test speech, and obtains the comprehensive reward for the next round of voice interaction. Step S6: Update the parameters of the reinforcement learning policy network based on the trajectory data of multi-round voice interaction. The trajectory data includes the state, action and comprehensive reward of the corresponding round. Construct a training sample set based on the level of comprehensive reward, generate the optimized product of the vehicle voice model from the training sample set, and push it to the vehicle application.

[0008] Furthermore, after collecting the multi-source interaction data generated during the vehicle-mounted system voice interaction process, the method further includes preprocessing the collected interaction data, the preprocessing including: Filter out invalid samples that are rejected by the voice assistant; Filter out illusion samples that exceed the preset instruction set; Data containing multiple instruction actions is split into multiple samples; A multi-source confidence vector is constructed based on speech recognition confidence, optical character recognition confidence, and interface control matching degree; The multi-source confidence vector is used to form a component of the current state of the input reinforcement learning policy network.

[0009] Furthermore, the step of extracting semantic vectors from the first tower and the second tower respectively, and calculating the semantic similarity between the two extracted semantic vectors, includes: The first tower extracts the CLS vector from the text corresponding to the user's voice input, and the second tower extracts the CLS vector from the concatenated text input. The two CLS vectors are reduced to the same dimension through a fully connected layer; The cosine similarity between the two vectors after dimensionality reduction is calculated and used as the semantic similarity.

[0010] Furthermore, determining the interface state matching degree based on the interface state change data includes: Optical character recognition is performed on the vehicle screen to obtain text elements and their coordinates. The interface node tree is obtained through the accessibility interface, and the positional changes of interface controls between adjacent frames are determined by cross-frame anchor point tracking. The interface data from the above multiple sources are aligned and merged according to timestamps to construct a multi-source interface fusion index with controls as nodes and spatial adjacency and hierarchical inclusion relationships between controls as edges; Based on the multi-source interface fusion index, compare the interface state snapshots of two frames before and after the voice command execution, identify newly added controls, disappeared controls, and controls with changed attributes, and obtain the actual control difference. The predefined expected state assertions in the test cases are matched item by item with the actual control differences. The interface state matching degree is determined based on the proportion of successfully matched assertions and the changes of controls in the actual control differences that exceed the expected state assertions.

[0011] Furthermore, the current state also includes vehicle configuration, scene label, dialect category, and multi-source confidence vector; In the action space, the actions for generating test cases include instruction text templates, noise configuration parameters, and dialect types; the actions for adjusting matching thresholds include semantic similarity thresholds and intent matching thresholds; and the actions for selecting optimization strategies for the vehicle-mounted voice model include prompt word updates, low-rank adaptation fine-tuning, and direct preference optimization sample generation.

[0012] Furthermore, the reinforcement learning policy network includes a shared backbone network and multiple action heads. The shared backbone network takes the current state as input and outputs a feature vector. Each action head corresponds to a type of action in the action space and outputs the probability distribution of the corresponding action based on the feature vector. Among them, the action head that generates test cases and the action head that selects optimization strategies output the probability distribution of discrete actions, and the action head that adjusts the matching threshold outputs the probability distribution of continuous actions. The step of updating the parameters of the reinforcement learning policy network based on the trajectory data of each of the multiple rounds of voice interaction includes: using a proximal policy optimization algorithm to calculate the advantage estimate and importance sampling ratio based on the trajectory data, and updating the parameters of the reinforcement learning policy network by pruning the objective function.

[0013] Furthermore, the step of inputting the information of the assessed weaknesses into a large language model to generate adversarial test instruction text targeting the assessed weaknesses includes: Obtain the context information corresponding to the evaluation weakness, the context information including at least one of the following: the pattern of low reward samples under the evaluation weakness, historical failure cases, and the voice assistant capability boundary of the current vehicle model; The context information and action parameters are assembled into prompt words and submitted to the large language model. The action parameters include target intent category, scene label, dialect type and noise configuration. The text output by the large language model is subjected to structured parsing and verification. The verification includes field integrity verification, text compliance verification, and correlation verification between the adversarial test instruction text and the evaluation weakness. When the verification fails, the error message is appended to the prompt word and the large language model is re-requested for generation, or after the retry fails, a template instruction matching the evaluation weakness is selected from the preset adversarial instruction template library as the adversarial test instruction text. The step of synthesizing test speech based on the adversarial test instruction text includes: selecting a corresponding speech synthesis speaker configuration according to the dialect type, synthesizing speech through a speech synthesis engine, and mixing the synthesized speech with a noise signal selected from a noise library according to the noise configuration according to the signal-to-noise ratio to obtain the test speech.

[0014] Furthermore, the step of constructing a training sample set based on the comprehensive reward of each round of voice interaction includes: taking the interaction samples with a comprehensive reward higher than a first preset reward threshold as positive examples and the interaction samples with a comprehensive reward lower than a second preset reward threshold as negative examples, and constructing a direct preference optimization training sample set. The optimization output includes at least one of the following: prompt word update package, low-rank adaptation fine-tuning weights of the vehicle voice model, and update parameters of the multi-source interface fusion index model. The step of pushing the optimized product to the vehicle-mounted application includes: pushing the optimized product to the vehicle-mounted application via the OTA channel.

[0015] A voice interaction testing optimization system based on reinforcement learning, comprising: The data acquisition module is used to collect multi-source interactive data generated during the voice interaction process of the vehicle system. The multi-source interactive data includes the text corresponding to the user's voice input, the command text and voice response text output by the vehicle system, interface state change data, bus execution signals, and response delay. The semantic evaluation module includes a dual-tower semantic evaluation model, which is used to input the text corresponding to the user's voice input into the first tower of the dual-tower semantic evaluation model, and to input the command text and voice response text output by the vehicle system into the second tower of the dual-tower semantic evaluation model that shares parameters with the first tower. The first tower and the second tower respectively extract semantic vectors and calculate semantic similarity, and determine the semantic similarity as semantic reward. The reward fusion module is used to determine the interface state matching degree based on the interface state change data and the bus signal matching degree based on the bus execution signal. It then performs a weighted summation of the semantic reward, the interface state matching degree, and the bus signal matching degree, and deducts the penalty term corresponding to the response delay to obtain a comprehensive reward. The strategy decision module includes a reinforcement learning policy network, which is used to input the current state, including at least the historical reward distribution determined based on the comprehensive reward of each round of voice interaction, into the reinforcement learning policy network. The reinforcement learning policy network outputs the probability distribution of each candidate action in the action space and selects an action based on the probability distribution. The action space includes optimization strategies for generating test cases, adjusting matching thresholds, and selecting the vehicle voice model. The test case generation module is used to determine the evaluation weaknesses based on the intent categories of interaction samples whose comprehensive reward is lower than a preset reward threshold in each round of voice interaction when the selected action is to generate test cases. It then uses a large language model to generate adversarial test instruction text targeting the evaluation weaknesses and synthesizes test speech based on the adversarial test instruction text to send to the vehicle system for execution. The model feedback module is used to construct a training sample set based on the comprehensive reward of each round of voice interaction, generate an optimized product of the vehicle voice model based on the training sample set, and push the optimized product to the vehicle application.

[0016] A car infotainment system includes: The voice interaction device is used to perform voice recognition and semantic understanding on the user's voice input, output the vehicle system command text and voice response text, and execute the corresponding control commands; The data acquisition device is used to collect data on the interface status changes, bus execution signals, and response delays of the vehicle's infotainment system, and upload them to the cloud server. The local evaluation device is equipped with a vehicle-mounted evaluation model obtained by distillation of a dual-tower semantic evaluation model deployed in the cloud. It is used to perform local voice interaction evaluation through the vehicle-mounted evaluation model when cloud evaluation is unavailable, based on the network quality monitoring results. A model update device is used to receive and apply the pushed optimization products, wherein the optimization products include at least one of the following: prompt word update package, low-rank adaptation fine-tuning weights of the vehicle voice model, and update parameters of the multi-source interface fusion index model.

[0017] Beneficial effects: This invention quantifies the evaluation results of in-vehicle voice interaction into a comprehensive reward, which drives the decision-making of the reinforcement learning strategy network and the optimization of the in-vehicle voice model, making the evaluation and optimization stages a closed loop through a unified signal connection. Compared with the existing technology where the evaluation results are only used to determine whether to pass or fail and cannot be transformed into a basis for model improvement, the data obtained from each round of evaluation is used to update the test strategy and accumulate optimization samples, and evaluation and optimization are no longer separated.

[0018] In terms of evaluation accuracy, the dual-tower semantic evaluation model calculates the semantic similarity between the user's voice input text and the command text and voice response text output by the vehicle system. It can identify correct responses with different expressions but consistent semantics, and also detect incorrect responses with similar expressions but semantic deviations, avoiding misjudgments caused by relying on text literal comparison. The comprehensive reward incorporates interface state matching degree and bus signal matching degree on the basis of semantic reward, and imposes a penalty on response delay, so that the evaluation covers both semantic understanding and command execution levels at the same time, and the evaluation results can more completely describe the interaction quality.

[0019] In terms of test case generation, test cases are driven by the evaluation performance of each round of testing: the policy network selects actions based on states containing historical reward distributions, and intent categories with clusters of low-reward samples are identified as evaluation weaknesses. The large language model then generates targeted adversarial test instruction text accordingly. New test cases are focused on the weaknesses exposed by the evaluation rather than being randomly expanded. The test coverage expands autonomously with the iteration process, and the design of test cases no longer relies on manual pre-configuration and static maintenance.

[0020] In terms of continuous optimization, trajectory data from multiple rounds of interaction is used to update the policy network parameters. Interaction samples are constructed into a training sample set according to the overall reward, and then transformed into the optimized output of the vehicle-mounted voice model and pushed to the vehicle-mounted application. The test strategy and the tested model improve together under the same overall reward signal, and the improvement of interaction quality continues with closed-loop iteration, eliminating the need for manual translation and organization between evaluation and optimization. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the main method steps of the present invention; Figure 2 This is a schematic diagram of the strategy network structure of the present invention; Figure 3 This is a schematic block diagram of the system structure of the present invention. Detailed Implementation

[0023] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application are described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0024] Example 1: Reference Figure 1 and Figure 2 As shown, this embodiment provides a voice interaction testing optimization method based on reinforcement learning. This method quantifies the evaluation results of in-vehicle voice interaction into reward signals for reinforcement learning, thereby driving the joint optimization of the test case generation strategy and the in-vehicle voice model, forming a closed loop where evaluation, decision-making, generation, and feedback are interconnected. This method can be executed collaboratively by the processing device deployed on the cloud server and the in-vehicle terminal, and its overall process includes steps S1 to S6.

[0025] Step S1: Collect multi-source interactive data generated by the vehicle's voice interaction. The multi-source interactive data includes user voice input text, vehicle output command text and voice response text, interface state change data, bus execution signals, and response delay.

[0026] Specifically, data is collected through the cockpit voice assistant log system and automated testing client. The collected data covers the entire interaction chain: the original audio of the user's voice input and its corresponding recognized text and recognition confidence level, the intent recognition results obtained from natural language understanding, the command text output by the vehicle system, the voice response text output by the vehicle system, interface state change data (obtained through multi-source interface fusion indexing), execution signals from the CAN bus or FlexRay bus, response delays recorded with timestamps, and ECU logs. The collected data is uniformly uploaded to a big data platform for storage. The synchronous collection of multi-source data provides a data foundation for subsequent evaluation of each stage from semantic understanding to command execution.

[0027] After collecting multi-source interaction data, the process also includes preprocessing the collected interaction data. The preprocessing includes: filtering invalid samples rejected by the voice assistant; filtering phantom samples that exceed the preset instruction set, i.e., samples where the vehicle system outputs responses that do not belong to its instruction set range; splitting data containing multiple instruction actions into multiple samples, for example, if a user's sentence contains both air conditioning adjustment and navigation requests, it is split into two corresponding interaction samples; and constructing a multi-source confidence vector based on speech recognition confidence, optical character recognition confidence, and interface control matching degree.

[0028] The multi-source confidence vector is represented as: ; in, The confidence level for voice recognition is output by the vehicle's voice recognition engine. The confidence level for optical character recognition is output by the interface text recognition engine. Interface control matching degree reflects the degree of consistency between the actual controls on the interface and the expected controls; For bus signal confidence. In one implementation, The confidence vector is determined based on the cyclic redundancy check pass rate, the proportion of response frames, and the proportion of error frames in the bus messages, which is a conventional signal quality measurement method in this field. The multi-source confidence vector is used to form a component of the current state of the input reinforcement learning policy network, enabling policy decisions to be aware of the reliability of each data source.

[0029] Step S2: Input the user's voice input text into the first tower of the dual-tower semantic evaluation model, and input the command text and voice response text concatenated into the second tower that shares parameters with the first tower. Extract semantic vectors respectively and calculate the semantic similarity between the two semantic vectors. Determine the semantic similarity as the semantic reward.

[0030] The dual-tower semantic evaluation model employs a twin structure with shared parameters. Both towers share the same coding network parameters, mapping the two inputs to the same semantic space and ensuring similarity calculations are performed under a consistent spatial metric. This evaluation calculation is triggered when the test cases are executed and the multi-source data acquisition module uploads the vehicle system response data to the cloud.

[0031] The step of extracting semantic vectors and calculating the semantic similarity between the two semantic vectors specifically includes: the first tower extracts the CLS vector from the input user voice input text, and the second tower extracts the CLS vector from the input concatenated text; the two CLS vectors are reduced to the same dimension by fully connected layers; and the cosine similarity between the two vectors after dimensionality reduction is calculated as the semantic similarity.

[0032] In this embodiment, both towers are based on the BERT structure. After encoding the input text, each tower takes the output vector (768 dimensions) at the CLS position, and then passes it through two fully connected layers for progressive dimensionality reduction (768→128→32) to obtain a semantic vector with a dimension of 32. The two dimensionality reductions compress the vector dimension while preserving semantic distinguishability, thus reducing the overhead of similarity calculation.

[0033] Let u be the semantic vector obtained after dimensionality reduction of the first tower, and v be the semantic vector obtained after dimensionality reduction of the second tower. The cosine similarity is calculated as follows: ; in, This represents the dot product of two vectors. and Let represent the magnitudes of the two vectors respectively. The similarity s ranges from 1 to 2. The similarity was determined as the semantic reward. Semantic rewards characterize the consistency between the vehicle's system response and the user's input intent at the semantic level. They can identify correct responses with different expressions but consistent semantics, as well as incorrect responses with similar expressions but semantic discrepancies, thus avoiding misjudgments caused by literal comparison.

[0034] While calculating the semantic reward, this step calculates the interface state matching degree in parallel. Matching degree with bus signals The normalized value of the response latency is obtained and used for calculating the overall reward. The interface state matching degree is determined based on the control difference results of the multi-source interface fusion index, and the determination process is as follows.

[0035] During the construction of the multi-source interface fusion index, various interface information collection methods are deployed on the vehicle's infotainment system. The local OCR engine performs OCR recognition on a screenshot of the vehicle's screen every 200 milliseconds, extracting text elements on the screen (such as button labels, titles, and pop-up content), and recording the screen coordinates, font size, color, and OCR confidence score of each text element. The accessibility interface obtains the UI node tree of the current interface through the system's accessibility service. Each node contains a resource ID, class name, text content, content description, whether it is clickable, whether it is selected, and its parent-child node relationship. It also records the node's boundary rectangle on the screen. Cross-frame anchor point tracking uses optical flow or ORB feature point matching to track the positional changes of UI controls between adjacent frames, generating control displacement vectors to compensate for the delay between OCR and the accessibility interface during rapid interface changes (such as animation transitions). Control difference detection compares the UI node trees of two consecutive frames, identifies newly added, disappeared, and attribute-changed controls, and generates a difference list.

[0036] The interface data from the various sources mentioned above are aligned according to timestamps and then merged to construct a unified index structure. During merging, the node ID (or resource ID) of the accessibility interface is used as the unique identifier for the control. For controls that cannot be obtained through the accessibility interface (such as HTML elements within a WebView), the hash value of the OCR text and coordinates is used as a temporary identifier. For the text content of the same control, if both OCR and the accessibility interface provide the text, a confidence-weighted voting method is used to determine the final text. The accessibility interface has a weight of 1.0, and the OCR has a weight equal to its confidence level. The text from the source with the highest confidence level is ultimately selected. Semantic attributes are attached to each control, including control type (button, text, image, list item, etc.), function label (inferred through predefined rules or a lightweight classification model, such as navigation, air conditioning, media), state (selected, focused, available, visible), and parent-child hierarchical path. The merged UI state is represented as a sparse semantic graph, where nodes are controls and edges represent spatial adjacency (top, bottom, left, right) and hierarchical containment relationships between controls. This graph is stored in the vehicle's memory in JSON format and uploaded to the cloud as a snapshot every second.

[0037] When the vehicle's infotainment system executes a voice command, it compares the interface state snapshots of the two frames before and after the execution to identify the actual control differences that occurred. These differences include newly added controls, disappeared controls, and controls with changed attributes. Among these, controls with changed attributes record the changed attributes and their values ​​before and after the change.

[0038] In the test cases, the UI state changes that the voice command should cause are predefined and stored in the form of state change assertions. For example, for the command "Turn on the air conditioner", the expected state assertions include: the air conditioner control panel controls become visible, the temperature display control text changes to the set value, and the air conditioner switch control becomes selected.

[0039] When the selected evaluation is performed on a specific voice command, the predefined expected state assertions in the test cases are matched item by item with the actual control differences. The matching rules are as follows: if an assertion requires a control to become visible, check if the control is included in the new control list of the actual differences; if an assertion requires the control text to change to a specific value, check if the text attribute of the control in the actual differences changes to the expected value; if an assertion requires the control to be selected, check if the selection attribute of the control in the actual differences changes to true. Each successful assertion match earns 1 point, and each unsuccessful match earns 0 points; for partial matches (such as text values ​​that are close but not exactly equal), a score between 0 and 1 is awarded based on the text edit distance. If an unexpected control change occurs in the actual differences (such as accidentally turning on the seat heating), a penalty is applied based on the importance of the change.

[0040] The interface state matching degree is determined based on the proportion of successfully matched assertions and the changes in controls that exceed the expected state assertions in the actual control difference. The calculation formula is as follows: ; Where N is the total number of assertions, For the first The matching score of each assertion (values ​​from 0 to 1). This represents the number of control changes in the actual difference that exceed the expected state assertion. This is the penalty coefficient, which is set to 0.1 by default. The range of values ​​for is [ [0.5, 1], where negative values ​​indicate serious unexpected changes. Therefore, the interface state matching degree objectively reflects the correctness of the interface state changes after the vehicle's infotainment system executes voice commands.

[0041] Bus signal matching The method of comparing expected and actual signals is used to determine the signal: In the test cases, the host bus signals that the voice command should trigger and their expected values ​​or ranges are predefined. The actual acquired bus signals are compared item by item with the expected signals, and the ratio of successfully matched signals is used to determine the signal strength. The comparison and scoring method, along with the assertion-by-assertion matching method for interface state matching, is similar and belongs to the conventional signal verification methods in this field. The normalized value of the response delay... Divide the actual response delay by the preset delay limit and truncate to... The interval is obtained.

[0042] Step S3: Determine the interface state matching degree based on the interface state change data, determine the bus signal matching degree based on the bus execution signal, and sum the semantic reward, the interface state matching degree, and the bus signal matching degree by weight, and deduct the penalty term corresponding to the response delay to obtain the comprehensive reward.

[0043] The comprehensive reward is calculated using the following formula: ; in, For semantic reward, For interface state matching degree, For bus signal matching degree, To respond to the delayed normalized value, The weighting coefficients are satisfied. The comprehensive reward system covers both semantic understanding and instruction execution, and penalizes slow responses, allowing the evaluation results to fully characterize the interaction quality.

[0044] The weighting coefficients are dynamically determined based on the test conditions. In one implementation, the weighting coefficients are obtained by adjusting a preset baseline value using scaling factors output by the reinforcement learning policy network and then normalizing the adjustment. The adjustment range is limited to a preset range to constrain the drift of the evaluation benchmark and maintain the neutrality of the reward signal. In another implementation, the weighting coefficients are determined by a weight determination mechanism independent of the reinforcement learning policy network training process (e.g., looking up a table by vehicle type, scene label, or dialect category) based on the current test conditions, ensuring that the reward calculation standard does not change with the optimized policy. Both methods can guarantee the stability of the comprehensive reward as an evaluation signal.

[0045] Step S4: Input the current state into the reinforcement learning policy network, which outputs the probability distribution of each candidate action in the action space, and selects an action to execute based on the probability distribution.

[0046] The current state includes at least the historical reward distribution formed by the combined rewards of each round of voice interaction, as well as vehicle configuration, scene labels (such as highway, city, parking, charging), dialect category, and the multi-source confidence vector constructed in step S1. The historical reward distribution enables the policy network to perceive historical evaluation performance, and the confidence vector enables it to perceive the reliability of the data, thereby making decisions with sufficient information.

[0047] The action space includes generating test cases, adjusting matching thresholds, and selecting optimization strategies for the vehicle-mounted speech model. Specifically, generating test cases involves actions such as selecting instruction text templates, noise configuration parameters, and dialect types; adjusting matching thresholds includes semantic similarity thresholds and intent matching thresholds; and selecting optimization strategies for the vehicle-mounted speech model includes actions such as cue word updates, low-rank adaptation fine-tuning (LoRA), and direct preference optimization (DPO) sample generation.

[0048] In terms of the specific structure of the policy network, the reinforcement learning policy network includes a shared backbone network and multiple action heads. The shared backbone network is a three-layer fully connected neural network (256→512→256), which takes the current state as input and outputs a feature vector. All action heads share this feature representation, thereby utilizing cross-task correlations. The action heads that generate test cases are sequentially processed through Linear(256,128), ReLU, ... Following Softmax, it outputs the probability distribution of combined discrete actions (combinations of instruction templates, noise configurations, and dialect types). The size of the action space; the action head of the selected model optimization strategy is sequentially processed through Linear(256,64), ReLU, ... Following Softmax, it outputs the discrete probability distributions of the three optimization strategies. The size of the action space is defined; the action head with the adjusted matching threshold is passed through Linear(256,128), ReLU, and Linear(128,3) and then Tanh to output the mean of the three-dimensional continuous action. In addition, the learnable log standard deviation is output through an independent linear layer.

[0049] For continuous actions that adjust the matching threshold, the sampling method is to sample from a Gaussian distribution: ; Where μ is the mean vector output by Tanh (each component takes values ​​of...) 1,1)). The standard deviation is used. The sampled values ​​are then linearly mapped to the actual threshold range. ; Where z is the value truncated to [ ] after sampling. The values ​​of [1,1], and This refers to a preset lower and upper limit for the threshold, such as a semantic similarity threshold. The mapping range is [0.6, 1.0].

[0050] Since the three action heads are conditionally independent given shared features, the joint action... The probability is the product of the probabilities of each action head: ; in, This is the current state. and Let be the probability of the discrete action head. The probability density function value is a continuous Gaussian distribution. During sampling, samples are independently taken from the distributions of the three action heads and combined to obtain the final action.

[0051] Regarding the action path of adjusting the matching threshold: In one implementation, the semantic similarity threshold is used to determine whether the response meets the semantic matching requirements, and the intent matching threshold is used to determine whether the intent recognition is correct. Both only affect the pass / fail determination of the test results and do not participate in the calculation of the comprehensive reward, so that the adjustment of the judgment criteria will not directly change the reward signal and avoid the evaluation benchmark from drifting with the strategy. In another implementation, the threshold is used as the normalization basis of the corresponding matching degree to participate in the reward calculation. In this case, the coupling relationship between threshold adjustment and reward calculation is clearly limited to a preset range.

[0052] Step S5: When the selected action is to generate test cases, the intent categories of interaction samples whose comprehensive reward is lower than the preset reward threshold are identified as evaluation weaknesses. The evaluation weaknesses are input into the large language model to generate adversarial test instruction text for the evaluation weaknesses. Test speech is synthesized based on the adversarial test instruction text and sent to the vehicle system for execution. The process returns to step S1 to collect multi-source interaction data generated when the vehicle system executes the test speech, and obtains the comprehensive reward for the next round of voice interaction.

[0053] In this step, the policy network and the large language model collaborate in a hierarchical manner: the policy network handles decision-making and does not directly generate text, only outputting action types and corresponding parameters; the large language model handles the actual text generation. Their division of labor is shown in Table 1. Table 1: Division of Labor between PPO Policy Network and Large Language Model

[0054] Obtain the contextual information corresponding to the evaluation weakness. The contextual information includes at least one of the following: the pattern of low-reward samples under the evaluation weakness, historical failure cases, and the capability boundaries of the voice assistant of the current vehicle model. For example, a typical pattern of historical low-reward samples under a certain intent category is that when the user says "it's a bit hot," the vehicle system only turns on the air conditioning but does not close the windows. Historical failure cases include navigation destination recognition errors. The capability boundaries of the current vehicle model include not supporting cross-domain compound commands.

[0055] The contextual information and action parameters are assembled into prompt words and submitted to the large language model. The action parameters include the target intent category, scene label, dialect type, and noise configuration, and may further include a difficulty level to indicate the generation of boundary classes or adversarial instructions. The large language model can be, for example, a model like Qianwen 3. For instance, targeting the weakness of "combined air conditioning and navigation instructions," the adversarial test instruction text generated by the large language model is "I'm a little hot, please find me the nearest charging station and navigate there." When the specified dialect is Sichuanese, the generated text is "I'm a little hot, please find me the nearest charging station and navigate there."

[0056] To ensure the output of the large language model can be correctly parsed by subsequent processes, the output format requirements are explicitly specified in the model's prompts. The model is forced to return results in structured JSON format, and the prompts provide both positive examples that conform to the format requirements and negative examples that do not. The output format includes fields such as instruction text, target intent category, applicable scenario label, difficulty level, and other optional representations.

[0057] The text output by the large language model is subjected to structured parsing and validation. This validation includes field integrity verification, text compliance verification, and relevance verification between the adversarial test instruction text and the evaluation weakness. During parsing, the first valid JSON object is extracted from the model output, supporting nested curly brace matching; if extraction fails, regular expressions are used for layer-by-layer matching, or keyword truncation is performed. Field integrity verification checks whether required fields are complete; if missing, default values ​​are inferred from the context (e.g., the difficulty level defaults to medium). Text compliance verification includes: instruction text length should be between 5 and 200 characters; retrying is triggered if it is too short or too long; determining whether it is the target language of the test using a language detection model; and filtering instructions containing illegal content. Relevance verification calculates the semantic similarity between the instruction text and the current weakness intent category using a lightweight sentence embedding model; if the similarity is below a threshold (e.g., 0.5), it is considered off-topic and a retry is triggered.

[0058] When verification fails, an error message is appended to the prompt word, and the large language model is re-requested for generation. Alternatively, after a failed retry, a template instruction matching the evaluation weakness is selected from a pre-built adversarial instruction template library as the adversarial test instruction text. The template library contains hundreds of instruction texts pre-written by experts, covering common weaknesses. Three consecutive failures result in an exception log and notification to the test management platform, skipping the current generation and allowing the policy network to select other actions (such as adjusting the threshold or selecting an optimization strategy). Successfully generated test cases, after execution and receiving a comprehensive reward, have their generation process quality information (whether retried, verification time, and whether the final source is the large model or the template library) stored in the test case library's metadata as feedback signals for subsequent optimization of prompt word strategies. Thus, the free text output of the large language model is transformed into structured, verifiable, and executable test cases.

[0059] The step of synthesizing test speech based on the adversarial test instruction text includes: selecting a corresponding speech synthesis speaker configuration according to the dialect type, synthesizing speech through a speech synthesis engine, and mixing the synthesized speech with a noise signal selected from a noise library according to the noise configuration at a signal-to-noise ratio to obtain the test speech. The noise library may contain, for example, noise audio such as urban road noise, high-speed wind noise, and air conditioning noise, and the dialect library may contain, for example, speaker configurations for Sichuan dialect and Cantonese.

[0060] The configured test cases are injected into the automated test client, where they are synthesized into speech by a speech synthesis engine, noise is added, and then the results are sent to the vehicle's infotainment system or simulation environment for execution. After execution, the vehicle's infotainment system collects response data, calculates the comprehensive reward R_{t} for this round, and feeds it back to the policy network for the next update.

[0061] In a specific application scenario, the cloud issues a test command: "In a highway scenario, the passenger says 'Navigate to Chunxi Road' in Sichuan dialect, while the driver says 'Close the windows'." The command is analyzed using environmental and user behavior simulation to generate simulated high-speed GPS signals, summer high-temperature parameters, two voice signals, and their time alignment. Acoustic simulation superimposes the two voice signals with high-speed wind noise, simulating the difference in sound source location between the driver and passenger, and outputs a mixed audio. Vehicle system simulation executes the two commands (close windows and navigate) based on the recognition results and outputs the execution status via bus simulation. Network simulation maintains preset network conditions. All data is then aggregated for evaluation.

[0062] Step S6: Update the parameters of the reinforcement learning policy network based on the trajectory data of multi-round voice interaction. The trajectory data includes the state, action and comprehensive reward of the corresponding round. Construct a training sample set based on the level of comprehensive reward, generate the optimized product of the vehicle voice model from the training sample set, and push it to the vehicle application.

[0063] The policy update employs the Proximal Policy Optimization (PPO) algorithm. The trajectories (s_{t}, a_{t}, R_{t}) collected in each round are input into the PPO update process, where s_{t} is the current state in round t, a_{t} is the selected action, and R_{t} is the total reward for that round. First, the advantage estimate is calculated by the value network, and then the policy network parameters are updated by pruning the objective function.

[0064] In multi-action head scenarios, the importance sampling ratio is the product of the ratios of each action head: ; in, As the current strategy, The old strategy before the update. Let j be the action of the j-th action head in round t. Equivalently, the log probabilities of the three action heads are added together to obtain the total log probability: ; The objective function for PPO pruning is: ; in, Indicates the expectation of the time step. For advantage estimation, clip(·) is used to clip the importance ratio to [1]. The clipping function for the interval [ε, 1+ε]. These are the pruning parameters. During backpropagation, the gradients are propagated back to the shared backbone network via the output layers of the three action heads, enabling synchronous updates of the parameters of each action head and the backbone.

[0065] The value network and policy network share the backbone network and have independent value heads V(s), with their loss being the mean squared error. ; in, The valuation of the current state for the value network. This is used to accumulate rewards for discounts. In frameworks like PyTorch, the log probabilities of the three action heads are summed to obtain the total log probability, which is then substituted into the loss function, allowing the framework to automatically perform gradient propagation.

[0066] In terms of model feedback, interaction samples with a comprehensive reward higher than a first preset reward threshold are used as positive examples, and interaction samples with a comprehensive reward lower than a second preset reward threshold are used as negative examples to construct a direct preference optimization training sample set. For example, samples with R>0.85 are used as positive examples, and samples with R<0 are used as negative examples. In one implementation, high-reward samples and low-reward samples under the same intent category are paired to form preference sample pairs, which are used as preferred and inferior responses, respectively; in another implementation, for multiple responses under the same user's voice input text, they are paired after being sorted from high to low comprehensive reward. Preference sample pairs are used for direct preference optimization training.

[0067] The optimization output includes at least one of the following: a prompt word update package, low-rank adaptation fine-tuning weights for the vehicle-mounted voice model, and updated parameters for the multi-source interface fusion index model. The optimization output is pushed to the vehicle-mounted application via OTA: the prompt word or system command update package is pushed to the vehicle-mounted voice assistant; the low-rank adaptation fine-tuning weights for the lightweight vehicle-mounted model are pushed and loaded; and the updated parameters for the multi-source interface fusion index model are pushed to the vehicle-mounted application. Thus, the evaluation data is directly converted into a basis for improving the vehicle-mounted voice model.

[0068] Regarding vehicle-cloud collaboration, the test client continuously monitors network quality. When cloud-based execution becomes unavailable, it automatically downgrades to a lightweight model on the vehicle's in-vehicle system for evaluation. This lightweight model is distilled from a dual-tower semantic evaluation model in the cloud. Under the cloud-vehicle architecture, the evaluation agent is deployed in the cloud, while the in-vehicle system is responsible for collecting multi-source interface fusion indexes and execution signals, thus achieving a division of labor between cloud computing power and in-vehicle data collection.

[0069] In addition, the large language model also supports root cause analysis: when the intelligent analysis and evaluation identifies batch test failures, it submits the log summary of the failure samples, multi-source data snapshots, and system topology relationships to the large language model. The large language model combines system knowledge to perform causal reasoning and generate a report that includes fault chain analysis, root cause location, and repair suggestions, continuously enriching the test knowledge base and issue database.

[0070] Example 2: Reference Figure 2 As shown, this embodiment provides a voice interaction testing optimization system based on reinforcement learning, including a data acquisition module, a semantic evaluation module, a reward fusion module, a strategy decision module, a test case generation module, and a model feedback module.

[0071] The data acquisition module is used to collect multi-source interactive data generated during the vehicle-mounted system's voice interaction process. The multi-source interactive data includes the text corresponding to the user's voice input, the command text and voice response text output by the vehicle-mounted system, interface state change data, bus execution signals, and response delays.

[0072] The semantic evaluation module includes a dual-tower semantic evaluation model, which is used to input the text corresponding to the user's voice input into the first tower of the dual-tower semantic evaluation model, and to input the command text and voice response text output by the vehicle system into the second tower of the dual-tower semantic evaluation model that shares parameters with the first tower. The first tower and the second tower respectively extract semantic vectors and calculate semantic similarity, and determine the semantic similarity as semantic reward.

[0073] The reward fusion module is used to determine the interface state matching degree based on the interface state change data and the bus signal matching degree based on the bus execution signal. It then performs a weighted summation of the semantic reward, the interface state matching degree, and the bus signal matching degree, and deducts the penalty term corresponding to the response delay to obtain a comprehensive reward.

[0074] The strategy decision module includes a reinforcement learning policy network, which is used to input the current state, including at least the historical reward distribution determined based on the comprehensive reward of each round of voice interaction, into the reinforcement learning policy network. The reinforcement learning policy network outputs the probability distribution of each candidate action in the action space and selects an action based on the probability distribution. The action space includes optimization strategies for generating test cases, adjusting matching thresholds, and selecting the vehicle voice model.

[0075] The test case generation module is used to determine the evaluation weaknesses based on the intent categories of interaction samples whose comprehensive reward is lower than a preset reward threshold in each round of voice interaction when the selected action is to generate test cases. It then uses a large language model to generate adversarial test instruction text targeting the evaluation weaknesses and synthesizes test speech based on the adversarial test instruction text to send to the vehicle system for execution.

[0076] The model feedback module is used to construct a training sample set based on the comprehensive reward of each round of voice interaction, generate an optimized product of the vehicle voice model based on the training sample set, and push the optimized product to the vehicle application.

[0077] The specific process of each module is the same as in Example 1, and will not be repeated here.

[0078] The system in this embodiment also includes a test management platform, which serves as the command center and human-computer interaction interface for the entire test system. It provides a graphical interface for configuring test scenarios and test cases, scheduling test tasks, monitoring real-time status and visualizing test results, and is responsible for communicating, coordinating and controlling with other modules in the system.

[0079] The cloud server is the core of the system's cloud computing, carrying four functional components: virtual cloud vehicle system, dual-tower semantic evaluation model, reinforcement learning policy network, and large language model. It is responsible for simulating real OTA cloud services and performing intelligent analysis and decision-making.

[0080] The virtual cloud vehicle system receives upgrade task instructions or test case trigger signals from the test management platform and automatically simulates the complete OTA interaction link between the real cloud and the vehicle system. When a test task starts, the virtual cloud vehicle system first retrieves the configuration information of the currently tested vehicle model (such as ECU list, software version number, differential algorithm type, security certificate fingerprint, etc.) from the big data platform, uses these structured parameters as the content to fill the interaction protocol, and then simulates the execution of interactive behaviors such as upgrade task distribution, upgrade package chunk transmission, differential calculation and merging, two-way security authentication handshake, and upgrade progress status report reception according to the communication protocol stack of the real cloud (HTTP / HTTPS, MQTT, COAP, etc.). For abnormal test scenarios, the virtual cloud vehicle system supports dynamically injecting abnormal events such as upgrade package corruption, signature verification failure, and download interruption to verify the robustness of the vehicle system under various abnormal conditions.

[0081] In terms of implementation, the virtual cloud vehicle system is built on a microservice containerized architecture, decomposing the functional components of the real cloud OTA service into independent simulated services and deploying them on a Kubernetes cluster. Specifically, these include: an upgrade task management service, simulating RESTful interfaces for creating, issuing, pausing, and canceling upgrade tasks in the cloud; an upgrade package distribution service, simulating CDN or P2P chunked transmission, supporting breakpoint resumption and fragment verification; a differential computation service, integrating open-source differential algorithms such as bsdiff and xdelta to simulate the process of generating differential upgrade packages in the cloud; a security authentication service, simulating PKI certificate issuance, signature verification, and device identity authentication processes (based on OpenSSL or a self-built CA); and a status report receiving service, simulating MQTT / HTTP interfaces for receiving status information such as upgrade progress, error codes, and version numbers reported by the vehicle system from the cloud. All services communicate through a service mesh (such as Istio), facilitating the injection of chaos engineering experiments such as latency and faults.

[0082] In terms of communication protocol stack simulation, a communication endpoint consistent with the real cloud is built using a protocol simulation framework: HTTP / HTTPS uses WireMock to record real cloud API responses to generate stubs, supporting dynamic returns of different status codes and response bodies; MQTT uses Mosquitto to build a message broker and configure Topic rules to simulate issuing upgrade commands and receiving vehicle status from the cloud; COAP uses the California framework to build a server to simulate IoT communication in a resource-constrained environment. All protocol simulators can dynamically inject anomalies such as network latency, packet loss, and connection resets using chaos engineering plugins (such as Chaos Mesh).

[0083] In terms of upgrade package and differential algorithm simulation, metadata (size, hash, partition information) of the real upgrade package is obtained from the big data platform, and the corresponding virtual upgrade package (which can be random data or a desensitized copy of the real image) is generated in the virtual cloud vehicle system. The differential calculation service uses bsdiff to generate differential patches between the old and new versions, and supports anomalies such as injected package corruption (randomly tampering with several bytes in binary data), signature failure (signing with an incorrect private key or removing the signature segment), version number conflict (returning a version number that is inconsistent with the current version of the vehicle system), and fragment missing (discarding some fragments during the block transmission process).

[0084] In terms of state management, an upgrade state machine is maintained for each virtual vehicle. States include idle, downloading, verifying, installing, activating, completed, and failed. The virtual cloud vehicle system advances the state machine based on the status messages reported by the vehicle system and triggers preset abnormal events in specific states (such as suddenly issuing a cancel upgrade command during the installation phase). The state machine engine uses a finite state machine framework (such as Spring Statemachine or Akka FSM) and supports state persistence and backtracking.

[0085] The virtual cloud vehicle system provides a RESTful management API, through which the test management platform can dynamically configure the currently simulated vehicle model and software version, upgrade package attributes (size, differential algorithm type, signature algorithm), exception injection rules (time point, exception type, duration), and network conditions. It also supports loading the entire test scenario at once using YAML / JSON configuration files, facilitating automated test pipeline calls. As a basic framework, reference implementations of the Uptane framework and cloud-based OTA service prototypes provided by open-source automotive cloud simulation platforms such as Eclipse Kuksa can also be consulted.

[0086] The multi-dimensional environment simulation cluster is the virtualization execution base of the system. It consists of four sub-modules: a network simulation management platform, a vehicle system simulation model, an environment and user behavior simulation platform, and an acoustic simulation module. It receives test commands from the cloud and returns the responses of the virtual vehicle to the data acquisition and monitoring module.

[0087] The network simulation management platform receives network configuration parameters from the test management platform or dynamic injection instructions from the cloud server, automatically constructing and controlling the communication link environment between the vehicle-mounted system and the cloud. When test cases need to simulate specific network conditions (such as weak signal areas, base station handover, network congestion), this submodule parses network parameters (network type 4G / 5G / V2X / Wi-Fi, bandwidth limit, round-trip latency, packet loss rate, jitter range, etc.) from the test case configuration, writes them into the ns-3 or OPNET-based network simulation kernel, and then monitors network traffic in real time during test execution, dynamically injecting abnormal events such as bandwidth limitations, high latency, packet loss, out-of-order packets, and connection interruptions according to preset rules. This submodule also supports time-series orchestration of network status, such as normal for the first 30 seconds, a 10-second delay increasing to 500 milliseconds, and recovery for the last 20 seconds, to accurately reproduce complex network fluctuation scenarios in the real world.

[0088] The vehicle system simulation model receives upgrade packages, control commands, or diagnostic requests from the cloud and automatically simulates the software logic and hardware interface behavior of a real vehicle controller. It comprises two core components. The virtual ECU model loads the ECU topology (including gateway, cockpit domain controller, body domain controller, powertrain domain controller, etc.) and software logic descriptions (state machine, signal mapping table, diagnostic service implementation) of the current vehicle model under test from a big data platform. It simulates the firmware behavior of a real ECU in software form, capable of receiving and running real upgrade packages (binary images), performing flashing, verification, and activation processes, and generating response signals consistent with those of a real ECU (such as ACK / NACK, status codes, and fault codes). The vehicle bus simulation network constructs a vehicle bus communication network such as CANFD, Ethernet, and LIN between the virtual ECU models. It reads bus topology parameters (node ​​ID, message period, signal definition, gateway routing table) from the test configuration and simulates the sending and receiving of bus messages, arbitration, error frame injection, and load stress testing in real time during test execution. The response signals generated by the virtual ECU are transmitted through the bus simulation network. Abnormal events in the bus simulation network (such as message loss or bus shutdown) will also be sensed and reacted to by the virtual ECU, realizing the simulation of the behavior of the whole vehicle system.

[0089] The environment and user behavior simulation platform receives scenario description parameters defined in the test cases and automatically generates external vehicle environmental conditions and in-vehicle user interaction behaviors. This submodule parses environmental parameters (number of GPS satellites, signal strength, latitude and longitude offset, ambient temperature, humidity, altitude, light intensity, etc.) and user behavior sequences (arranged along a timeline, such as the driver saying "turn on the air conditioning" at second 0, the passenger saying "navigate to the airport" at second 5, and the driver saying "cancel" at second 10) from the test cases, and injects the parameters into the corresponding simulation engine: the GPS signal generator outputs NMEA statements to the virtual GNSS receiver; the temperature model adjusts the temperature sensor values ​​of the virtual ECU; and the user behavior simulator generates multiple voice signals along the timeline through the TTS engine and superimposes them onto the input channel of the acoustic simulation module. This submodule also supports the generation of random behaviors (such as randomly spaced door opening and closing, seat adjustment) to simulate the uncertainty of real users.

[0090] The acoustic simulation module receives voice signals and environmental noise parameters from the environment and user behavior simulation platform, automatically constructing a high-fidelity in-vehicle acoustic scene and supporting multi-zone sound source localization simulation. This sub-module reads acoustic parameters from the test configuration (wind noise spectrum corresponding to vehicle speed, tire noise characteristics corresponding to tire model, fan noise curve corresponding to air conditioning setting, geometric layout of microphone arrays in each zone, sound source location coordinates, etc.), and mixes clean voice signals with noise signals at a specified signal-to-noise ratio based on a room impulse response (RIR) convolution model or beamforming simulation engine, simulating the time delay and attenuation of sound reaching the microphone array from different directions. This sub-module supports simultaneous simulation of sound sources in up to four independent zones (driver's seat, passenger's seat, left rear, right rear), each zone can be configured with independent dialect type, volume, and speech rate, realistically reproducing the acoustic mixing effect in multi-person dialogue scenarios. The generated mixed audio signal is sent to the vehicle's voice recognition front end.

[0091] The real-vehicle test agent and digital twin synchronization engine provide the test management platform with a two-way closed-loop verification capability between virtual simulation and physical vehicles. This module starts working when the simulation test is completed and intelligent analysis and evaluation identify potential defective use cases, or when sporadic interaction anomalies are captured during real-vehicle road testing (such as voice recognition failure under specific electromagnetic interference).

[0092] For forward synchronization (simulation to real vehicle), the digital twin synchronization engine retrieves test cases marked as suspected defects from the simulation test result database. This includes complete test inputs (voice command audio files, noise configuration parameters, network condition snapshots), simulation execution logs, and descriptions of the deviation between expected and actual results. The simulated defect test cases are packaged into executable test scripts for the real vehicle, with additional reproduction condition instructions (e.g., execution under the same vehicle model and software version). These scripts are then distributed via the vehicle network to the real vehicle test agent deployed on the actual test vehicle. Upon receiving the instructions, the real vehicle test agent automatically executes the test process according to the script, playing voice commands via the in-vehicle TTS, collecting vehicle system responses, recording UI screenshots and bus messages, and transmitting the results back to the test management platform in real time. The digital twin synchronization engine then outputs a forward synchronization report, including conclusions from the real vehicle reproduction verification (whether the defect actually exists, the reproduction success rate, and consistency analysis with simulation results).

[0093] For reverse synchronization (real vehicle to simulation), the engine extracts abnormal event fragments from the real-time data stream uploaded by the real vehicle test agent, including the original voice signal, ASR text, UI status sequence, CAN bus signal waveform, GPS coordinates, timestamp, and on-site environmental notes (such as whether there are high-voltage power lines nearby, weather conditions). First, based on timestamps and anomaly markers (such as a sudden drop in ASR confidence, a surge in bus error frames, and GPS positioning jumps), abnormal event segments are segmented, and each segment is characterized by multiple dimensions: Spectral analysis (FFT) is performed on the time series of CAN bus error frames to extract the interference frequency, duty cycle, and occurrence period; simultaneously, the change pattern of the interference source field strength is estimated using the radio frequency signal strength indication (RSSI) curve received by the vehicle antenna; the signal-to-noise ratio sequence, number of satellites, and positioning accuracy factor (HDOP / VDOP) in the NMEA statement output by the GPS receiver are analyzed to extract the attenuation start and end times, attenuation amplitude, and recovery time constant; ambient temperature and battery temperature are read from the CAN signal of the vehicle thermal management system, the temperature change rate is calculated, and the timestamp is used to determine whether it is accompanied by weather conditions such as rain, snow, or sun exposure; finally, the signal-to-noise ratio, noise spectrum envelope (Mel spectrum), and echo intensity are extracted from the original speech signal.

[0094] The feature parameters are then converted into configuration parameters recognizable by the simulation engine: the main frequency and duty cycle extracted by FFT are converted into interference injection parameters for the network simulation management platform (e.g., jamming_frequency = 2.4GHz, duty_cycle = 30%, period = 100ms), or the CAN error frame pattern is converted into an error frame injection script for the vehicle bus simulation network; the GPS signal-to-noise ratio sequence is fitted to an exponential decay function. ; in, The initial signal-to-noise ratio, For the attenuation floor value, The decay time constant (e.g.) = 5.2s, = 15dB), converting it into the GPS signal attenuation configuration of the environment and user behavior simulation platform; converting the temperature change rate into an environmental temperature change script (such as temperature_ramp: from 25 to 45 over 300s, i.e., linearly increasing from 25℃ to 45℃ within 300 seconds); converting the noise spectrum envelope into the noise configuration parameters of the acoustic simulation module (such as wind_noise_spectrum and snr = 8dB).

[0095] The system automatically generates structured simulation scenario configuration files (YAML format) based on the parameterization results. These files include the scenario name and the source vehicle test ID, configuration blocks for each submodule (network, vehicle, environment, acoustics), and timeline arrangement. For example, a scenario from a real vehicle test involving electromagnetic interference and GPS attenuation: network signal at second 0. 85dBm, GPS signal-to-noise ratio 45dB, 12 satellites, temperature 35℃; starting from the 30th second, inject CAN bus error frames (frequency 2.4GHz, duty cycle 30%) and start GPS signal-to-noise ratio attenuation (time constant 5.2s); at the 60th second, GPS signal-to-noise ratio drops to 15dB, 4 satellites, temperature 42℃.

[0096] The generated simulation scenario configuration file is distributed to each submodule of the multi-dimensional environment simulation cluster. A short 10-second pre-simulation is executed first to verify that key parameters are correctly loaded and produce the expected results. After the pre-simulation passes, the scenario is added to the routine test case library and marked as a real-vehicle traceability scenario for use in subsequent version regression testing. Finally, a reverse synchronization configuration package is output for the test management platform to incorporate into the routine test case library. Through this bidirectional synchronization mechanism, a continuously evolving data loop is formed between the virtual simulation and the physical vehicle.

[0097] The real-vehicle testing agent is installed in the glove box or trunk of the test vehicle as an embedded industrial control computer (such as one based on NVIDIA Jetson or Intel NUC). It connects to the vehicle system through various physical interfaces: a CAN-to-USB module (such as PEAK-System PCAN-USB) connects to the OBD-II interface to the vehicle's CAN / CANFD bus for sending diagnostic requests, reading ECU status, and monitoring bus messages; an onboard Ethernet adapter box (such as 100BASE-T1 to USB) connects to the onboard Ethernet network for high-speed data interaction with the cockpit domain controller and gateway; the industrial control computer's audio output is connected to the vehicle's microphone input circuit via a 3.5mm audio cable or Bluetooth module to enable automatic playback of voice commands; a USB-to-serial module simulates touchscreen clicks or knob operations to trigger specific functions of the vehicle's interface; a USB camera is installed in the rearview mirror to capture UI changes on the vehicle's screen as a backup verification method for OCR; and a GPS signal splitter simultaneously feeds the vehicle's GPS antenna signal into the industrial control computer to record precise positioning information. When the real vehicle testing agent performs the test, all data is stamped with a unified timestamp and then sent back to the test management platform in real time.

[0098] The data acquisition and monitoring module acts as the sensory nerve of the system, deployed at key nodes in both the simulation and real vehicle environments to achieve end-to-end, real-time, and synchronous data acquisition. The acquired data includes voice signals (raw audio and sound source localization information), ASR recognition text and confidence level, NLU intent recognition results, vehicle system output command text, vehicle system voice response text, UI interface state changes (through multi-source interface fusion indexing), CAN / FlexRay bus signals, response delay timestamps, and ECU logs. All acquired data is uploaded to the big data platform.

[0099] The intelligent analysis and evaluation engine is the core of the system's analysis and decision-making, comprising a big data platform, a dual-tower semantic evaluation model, a multi-source reward fusion module, a reinforcement learning policy network, an adversarial test case generation engine, and a model-feedback OTA push module. The big data platform stores and manages massive amounts of historical test data, vehicle configurations, and problem databases, forming test assets. The dual-tower semantic evaluation model calculates semantic rewards according to step S2 of Example 1. The multi-source reward fusion module calculates comprehensive rewards according to step S3 of Example 1. The reinforcement learning policy network makes decisions according to step S4 of Example 1. The adversarial test case generation engine identifies current evaluation weaknesses (intent categories and scenarios where low-reward samples cluster), uses a large language model to generate adversarial instruction text based on these weaknesses, configures acoustic conditions based on a noise database and a dialect database, and injects the generated test cases into the automated testing client for execution. The model feedback and OTA push module constructs the DPO training set by using high-reward samples (e.g., R>0.85) as positive examples and low-reward samples (e.g., R<0) as negative examples. It then pushes the prompt words or system command update package, LoRA fine-tuning weights, and updated multi-source interface fusion index model parameters to the vehicle terminal via OTA.

[0100] During system operation, a complete reinforcement learning closed loop is formed: the data acquisition and monitoring module collects multi-source interaction data and uploads it to the cloud; the dual-tower semantic evaluation model calculates semantic rewards; the multi-source reward fusion module calculates comprehensive rewards; the reinforcement learning policy network selects actions based on the current state; when the action is to generate test cases, the adversarial test case generation engine generates test cases with the help of a large language model and sends them out for execution; the execution results are then collected and evaluated again; the model feedback and OTA push module updates the policy network based on the comprehensive rewards and generates optimized products to push to the vehicle's infotainment system. This cycle repeats continuously, so that the test coverage and the quality of the vehicle's voice interaction continue to improve with iteration.

[0101] Example 3: This embodiment provides an in-vehicle infotainment system, including a voice interaction device, a data acquisition device, a local evaluation device, and a model update device.

[0102] The voice interaction device is used to perform speech recognition and semantic understanding on user voice input, output vehicle system command text and voice response text, and execute corresponding control commands. The voice interaction device includes a voice recognition front-end, a semantic understanding module, and a command execution module connected in sequence: the voice recognition front-end receives mixed audio collected by the microphone array and outputs the recognized text and confidence level; the semantic understanding module performs intent recognition on the recognized text; the command execution module calls the vehicle function interface (such as air conditioning, windows, navigation) according to the recognized intent, generates vehicle system command text, and at the same time, the voice response generation module outputs the voice response text, which is then broadcast through the speaker.

[0103] The data acquisition device is used to collect interface state change data, bus execution signals, and response delays from the vehicle's infotainment system and upload them to a cloud server. The collection of interface state change data is achieved through a multi-source interface fusion index deployed on the vehicle's infotainment system. Its construction method is the same as described in Embodiment 1: the local OCR engine recognizes screen screenshots every 200 milliseconds, the accessibility interface obtains the UI node tree, cross-frame anchor point tracking determines the positional changes of controls between adjacent frames, and multi-source data is fused according to timestamps into a sparse semantic graph with controls as nodes and spatial adjacency and hierarchical inclusion relationships as edges. A snapshot is uploaded every second. Bus execution signals are collected via the vehicle's CAN / FlexRay bus interface, and response delays are recorded with a unified timestamp. All data is uploaded to the cloud in real time for use by the cloud-based dual-tower semantic evaluation model and reward fusion module.

[0104] The local evaluation device deploys a vehicle-mounted evaluation model derived from a cloud-deployed dual-tower semantic evaluation model. This model is used to perform local voice interaction evaluation when cloud-based evaluation is unavailable, based on network quality monitoring results. The test client continuously monitors network quality and automatically downgrades to a lightweight vehicle-mounted model when cloud-based evaluation becomes unavailable. This lightweight model is a distilled version of the dual-tower semantic evaluation model, performing semantic similarity calculations within the vehicle-mounted device's computing power constraints, ensuring the evaluation loop remains unbroken during network anomalies. In this cloud-vehicle-mounted architecture, the evaluation agent is deployed in the cloud, while the vehicle-mounted device is responsible for collecting multi-source interface fusion indexes and execution signals. The cloud and vehicle-mounted devices collaborate according to network conditions.

[0105] The model update device receives and applies the pushed optimization products, which include at least one of the following: prompt word update package, low-rank adaptation fine-tuning weights of the vehicle's voice model, and update parameters of the multi-source interface fusion index model. The model update device receives the optimization products pushed by the cloud-based model feedback module via an OTA channel: upon receiving the prompt word update package, it updates the system commands of the vehicle's voice assistant; upon receiving the low-rank adaptation fine-tuning weights, it loads them into the lightweight vehicle model; and upon receiving the multi-source interface fusion index model parameters, it updates the local index model. Thus, the voice interaction capabilities of the vehicle's system are continuously updated with the iteration of the cloud-based testing loop.

[0106] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0107] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for optimizing voice interaction testing based on reinforcement learning, characterized in that, Includes the following steps: Step S1: Collect multi-source interactive data generated by the vehicle's voice interaction. The multi-source interactive data includes user voice input text, vehicle output command text and voice response text, interface state change data, bus execution signals and response delay. Step S2: Input the user's voice input text into the first tower of the dual-tower semantic evaluation model, and input the command text and voice response text concatenated into the second tower that shares parameters with the first tower. Extract semantic vectors respectively and calculate the semantic similarity between the two semantic vectors. Determine the semantic similarity as the semantic reward. Step S3: Determine the interface state matching degree based on the interface state change data, determine the bus signal matching degree based on the bus execution signal, and sum the semantic reward, the interface state matching degree, and the bus signal matching degree by weight and deduct the penalty item corresponding to the response delay to obtain the comprehensive reward; Step S4: Input the current state into the reinforcement learning policy network, and the reinforcement learning policy network outputs the probability distribution of each candidate action in the action space. Then, select an action to execute based on the probability distribution. The current state includes at least the historical reward distribution formed by the comprehensive reward of each round of voice interaction. The action space includes the optimization strategies for generating test cases, adjusting the matching threshold, and selecting the vehicle voice model. Step S5: When the selected action is to generate test cases, the intent category of the interaction samples whose comprehensive reward is lower than the preset reward threshold is identified as the evaluation weakness. The evaluation weakness is input into the large language model to generate adversarial test instruction text for the evaluation weakness. Test speech is synthesized based on the adversarial test instruction text and sent to the vehicle system for execution. The process returns to step S1 to collect multi-source interaction data generated when the vehicle system executes the test speech, and obtains the comprehensive reward for the next round of voice interaction. Step S6: Update the parameters of the reinforcement learning policy network based on the trajectory data of multi-round voice interaction. The trajectory data includes the state, action and comprehensive reward of the corresponding round. Construct a training sample set based on the level of comprehensive reward, generate the optimized product of the vehicle voice model from the training sample set, and push it to the vehicle application.

2. The method according to claim 1, characterized in that, After collecting the multi-source interaction data generated during the vehicle-mounted system voice interaction, the method further includes preprocessing the collected interaction data, which includes: Filter out invalid samples that are rejected by the voice assistant; Filter out illusion samples that exceed the preset instruction set; Data containing multiple instruction actions is split into multiple samples; A multi-source confidence vector is constructed based on speech recognition confidence, optical character recognition confidence, and interface control matching degree; The multi-source confidence vector is used to form a component of the current state of the input reinforcement learning policy network.

3. The method according to claim 1, characterized in that, The step of extracting semantic vectors from the first tower and the second tower respectively, and calculating the semantic similarity between the two extracted semantic vectors, includes: The first tower extracts the CLS vector from the text corresponding to the user's voice input, and the second tower extracts the CLS vector from the concatenated text input. The two CLS vectors are reduced to the same dimension through a fully connected layer; The cosine similarity between the two vectors after dimensionality reduction is calculated and used as the semantic similarity.

4. The method according to claim 1, characterized in that, Determining the interface state matching degree based on the interface state change data includes: Optical character recognition is performed on the vehicle screen to obtain text elements and their coordinates. The interface node tree is obtained through the accessibility interface, and the positional changes of interface controls between adjacent frames are determined by cross-frame anchor point tracking. The interface data from the above multiple sources are aligned and merged according to timestamps to construct a multi-source interface fusion index with controls as nodes and spatial adjacency and hierarchical inclusion relationships between controls as edges; Based on the multi-source interface fusion index, compare the interface state snapshots of two frames before and after the voice command execution, identify newly added controls, disappeared controls, and controls with changed attributes, and obtain the actual control difference. The predefined expected state assertions in the test cases are matched item by item with the actual control differences. The interface state matching degree is determined based on the proportion of successfully matched assertions and the changes of controls in the actual control differences that exceed the expected state assertions.

5. The method according to claim 1, characterized in that: The current state also includes vehicle configuration, scene label, dialect category, and multi-source confidence vector; In the action space, the actions for generating test cases include instruction text templates, noise configuration parameters, and dialect types; the actions for adjusting matching thresholds include semantic similarity thresholds and intent matching thresholds; and the actions for selecting optimization strategies for the vehicle-mounted voice model include prompt word updates, low-rank adaptation fine-tuning, and direct preference optimization sample generation.

6. The method according to claim 1, characterized in that: The reinforcement learning policy network includes a shared backbone network and multiple action heads. The shared backbone network takes the current state as input and outputs a feature vector. Each action head corresponds to a type of action in the action space and outputs the probability distribution of the corresponding action based on the feature vector. Among them, the action head that generates test cases and the action head that selects optimization strategies output the probability distribution of discrete actions, and the action head that adjusts the matching threshold outputs the probability distribution of continuous actions. The step of updating the parameters of the reinforcement learning policy network based on the trajectory data of each of the multiple rounds of voice interaction includes: using a proximal policy optimization algorithm to calculate the advantage estimate and importance sampling ratio based on the trajectory data, and updating the parameters of the reinforcement learning policy network by pruning the objective function.

7. The method according to claim 1, characterized in that, The step of inputting the information of the assessed weaknesses into a large language model to generate adversarial test instruction text targeting the assessed weaknesses includes: Obtain the context information corresponding to the evaluation weakness, the context information including at least one of the following: the pattern of low reward samples under the evaluation weakness, historical failure cases, and the voice assistant capability boundary of the current vehicle model; The context information and action parameters are assembled into prompt words and submitted to the large language model. The action parameters include target intent category, scene label, dialect type and noise configuration. The text output by the large language model is subjected to structured parsing and verification. The verification includes field integrity verification, text compliance verification, and correlation verification between the adversarial test instruction text and the evaluation weakness. When the verification fails, the error message is appended to the prompt word and the large language model is re-requested for generation, or after the retry fails, a template instruction matching the evaluation weakness is selected from the preset adversarial instruction template library as the adversarial test instruction text. The step of synthesizing test speech based on the adversarial test instruction text includes: selecting a corresponding speech synthesis speaker configuration according to the dialect type, synthesizing speech through a speech synthesis engine, and mixing the synthesized speech with a noise signal selected from a noise library according to the noise configuration according to the signal-to-noise ratio to obtain the test speech.

8. The method according to claim 1, characterized in that: The step of constructing a training sample set based on the comprehensive reward of each round of voice interaction includes: taking the interaction samples with a comprehensive reward higher than a first preset reward threshold as positive examples and the interaction samples with a comprehensive reward lower than a second preset reward threshold as negative examples, and constructing a direct preference optimization training sample set. The optimization product includes at least one of the following: prompt word update package, low-rank adaptation fine-tuning weights of vehicle voice model, and update parameters of multi-source interface fusion index model. The step of pushing the optimized product to the vehicle-mounted application includes: pushing the optimized product to the vehicle-mounted application via the OTA channel.

9. A voice interaction testing optimization system based on reinforcement learning, characterized in that, include: The data acquisition module is used to collect multi-source interactive data generated during the voice interaction process of the vehicle system. The multi-source interactive data includes the text corresponding to the user's voice input, the command text and voice response text output by the vehicle system, interface state change data, bus execution signals, and response delay. The semantic evaluation module includes a dual-tower semantic evaluation model, which is used to input the text corresponding to the user's voice input into the first tower of the dual-tower semantic evaluation model, and to input the command text and voice response text output by the vehicle system into the second tower of the dual-tower semantic evaluation model that shares parameters with the first tower. The first tower and the second tower respectively extract semantic vectors and calculate semantic similarity, and determine the semantic similarity as semantic reward. The reward fusion module is used to determine the interface state matching degree based on the interface state change data and the bus signal matching degree based on the bus execution signal. It then performs a weighted summation of the semantic reward, the interface state matching degree, and the bus signal matching degree, and deducts the penalty term corresponding to the response delay to obtain a comprehensive reward. The strategy decision module includes a reinforcement learning policy network, which is used to input the current state, including at least the historical reward distribution determined based on the comprehensive reward of each round of voice interaction, into the reinforcement learning policy network. The reinforcement learning policy network outputs the probability distribution of each candidate action in the action space and selects an action based on the probability distribution. The action space includes optimization strategies for generating test cases, adjusting matching thresholds, and selecting the vehicle voice model. The test case generation module is used to determine the evaluation weaknesses based on the intent categories of interaction samples whose comprehensive reward is lower than a preset reward threshold in each round of voice interaction when the selected action is to generate test cases. It then uses a large language model to generate adversarial test instruction text targeting the evaluation weaknesses and synthesizes test speech based on the adversarial test instruction text to send to the vehicle system for execution. The model feedback module is used to construct a training sample set based on the comprehensive reward of each round of voice interaction, generate an optimized product of the vehicle voice model based on the training sample set, and push the optimized product to the vehicle application.

10. A car infotainment system, characterized in that, include: The voice interaction device is used to perform voice recognition and semantic understanding on the user's voice input, output the vehicle system command text and voice response text, and execute the corresponding control commands; The data acquisition device is used to collect data on the interface status changes, bus execution signals, and response delays of the vehicle's infotainment system, and upload them to the cloud server. The local evaluation device is equipped with a vehicle-mounted evaluation model obtained by distillation of a dual-tower semantic evaluation model deployed in the cloud. It is used to perform local voice interaction evaluation through the vehicle-mounted evaluation model when cloud evaluation is unavailable, based on the network quality monitoring results. A model update device is used to receive and apply the pushed optimization products, wherein the optimization products include at least one of the following: prompt word update package, low-rank adaptation fine-tuning weights of the vehicle voice model, and update parameters of the multi-source interface fusion index model.