Vehicle voice interaction test method and related device

Abstract

The application discloses a vehicle-mounted voice interaction test method and related equipment, and relates to the technical field of vehicle testing. The method comprises the following steps: a preset large language model is used to analyze and process a knowledge graph in the field of vehicle-mounted voice, so as to obtain a test case; a test text corpus is subjected to voice synthesis processing, so as to obtain instruction voice; the instruction voice is sent to a vehicle-mounted voice system to be tested, and response voice returned by the vehicle-mounted voice system to be tested in response to the instruction voice is acquired; the response voice is subjected to voice analysis processing, so as to obtain response text; a preset large language model is used to perform semantic verification on an expected response result and the response text, so as to obtain a test result for the vehicle-mounted voice system to be tested. The application drives test case generation through a large language model, and combines voice synthesis, voice analysis and a semantic level verification mechanism, so as to construct an automatic closed-loop test process based on real voice interaction, thereby improving the efficiency of vehicle-mounted voice system testing and the accuracy of semantic evaluation.

Description

Technical Field

[0001] This application relates to the field of vehicle testing technology, and more specifically, to an in-vehicle voice interaction testing method and related equipment. Background Technology

[0002] With the continuous development of intelligent connected vehicle technology, in-vehicle voice interaction systems have become an important human-machine interaction method to enhance the driving experience and driving safety, and are widely used in various in-vehicle functional scenarios such as navigation control, entertainment playback, air conditioning adjustment, and telephone communication. As an important component of the in-vehicle intelligent cockpit system, the recognition accuracy, semantic understanding ability, and response rationality of the voice interaction system directly affect the user experience and product competitiveness. Therefore, how to conduct comprehensive, accurate, and efficient testing of in-vehicle voice systems has become a key aspect of the research and development process of in-vehicle intelligent systems.

[0003] In related technologies, testing methods for in-vehicle voice interaction systems typically rely on manually constructed test cases, verified through human voice input or recording playback. After testing, the system response results are then manually compared to determine if they meet expectations. This approach is not only inefficient but also highly dependent on the experience of testers, making test results susceptible to subjective judgment and difficult to implement on a large scale and in a standardized manner. Furthermore, in scenarios with complex semantic expressions or diverse voice inputs, manually constructed test texts have limited coverage and cannot fully reflect the voice expression habits of real users. In other words, related technologies suffer from low testing efficiency, insufficient coverage, and low evaluation accuracy in in-vehicle voice interaction systems. Summary of the Invention

[0004] In the summary section of this application, the relevant technical solutions are described in general terms, and a series of simplified concepts are introduced. These concepts will be further elaborated in the detailed embodiments section. This summary section should not be construed as limiting the key or essential technical features of the claimed solutions, nor is it intended to limit the scope of protection of the claimed solutions.

[0005] The in-vehicle voice interaction testing method and related equipment provided in this application can drive test case generation through a large language model, and combine speech synthesis, speech parsing and semantic level verification mechanisms to build an automated closed-loop testing process based on real voice interaction, thereby improving the efficiency of in-vehicle voice system testing and the accuracy of semantic evaluation.

[0006] Firstly, this application provides a method for testing in-vehicle voice interaction, comprising: parsing and processing a knowledge graph in the field of in-vehicle voice using a preset large language model to obtain test cases, wherein the test cases include test text corpus and expected response results; performing speech synthesis processing on the test text corpus to obtain instruction speech; sending the instruction speech to an in-vehicle voice system under test and obtaining the response speech returned by the in-vehicle voice system under test in response to the instruction speech; performing speech parsing processing on the response speech to obtain response text; and performing semantic verification on the expected response results and the response text using the preset large language model to obtain test results for the in-vehicle voice system under test.

[0007] In some implementations, the step of parsing and processing the knowledge graph of the in-vehicle voice domain using a preset large language model to obtain test cases includes: generating a first prompt word based on the knowledge graph, interaction style, and interaction complexity; inputting the first prompt word into the preset large language model so that the preset large language model outputs the test text corpus that conforms to the interaction style and the interaction complexity; generating a second prompt word based on the test text corpus; inputting the second prompt word into the preset large language model so that the preset large language model outputs the expected response result corresponding to the test text corpus; generating a third prompt word based on a preset test case arrangement template, the test text corpus, and the expected response result; and inputting the third prompt word into the preset large language model so that the preset large language model outputs the test cases.

[0008] In some implementations, the interaction style includes at least one of an imperative expression style, an interrogative expression style, and a casual expression style; the interaction complexity includes at least one of a first-level complexity, a second-level complexity, and a third-level complexity, wherein the first-level complexity represents the complexity of interactive text containing a single user intent, the second-level complexity represents the complexity of interactive text containing multiple related user intents, and the third-level complexity represents the complexity of interactive text containing logical connectives and preset conditional clauses.

[0009] In some embodiments, before performing speech synthesis processing on the test text corpus to obtain the command speech, the in-vehicle voice interaction testing method further includes: obtaining speech synthesis configuration parameters, wherein the speech synthesis configuration parameters include at least one of language, speech rate, timbre, and background noise type; the step of performing speech synthesis processing on the test text corpus to obtain the command speech includes: inputting the speech synthesis configuration parameters and the test text corpus into a preset speech synthesis model, and performing speech synthesis processing on the test text corpus through the preset speech synthesis model to obtain the command speech.

[0010] In some implementations, before sending the command voice to the vehicle voice system under test, the vehicle voice interaction testing method further includes: obtaining a preset wake-up word associated with the vehicle voice system under test; generating a wake-up voice based on the preset wake-up word; and sending the wake-up voice to the vehicle voice system under test to wake up the vehicle voice system under test.

[0011] In some implementations, obtaining the response voice returned by the vehicle-mounted voice system under test includes: detecting the audio stream from the vehicle-mounted voice system under test; if the signal energy of the audio stream is detected to be greater than a first energy threshold, and the duration of the state greater than the first energy threshold is greater than a first duration threshold, then determining a start node from the audio stream; if the signal energy of the audio stream is detected to be less than or equal to a second energy threshold, and the duration of the state less than or equal to the second energy threshold is greater than a second duration threshold, then determining an end node from the audio stream; and extracting the response voice from the audio stream based on the start node and the end node.

[0012] In some embodiments, the step of performing speech parsing processing on the response speech to obtain response text includes: performing speech recognition on the response speech to obtain target recognition text and recognition confidence; and encapsulating the target recognition text, the recognition confidence, and the speech duration of the response speech based on a preset structured encapsulation format to obtain the response text.

[0013] In some embodiments, the step of performing speech recognition on the response speech to obtain the target recognition text and recognition confidence score includes: performing speech recognition on the response speech using a preset speech recognition model to obtain the initial recognition text and the recognition confidence score; if the recognition confidence score is less than a preset confidence threshold, then correcting the initial recognition text based on the test text corpus to obtain the target recognition text; if the recognition confidence score is greater than or equal to the preset confidence threshold, then determining the initial recognition text as the target recognition text.

[0014] In some implementations, the step of semantically validating the expected response result and the response text using the preset large language model to obtain test results for the vehicle voice system under test includes: structurally encapsulating the expected response result and the response text based on a preset prompt word template to obtain semantic verification prompt words; inputting the semantic verification prompt words into the preset large language model to obtain the semantic matching judgment result output by the preset large language model; and generating the test results based on the semantic matching judgment result and the acquisition time information of the response voice, wherein the test results include wake-up success rate, command recognition accuracy, semantic matching pass rate determined according to the semantic matching judgment result, and response delay determined according to the acquisition time information.

[0015] In some implementations, the test results also include the accuracy of voice region recognition; the method further includes: obtaining the acoustic parameter configuration corresponding to the test voice region identifier; sending the command voice to the vehicle voice system under test includes: sending the command voice to the vehicle voice system under test based on the acoustic parameter configuration; and structurally encapsulating the expected response result and the response text based on a preset prompt word template to obtain semantic verification prompt words includes: combining and encapsulating the test text corpus, the expected response result, the response text, and the test voice region identifier based on the preset prompt word template to obtain the semantic verification prompt words.

[0016] In some implementations, before performing semantic reasoning on the knowledge graph of the in-vehicle voice domain using a preset large language model to obtain test cases, the in-vehicle voice interaction testing method further includes: obtaining the voice interaction specification document and functional requirement document of the target vehicle model; parsing the voice interaction specification document and the functional requirement document to obtain the knowledge graph, wherein the knowledge graph includes vehicle function nodes, interaction logic, and scene association relationships.

[0017] Secondly, this application also provides an in-vehicle voice interaction testing device, comprising: a test case acquisition unit, used to parse and process a knowledge graph in the field of in-vehicle voice through a preset large language model to obtain test cases, wherein the test cases include test text corpus and expected response results; a speech generation unit, used to perform speech synthesis processing on the test text corpus to obtain instruction speech; a feedback acquisition unit, used to send the instruction speech to the in-vehicle voice system under test and obtain the response speech returned by the in-vehicle voice system under test; a speech parsing unit, used to perform speech parsing processing on the response speech to obtain response text; and a test verification unit, used to perform semantic verification on the expected response results and the response text through the preset large language model to obtain test results for the in-vehicle voice system under test.

[0018] Thirdly, this application also provides an electronic device, including: a memory and a processor, wherein the processor is configured to execute a computer program stored in the memory to implement the steps of the in-vehicle voice interaction testing method described in the first aspect.

[0019] Fourthly, this application also provides a computer-readable storage medium storing computer-executable instructions or a computer program, wherein when the computer-executable instructions or the computer program are executed by a processor, the steps of the in-vehicle voice interaction testing method described in the first aspect are implemented.

[0020] Fifthly, this application also provides a computer program product, including a computer program or computer executable instructions, wherein when the computer program or computer executable instructions are executed by a processor, the steps of the in-vehicle voice interaction testing method provided in the embodiments of this application are implemented.

[0021] In summary, this application uses a pre-defined large language model to analyze and process the knowledge graph in the field of in-vehicle voice, automatically generating test cases containing test text corpora and expected response results. This achieves intelligent construction of test content. Compared to relying on manually written test cases, it can generate complete and systematic test data based on structured domain knowledge, improving test case generation efficiency and enhancing the systematicity and completeness of test content. By performing speech synthesis processing on the test text corpora to generate command speech and sending it to the in-vehicle voice system under test, the testing process is based on real voice input, which is closer to the actual use scenario of the in-vehicle voice system and avoids the bias caused by testing based solely on text interface, thereby improving the realism of the test environment. The method enhances the reference value of test results. After obtaining the response speech returned by the system under test, speech parsing is performed on the response speech to obtain the response text, realizing a complete closed-loop verification from speech input to speech output. This ensures that the test basis comes from the real speech output results of the in-vehicle voice system, rather than internal data interfaces, thereby enhancing the consistency between test results and actual user interaction experience. A large language model is used to perform semantic verification on the expected response results and response text, achieving test result judgment based on semantic understanding. It does not rely on simple keyword matching, but judges whether the response content meets expectations from a semantic level, improving the accuracy and rationality of test judgments and enhancing the evaluation effect of the semantic understanding capability of the in-vehicle voice system. In summary, the in-vehicle voice interaction testing method provided in this application drives test case generation through a large language model and combines speech synthesis, speech parsing, and semantic-level verification mechanisms to construct an automated closed-loop testing process based on real speech interaction, thereby improving the efficiency of in-vehicle voice system testing and the accuracy of semantic evaluation. Attached Figure Description

[0022] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit this specification. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart illustrating an in-vehicle voice interaction testing method provided in an embodiment of this application; Figure 2 This application provides a schematic diagram of the composition structure of an in-vehicle voice interaction testing device. Figure 3 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] The terms used in the specification, claims, and drawings of this application, such as "first," "second," "third," "fourth," etc. (if any), are used to distinguish similar objects and not to describe a specific order or sequence. Therefore, it is to be understood that these terms can be used interchangeably where appropriate, allowing the described embodiments to be used in different orders, unless specifically required by the illustrations or description. Furthermore, the terms "is" and "has," and any variations thereof, are intended to cover, non-exclusively, all possible constituent elements. For example, a process, method, system, product, or apparatus comprising several steps or units is not necessarily limited to the steps or units explicitly listed, but may also include other steps or units not explicitly listed, or steps or units inherent to the process, method, product, or apparatus.

[0024] In this application, a "module" or "unit" refers to a computer program or part of a computer program that has a specific function and works in conjunction with other related parts to achieve a predetermined goal. These modules or units can be implemented by software, hardware (e.g., processing circuitry or memory), or a combination of both. One or more processors or memories can implement one or more modules or units. Furthermore, each module or unit can also be part of a larger module or unit.

[0025] The technical solutions of this application will be described in detail below with reference to the accompanying drawings of the embodiments. It should be noted that the described embodiments are only a part of this application, and not all embodiments. In the following description, the "some embodiments" mentioned are only a subset of all possible embodiments, which may be the same or different subsets, and different embodiments can be combined with each other without conflict.

[0026] Figure 1 This is a flowchart illustrating a vehicle-mounted voice interaction testing method provided in an embodiment of this application. For example, see [link to example]. Figure 1The in-vehicle voice interaction testing method provided in this application embodiment may include the following steps 101 to 105: Step 101: The knowledge graph of the vehicle speech domain is parsed and processed by a preset large language model to obtain test cases, which include test text corpus and expected response results.

[0027] In some examples, the preset large language model is a large language model adapted to the in-vehicle voice interaction testing scenario. This model has semantic understanding and knowledge reasoning capabilities in the field of in-vehicle voice, and can complete tasks related to parsing and processing the knowledge graph in the field of in-vehicle voice and generating test cases. A general large language model can be selected as the basic model, and the basic model can be fine-tuned and trained using corpus data in the field of in-vehicle voice interaction, in-vehicle voice test scenario data, and in-vehicle function interaction rule data, so that the model can be adapted to the specific needs of in-vehicle voice interaction testing. The knowledge graph in the field of in-vehicle voice is a structured graph that stores knowledge related to in-vehicle voice interaction. It serves as the data foundation for parsing and processing using a pre-set large language model. It can be obtained by structuring and associating various types of data, such as industry-standard specifications for in-vehicle voice interaction, vehicle voice function descriptions, real data from actual in-vehicle voice interaction scenarios, and operating rules for various in-vehicle functions. For example, a structured graph covering in-vehicle function modules such as navigation, entertainment, air conditioning, telephone, and vehicle control, and associating the voice interaction rules of each module under different scenarios such as vehicle driving, idling, and parking, is the knowledge graph in the field of in-vehicle voice of this application.

[0028] Parsing and processing is a collective term for a series of data processing operations performed by the pre-set large language model on the knowledge graph of the in-vehicle voice domain. It is the core link connecting the knowledge graph and test cases. This operation is guided by the needs of in-vehicle voice interaction testing and is completed by relying on the model's semantic understanding and knowledge reasoning capabilities. The pre-set large language model can call its own built-in core capabilities such as semantic analysis, knowledge extraction, and logical reasoning to perform layer-by-layer parsing and in-depth processing of the structured data in the knowledge graph of the in-vehicle voice domain. Test cases are standardized test units used to perform voice interaction testing on the in-vehicle voice system under test. They are the direct output of a pre-set large language model after parsing and processing the knowledge graph of the in-vehicle voice domain. They consist of test text corpus and expected response results, and can be directly used for the entire process of subsequent in-vehicle voice interaction testing. Alternatively, after parsing and processing by the pre-set large language model, the generated test text corpus and the corresponding expected response results can be integrated to form standardized test units, which are test cases. For example, a standardized unit generated for the in-vehicle air conditioning temperature control function, containing the corresponding test text corpus and the standard response content of the vehicle system, is an in-vehicle voice interaction test case for the air conditioning temperature control function.

[0029] Test text corpora are a component of test cases. They are text-based test commands that can be converted into spoken test instructions. This corpus is designed for various functional modules and actual interaction scenarios of the in-vehicle voice system and serves as the material for subsequent speech synthesis. For example, texts generated for in-vehicle navigation functions such as "Navigate to a nearby gas station," "Cancel the current navigation route," and "Check traffic congestion ahead" can all be used as test text corpora. Expected response results are also a component of test cases. They are standardized response contents that correspond one-to-one with the test text corpora. These results represent the standard response that the in-vehicle voice system under test should make upon receiving the corresponding test command, conforming to functional logic and interaction specifications. A pre-set large language model, combined with the functional interaction logic and response specifications of the in-vehicle voice system's knowledge graph, can be used to match the corresponding standard response content to the generated test text corpora. For example, the expected response result for the test text corpus "Turn on the air conditioning" is "The air conditioning has been turned on for you," and the expected response result for the test text corpus "Navigate to a nearby gas station" is "Nearby gas stations have been found for you. Do you want to start navigation?"

[0030] It should be noted that the preset large language model in this application embodiment can be based on a general large language model with core capabilities of Natural Language Understanding (NLU), Natural Language Generation (NLG), and knowledge reasoning. The general large language model can be a generative pre-trained Transformer (GPT) series model or a large language model meta-model. This AI (LLaMA) series of models, DeepSeek models, and other technologies, after being specifically fine-tuned for the particular scenarios of in-vehicle voice interaction testing, have resulted in a scenario-adapted large language model. This model serves as the core scheduling and intelligent processing unit throughout the entire in-vehicle voice interaction testing process, undertaking core tasks such as test case generation and semantic verification. The basic training of this model is based on a general large language model. During the training phase, it selects in-vehicle voice-specific corpora as training data, specifically including interaction rule data for in-vehicle navigation, air conditioning, and entertainment functions, in-vehicle voice test scenario design data, real user in-vehicle voice interaction corpora, and in-vehicle voice test case design sample data. An incremental fine-tuning training method is used, setting parameters such as training batches and learning rates to adapt to the characteristics of the in-vehicle corpora. This targeted training allows the model to learn professional knowledge of the in-vehicle voice domain and the specific processing logic for in-vehicle voice interaction test scenarios. After training, the model is deployed to the in-vehicle voice interaction testing system. This module supports real-time invocation according to the test business process. In the application of in-vehicle voice interaction testing scenarios, the input and output data of this model are precisely and intrinsically linked to each stage of the test. In the test case generation stage, the model input is a knowledge graph in the field of in-vehicle voice and test guidance prompts in a preset format. The output is a standardized test case containing test text corpus and expected response results, which directly connects to the subsequent speech synthesis processing stage. In the semantic verification stage, the model input is a structured and encapsulated semantic verification prompt, which integrates in-vehicle test-specific data such as test text corpus, expected response results, response text, and test voice zone identifiers. The output is a semantic matching judgment result, which directly connects to the test result statistics stage. Moreover, all input data of the model is standardized according to the text format preset by the test system, and the output data all meet the processing specifications of the subsequent test stages, realizing the deep integration of the model with the specific scenario of in-vehicle voice interaction testing and the seamless connection of each test stage.

[0031] By implementing step 101, the knowledge graph of the vehicle voice domain is parsed using a pre-set large language model, and test cases containing test text corpus and expected response results are generated. This transforms the test content construction from manual writing to an automatic generation method based on structured knowledge, which can improve the efficiency of test case generation. At the same time, it ensures that there is a clear correspondence between the test text and the expected response, enhances the systematicness and completeness of the test cases, thereby increasing the test coverage and reducing human omissions.

[0032] Step 102: Perform speech synthesis processing on the test text corpus to obtain the instruction speech.

[0033] In some examples, speech synthesis processing is a specialized data processing operation that converts test text corpora into speech signals that can be recognized by the in-vehicle voice system under test. It is a crucial step connecting text-based test commands with speech-based test commands. This operation relies on Text-to-Speech (TTS) technology, and the converted speech signal possesses acoustic characteristics that meet the recognition requirements of the in-vehicle voice system. Command speech is the speech-based test command obtained after speech synthesis processing of the test text corpus. It is an audio signal that can be directly sent to the in-vehicle voice system under test, and its acoustic characteristics match the speech recognition specifications of the in-vehicle voice system. For example, after processing the test text corpus "Turn off the car music," the resulting audio signal, "Turn off the car music," which can be recognized by the in-vehicle voice system, is command speech. The corresponding audio signal generated for "Check the remaining fuel level in the vehicle" also belongs to command speech.

[0034] By implementing step 102, the test text corpus is processed by speech synthesis to generate command speech, thus changing the test input format from text to speech. This conforms to the real usage of the in-vehicle voice system, simulates the actual user's voice scenario, improves the authenticity of the test process, avoids the bias caused by testing based solely on text interfaces, and enhances the reference value of the test results.

[0035] Step 103: Send a command voice to the vehicle voice system under test and obtain the response voice returned by the vehicle voice system under test in response to the command voice.

[0036] In some examples, the in-vehicle voice system under test is an in-vehicle voice interaction system that needs to complete the testing and verification of voice interaction functions. This system integrates core functions such as voice recognition, semantic understanding, function execution, and voice feedback, and is deployed in the vehicle's cockpit system to enable in-vehicle users to provide services such as voice control of the vehicle, information query, and entertainment interaction. For example, an in-vehicle voice interaction system that supports voice control of navigation, air conditioning, and entertainment functions in the cockpit of a new energy vehicle, or an in-vehicle voice interaction system that enables basic vehicle control and voice information query in a fuel vehicle, can both be considered in-vehicle voice systems under test.

[0037] Response voice is the voice feedback signal returned by the vehicle voice system under test after receiving a command voice, through internal speech recognition, semantic parsing, and function execution processes, to the test terminal. It is the voice response result of the vehicle voice system under test to the test command. For example, the voice signal "The air conditioner has been turned on for you" returned by the vehicle voice system under test after receiving the command voice "Turn on the air conditioner", and the voice signal "Navigate to a nearby park for you" returned after receiving the command voice "Navigate to a nearby park for you", are both response voices.

[0038] By implementing step 103, a command voice is sent to the in-vehicle voice system under test and its response voice is obtained, thereby verifying the complete interactive link of the in-vehicle voice system. This ensures that the test is based on the system's real voice output results and can reflect the comprehensive performance of modules such as voice recognition, semantic understanding, and voice broadcasting, thus improving the completeness and practical significance of the test.

[0039] Step 104: Perform speech parsing on the response speech to obtain the response text.

[0040] In some examples, speech parsing is a data processing operation that converts the audio response speech into semantically verifiable text data. It is a crucial step connecting the response speech and the response text. This operation relies on Automatic Speech Recognition (ASR) technology and needs to be adapted to the audio signal characteristics of the in-vehicle environment to ensure the accuracy of the speech-to-text conversion. The response text is the textual feedback result obtained after speech parsing of the response speech. It is the textual presentation of the response content of the in-vehicle voice system under test, and its content is consistent with the audio feedback content of the response speech. For example, after processing the response speech "Your in-vehicle music has been turned off," the resulting text content "Your in-vehicle music has been turned off" is the response text. Similarly, the corresponding text content obtained after processing the response speech "No nearby charging station information has been found" also belongs to the response text.

[0041] For example, the response speech obtained in step 103 can first be standardized and normalized according to the input specifications of the automatic speech recognition model to ensure that the audio signal can be effectively parsed by the model. Then, the normalized response speech is input into a pre-deployed automatic speech recognition model adapted to the vehicle scenario to start the speech parsing processing flow. After the automatic speech recognition model completes the entire process of audio feature extraction, speech signal recognition and text data generation, it outputs text data that is consistent with the content of the response speech. This text data is the response text used for subsequent semantic verification.

[0042] By implementing step 104, the response speech is processed to obtain the response text, and the speech output is converted into analyzable text data, which facilitates subsequent standardized comparison and verification. This improves the automation of test result processing, provides a foundation for batch and large-scale testing, and reduces subjective errors caused by manual listening.

[0043] Step 105: Semantic verification of the expected response results and response text is performed using a pre-set large language model to obtain the test results for the vehicle voice system under test.

[0044] In some examples, semantic verification is a semantic-level matching judgment and verification operation performed by a pre-set large language model on the expected response result and the response text. It is an important part of evaluating the response accuracy of the system under test in in-vehicle voice interaction testing. This operation relies on the natural language understanding capability of the pre-set large language model and focuses on the matching of the core semantics and user intent of the two, rather than a simple textual surface consistency judgment. The expected response result and the response text can be simultaneously input into the pre-set large language model, and the model will use its own natural language understanding capability to deeply analyze, compare and judge the core semantics and user intent fit of the two. All relevant analysis and judgment operations performed by the pre-set large language model constitute semantic verification. For example, if the expected response result "The air conditioner has been turned on for you" and the response text "The air conditioner has been successfully turned on" are input into the pre-set large language model, the model will perform a judgment operation on the core semantic consistency. If the expected response result "The route to shopping mall A has been planned for you" and the response text "No relevant address information was found" are input into the pre-set large language model, the model will perform a judgment operation on the core semantic inconsistency. Both of these are semantic verification.

[0045] The test results for the in-vehicle voice system under test are the conclusive results output by the preset large language model after semantic verification. These results evaluate the in-vehicle voice system's response to corresponding commands. They represent the final output of a single test item in in-vehicle voice interaction, directly reflecting the accuracy and effectiveness of the in-vehicle voice system's voice interaction in that test item. After the preset large language model completes semantic verification of the expected response and response text, an evaluation conclusion for the corresponding test item is generated based on the matching conclusion of their core semantics. This evaluation conclusion is the test result for the in-vehicle voice system under test. For example, if the semantic verification determines that the core semantics of the two are consistent, the model outputs the conclusion "Test passed"; if the semantic verification determines that the core semantics of the two are inconsistent, the model outputs the conclusion "Test failed". For example, after verifying the response to the "Turn off in-vehicle music" command, the model outputs the conclusion "This test item responds accurately, test passed," all of which are test results for the in-vehicle voice system under test.

[0046] By implementing step 105, Lily pre-sets a large language model to perform semantic verification on the expected response results and response text, realizing test result judgment based on semantic understanding. It does not rely on simple keyword matching rules, and can identify situations where the expressions are different but the semantics are consistent, improving the accuracy and rationality of the judgment, thereby improving the objectivity and accuracy of the test evaluation of the in-vehicle voice interaction system.

[0047] In summary, this application embodiment analyzes and processes the knowledge graph of the in-vehicle voice domain using a pre-set large language model, automatically generating test cases containing test text corpora and expected response results. This achieves intelligent construction of test content. Compared to relying on manually written test cases, it can generate complete and systematic test data based on structured domain knowledge, improving test case generation efficiency and enhancing the systematicity and completeness of test content. By performing speech synthesis processing on the test text corpora to generate command speech and sending it to the in-vehicle voice system under test, the testing process is based on real voice input, which is closer to the actual use scenario of the in-vehicle voice system and avoids the bias caused by testing based solely on text interfaces, thereby improving the realism of the testing environment. The method enhances the reliability and reference value of test results. After obtaining the response speech returned by the system under test, speech parsing is performed on the response speech to obtain the response text, realizing a complete closed-loop verification from speech input to speech output. This ensures that the test basis comes from the actual speech output results of the in-vehicle voice system, rather than internal data interfaces, thereby enhancing the consistency between test results and actual user interaction experience. A large language model is used to perform semantic verification on the expected response results and response text, achieving test result judgment based on semantic understanding. It does not rely on simple keyword matching, but judges whether the response content meets expectations from a semantic level, improving the accuracy and rationality of test judgments and enhancing the evaluation effect of the semantic understanding capability of the in-vehicle voice system. In summary, the in-vehicle voice interaction testing method provided in this application uses a large language model to drive test case generation and combines speech synthesis, speech parsing, and semantic-level verification mechanisms to construct an automated closed-loop testing process based on real speech interaction, thereby improving the efficiency of in-vehicle voice system testing and the accuracy of semantic evaluation.

[0048] In some embodiments, step 101 may include: generating a first prompt word based on a knowledge graph, interaction style, and interaction complexity; inputting the first prompt word into a preset large language model so that the preset large language model outputs test text corpus that conforms to the interaction style and interaction complexity; generating a second prompt word based on the test text corpus; inputting the second prompt word into the preset large language model so that the preset large language model outputs the expected response result corresponding to the test text corpus; generating a third prompt word based on a preset test case arrangement template, test text corpus, and expected response result; and inputting the third prompt word into the preset large language model so that the preset large language model outputs test cases.

[0049] In some examples, interaction style refers to the language expression form used by users to express needs or commands in in-vehicle voice interaction scenarios. It is an important dimension for generating test text corpora that fit the real interaction habits of in-vehicle users. Its design relies on the interaction language features of real users in the knowledge graph of the in-vehicle voice field. Based on the real user voice interaction scenario data and language expression features stored in the knowledge graph of the in-vehicle voice field, combined with the actual usage habits of in-vehicle voice interaction, interaction styles can be sorted out and summarized. For example, the command-style expression style that directly puts out functional operation needs, the inquiry-style expression style that initiates functional queries in the form of questions, and the chatty expression style without clear functional needs are all typical interaction styles in in-vehicle voice interaction.

[0050] Interaction complexity refers to the level of intent and logical connection of the user's expressed needs in an in-vehicle voice interaction scenario. It is an important dimension for classifying the difficulty of test text corpus design. The classification is based on the complexity of the interaction logic of each function in the knowledge graph of the in-vehicle voice domain. Different levels of interaction complexity can be classified based on the interaction logic of various in-vehicle functions stored in the knowledge graph of the in-vehicle voice domain, the various types of interaction needs of real users, and the scenario coverage requirements of in-vehicle voice testing. For example, simple interaction complexity with only a single function operation intent, medium interaction complexity with multiple related function operation intents, and complex interaction complexity with multiple logically related function needs are all typical interaction complexities in in-vehicle voice interaction.

[0051] The first prompt word is a set of instructional texts constructed to drive a pre-defined large language model to generate test text corpora that conform to a specified interaction style and complexity. This instructional text includes knowledge graph information from the in-vehicle voice domain, interaction style requirements, and interaction complexity requirements. It serves as the input basis for the pre-defined large language model to generate the test text corpora. Core information such as functional nodes, functional interaction logic, and scene relationships can be extracted from the knowledge graph of the in-vehicle voice domain and integrated according to the selected interaction style and complexity requirements, following the standard format of natural language instructions. For example, combining the knowledge graph information of in-vehicle navigation functions, the instructional interaction style, and the interaction complexity of a single intent, the following text is constructed: "Based on the interaction logic and scene association rules of in-vehicle navigation functions, generate test text corpora that conform to the instructional expression style and single functional intent, covering navigation start point setting, destination planning, and route cancellation functions." This text is the first prompt word. After inputting the aforementioned in-vehicle navigation-related first prompt word into the pre-defined large language model, the model will output test text corpora that conform to the instructional style and single intent complexity, such as "Set navigation start point as home," "Plan the optimal route to the high-speed rail station," and "Cancel the current navigation route."

[0052] The second prompt word is an instruction-style text constructed to drive the preset large language model to match the corresponding standardized expected response result to the generated test text corpus. It contains the complete content of the test text corpus and the response specifications of the in-vehicle voice system. It serves as the input basis for the preset large language model to generate the expected response result. The complete content of the test text corpus output by the preset large language model can be extracted and combined with the in-vehicle voice system function response logic and industry interaction specifications stored in the knowledge graph of the in-vehicle voice field. It is then integrated and constructed according to the standard format of natural language instructions. For example, for the test text corpus "turn on the car air conditioner", the following can be constructed: "Based on the interaction response specifications and system feedback rules of the in-vehicle air conditioner function, generate the corresponding standard expected response result of the in-vehicle voice system for the test text corpus 'turn on the car air conditioner'", and this text is the second prompt word. Through the standard input interface of the preset large language model, the constructed second prompt word is input into the preset large language model in the text format supported by the model. After the model parses the test text corpus content and vehicle voice system response specifications in the prompt word, it can output the corresponding standardized expected response result by combining the information of the knowledge graph. For example, after inputting the above-mentioned vehicle air conditioning-related second prompt word into the preset large language model, the model will output the expected response result "The vehicle air conditioning has been turned on for you" which corresponds to the test text corpus.

[0053] The third prompt word is an instruction-style text constructed to drive the preset large language model to integrate the test text corpus and the corresponding expected response results into standardized test cases. It includes a preset test case arrangement template, the complete content of the test text corpus, and the complete content of the expected response results. It serves as the input basis for the preset large language model to generate standardized test cases. The preset test case arrangement template for in-vehicle voice interaction testing can be retrieved, and the test text corpus and the corresponding expected response results can be completely integrated into the specified content nodes of the template. Combined with the standardized generation requirements of test cases, it is integrated and constructed according to the standard format of natural language instructions. For example, for the test text corpus "turn on the car air conditioner" and the expected response result "the car air conditioner has been turned on for you", the following text is constructed: "According to the preset arrangement template of in-vehicle voice interaction test cases, the test text corpus 'turn on the car air conditioner' and the corresponding expected response result 'the car air conditioner has been turned on for you' are integrated into standardized in-vehicle voice interaction test cases". This text is the third prompt word. The constructed third prompt words can be input into the preset large language model through the standard input interface, according to the text format supported by the model. After the model parses the preset test case arrangement template requirements, test text corpus and expected response results in the prompt words, it integrates the two according to the template specifications and outputs standardized test cases. For example, after inputting the above-mentioned third prompt words related to vehicle air conditioning into the preset large language model, the model will output standardized test cases for vehicle air conditioning function voice interaction that include the test text corpus and expected response results.

[0054] For example, firstly, the interaction logic and scenario-related information of the target function can be extracted from the knowledge graph of the in-vehicle voice domain. A first prompt word is constructed by combining the selected interaction style and complexity, and then input into a preset large language model to generate a test text corpus that meets the requirements. Next, based on the generated test text corpus and the in-vehicle voice system response specifications in the knowledge graph, a second prompt word is constructed and input into the model to obtain standardized expected response results that correspond one-to-one with the test text corpus. Then, a preset test case arrangement template for in-vehicle voice interaction testing is retrieved, and the test text corpus and corresponding expected response results are integrated into the template to construct a third prompt word, which is then input into the preset large language model. The preset large language model deeply analyzes the template requirements and content information in the third prompt word, integrates the test text corpus and expected response results according to the template's standardized format, and finally outputs standardized test cases that can be directly used for in-vehicle voice interaction testing.

[0055] By implementing the above embodiments, prompt words are constructed in stages and a large language model is driven to generate test text corpora, expected response results, and final test cases respectively. This achieves the structuring and automated arrangement of test case content, enabling the rapid batch generation of systematic test data during vehicle testing, reducing the workload of manual sorting and combination. At the same time, it ensures the logical consistency between test text and expected response, enhances the integrity of test cases, thereby improving test construction efficiency and expanding test coverage.

[0056] In some embodiments, the aforementioned interaction style may include at least one of the following: imperative expression style, interrogative expression style, and casual expression style; the interaction complexity may include at least one of the following: first-level complexity, second-level complexity, and third-level complexity. The first-level complexity is used to represent the interaction text complexity containing a single user intent, the second-level complexity is used to represent the interaction text complexity containing multiple related user intents, and the third-level complexity is used to represent the interaction text complexity containing logical connectives and preset conditional clauses.

[0057] In some examples, the imperative expression style is the typical language expression form in in-vehicle voice interaction scenarios where users directly issue functional operation commands to the in-vehicle voice system. Its expression is concise and direct, with a clear functional intent, and no unnecessary questions or casual conversational content. It is the most commonly used expression style in actual in-vehicle voice interaction. Based on real user functional operation interaction data stored in the knowledge graph of the in-vehicle voice field, we can sort out and summarize the typical language expression characteristics of users directly issuing operation commands to the vehicle system, and thus form standardized rules for defining the imperative expression style. For example, expressions such as "turn on the car air conditioning," "turn up the volume in the car," "plan the navigation route to the train station," and "close the car windows" all belong to the imperative expression style.

[0058] Inquiry-style expression is a typical form of language expression in in-vehicle voice interaction scenarios where users initiate information queries or function confirmations to the in-vehicle voice system in the form of questions. These expressions contain question attributes and aim to obtain specific information, requiring the in-vehicle voice system to provide corresponding answers or confirmation feedback. Based on real user information query interaction data stored in the knowledge graph of the in-vehicle voice domain, typical language expression characteristics of users initiating interactions in the form of questions can be identified and summarized, thereby forming standardized rules for defining inquiry-style expression. For example, expressions such as "Where is the nearest gas station?", "How much fuel is left in the car?", "How long until we reach our destination?", and "What's the weather like today?" all belong to the inquiry-style expression.

[0059] Casual conversation style refers to a typical form of language expression in in-vehicle voice interaction scenarios where users have no specific intention to perform functional operations or query information, but only engage in casual conversation with the in-vehicle voice system. It lacks specific in-vehicle functional requirements and aims at casual dialogue, primarily testing the in-vehicle voice system's ability to perform natural daily interactions. Based on real user casual dialogue interaction data stored in the knowledge graph of the in-vehicle voice domain, we can summarize the characteristics of casual conversation style where users have no specific functional requirements, and thus form standardized rules for defining casual conversation style. For example, expressions such as "Tell me an interesting little story," "Can you recite tongue twisters?", "Sing a children's song," and "Let's talk about recent hot topics" all belong to the casual conversation style.

[0060] Level 1 complexity refers to the complexity level of in-vehicle voice interaction text that contains only a single user intent. It is the basic level of interaction complexity, with text expression that is singular, pointing only to the operation or query requirement of a single in-vehicle function, without the connection of multiple intents or the restriction of logical conditions. Based on the independent interaction logic of each single in-vehicle function in the knowledge graph of the in-vehicle voice domain, combined with the basic coverage requirements of in-vehicle voice test cases, standardized definition rules for interaction text containing only a single user intent can be formulated. These rules serve as the basis for determining Level 1 complexity. For example, "turn off the in-vehicle music" contains only the single intent of turning off the music, "query the current temperature inside the car" contains only the single intent of querying the temperature, and "turn on the windshield wipers" contains only the single intent of turning on the windshield wipers. Such texts all belong to Level 1 complexity.

[0061] Level 2 complexity refers to the complexity of in-vehicle voice interaction text containing multiple interrelated user intents. This is a medium level of interaction complexity. The multiple user intents in this text are related to in-vehicle functions within actual usage scenarios, requiring the in-vehicle voice system to execute multiple related functional operations sequentially or simultaneously. Standardized rules for defining in-vehicle interaction text containing multiple related user intents can be developed based on the scenario association logic between various in-vehicle functions in the knowledge graph of the in-vehicle voice domain, combined with the scenario coverage requirements of in-vehicle voice test cases. These rules serve as the basis for determining Level 2 complexity. For example, "turn on the air conditioner and lower the interior temperature" contains two related intents: turning on the air conditioner and lowering the temperature; "play pop music and turn up the volume" contains two related intents: playing music and turning up the volume. Text of this type belongs to Level 2 complexity.

[0062] Level 3 complexity refers to the complexity of in-vehicle voice interaction text containing logical connectives and pre-defined conditional clauses. This is the highest level of interaction complexity. The text contains explicit logical connectives and pre-defined conditions, requiring the in-vehicle voice system to first judge these conditions and then execute the corresponding functional operation based on the judgment result. This primarily tests the in-vehicle voice system's logical understanding and conditional execution capabilities. Standardized definition rules for interaction text containing logical connectives and pre-defined conditional clauses can be formulated based on the conditional execution logic of in-vehicle functions in the knowledge graph of the in-vehicle voice domain, combined with the depth coverage requirements of in-vehicle voice test cases. These rules serve as the basis for determining Level 3 complexity. For example, "Turn on the air conditioner if the interior temperature is higher than 30 degrees" contains logical connectives and a temperature pre-defined condition, and "Remind me when the navigation is about to turn" contains logical connectives and a navigation scenario pre-defined condition; both of these texts belong to Level 3 complexity.

[0063] For example, firstly, based on the real interaction data and functional logic stored in the knowledge graph of the in-vehicle voice domain, standardized definition rules for three types of interaction styles and three levels of interaction complexity can be formulated, and these rules can be integrated into the construction process of the first prompt word. Then, according to the scenario coverage and depth testing requirements of in-vehicle voice interaction testing, one or more interaction styles and interaction complexity levels can be selected, and the generation requirements of the test text corpus can be specified in the first prompt word. Next, the constructed first prompt word is input into a preset large language model, which accurately parses the interaction style and complexity definition rules and generation requirements based on natural language understanding capabilities. Finally, the preset large language model combines the in-vehicle functional interaction logic and scenario association relationships in the knowledge graph of the in-vehicle voice domain to generate test text corpus that conforms to different interaction styles and interaction complexity levels, realizing the generation of diversified test text corpus in a hierarchical and categorized manner.

[0064] By implementing the above embodiments, the interaction style and interaction complexity level are limited, enabling the test corpus to cover different expression methods such as imperative, interrogative, and casual conversation, and to cover complex expression scenarios with single intent, multiple intents, and logical conditions. In the actual vehicle use environment, it is closer to the expression habits of real users, which can improve the depth of test scenario coverage, make up for the shortcomings of traditional single-instruction testing methods, and improve the accuracy of the evaluation of complex semantic understanding ability.

[0065] In some embodiments, before step 102, the aforementioned vehicle-mounted voice interaction testing method may further include: obtaining voice synthesis configuration parameters, wherein the voice synthesis configuration parameters may include at least one of language, speech rate, timbre and background noise type; the aforementioned step 102 may include: inputting the voice synthesis configuration parameters and test text corpus into a preset voice synthesis model, and performing voice synthesis processing on the test text corpus through the preset voice synthesis model to obtain instruction voice.

[0066] In some examples, speech synthesis configuration parameters are a set of parameters used to adjust the speech synthesis effect and define the acoustic and environmental features of the synthesized speech. They serve as the basis for a pre-built speech synthesis model to complete customized speech synthesis. Their function is to match the generated command speech with different voice interaction features in in-vehicle scenarios, thereby improving the realism of the test. Based on the scenario coverage requirements of in-vehicle voice interaction testing, corresponding parameter items can be selected from a pre-built speech synthesis parameter library for combination, or the parameter content can be customized according to specific test requirements to form suitable speech synthesis configuration parameters. For example, to simulate a scenario where a male user in a moving vehicle issues commands, the selected parameter set of "Chinese, medium speed, adult male voice, low engine noise" is the speech synthesis configuration parameter. To simulate a scenario where a foreign user in a moving vehicle issues commands, the selected parameter set of "English, fast, adult female voice, no background noise" also belongs to this type of parameter.

[0067] Language, speech rate, timbre, and background noise type are important components of speech synthesis configuration parameters. These parameters define the synthesized characteristics of the command speech from four dimensions: language type, speech rate, voice characteristics, and ambient noise background. Each parameter is set based on real-world in-vehicle voice interaction scenarios. Parameters can be selected from their respective preset sub-parameter libraries. The language sub-parameter library contains various language types supported by the in-vehicle voice system; the speech rate sub-parameter library contains different speech rate levels; the timbre sub-parameter library contains various typical human voice characteristics; and the background noise type sub-parameter library contains various typical ambient noises in in-vehicle scenarios. For example, language options include Chinese, English, and Cantonese; speech rate options include slow, medium, and fast; timbre options include adult male timbre, adult female timbre, and child's voice; and background noise type options include engine noise, wind noise, mild in-vehicle conversation noise, and no background noise.

[0068] The preset speech synthesis model is a pre-trained speech synthesis model specifically adapted to in-vehicle voice interaction testing scenarios. It has the ability to convert test text corpora into speech signals that meet the recognition requirements of in-vehicle voice systems based on the input speech synthesis configuration parameters, enabling customized and scenario-based command speech generation. A general speech synthesis model can be selected as the base model, and the base model can be fine-tuned and trained using real speech data from in-vehicle scenarios, speech feature data under different noise environments, and in-vehicle command speech data of different languages ​​and speech rates, so that the model can be adapted to the customized synthesis requirements of in-vehicle voice testing. For example, after fine-tuning for in-vehicle scenarios, a speech synthesis model can be synthesized to produce a female medium-speed Chinese command speech with wind noise, and a speech synthesis model can be synthesized to produce a male fast English command speech with engine noise after training. Both of these can be used as preset speech synthesis models.

[0069] The preset speech synthesis model in this application is a scenario-adaptive model built on speech synthesis technology and specifically fine-tuned for in-vehicle voice interaction testing scenarios. It serves as the execution unit for converting test text corpora into command speech and wake-up speech. The model incorporates four core functional modules: a text parsing module, an acoustic feature modeling module, a speech signal generation module, and a noise superposition module. These modules are hierarchically connected. The structured text feature data output by the text parsing module is directly input into the acoustic feature modeling module. The acoustic feature data generated by the acoustic feature modeling module is then fed into the speech signal generation module to output a basic speech signal. The basic speech signal can be fed into the noise superposition module as needed to perform scene noise superposition processing. All modules work together to achieve a complete conversion from text to speech signal. This model can be, for example, an end-to-end speech synthesis model (Tacotron series), a deep speech synthesis model (DeepVoice series), or a latent variable-based end-to-end speech synthesis model (Variational Inference with adversarial learning for end-to-end). The model is trained based on general TTS models such as Text-to-Speech (VITS). The training data uses corpora specific to in-vehicle voice interaction scenarios, including in-vehicle command text-to-speech pairings with different languages, speech rates, and timbres; mixed audio data of typical in-vehicle noise and pure speech; and acoustic feature data adapted to the recognition requirements of in-vehicle voice systems. The training employs a transfer learning fine-tuning approach, with a batch size of 32, a learning rate of 5e-5, and 100 epochs. Mel-spectrum distortion loss is used as the loss function. The focus is on optimizing the acoustic feature modeling module and the noise superposition module to ensure that the synthesized speech signal matches the acoustic features of the in-vehicle voice system and accurately simulates various noise environments in in-vehicle scenarios. After training, the model is deployed to in-vehicle voice interaction testing. The system's speech synthesis processing unit supports real-time invocation during the testing process. In practical applications of in-vehicle voice interaction testing scenarios, this model forms a precise intrinsic data association with the testing phase. Upstream, it receives test text corpora or wake-up word texts output from the test case generation phase. The input consists of text data formatted according to the model's preset structured format, as well as speech synthesis configuration parameters. The text data is passed to the text parsing module, and the language, speech rate, and timbre parameters regulate the synthesis logic of the acoustic feature modeling module. The background noise type parameter regulates the noise superposition rules of the noise superposition module. The model output is a command speech or wake-up speech that meets the configuration parameter requirements and can be accurately recognized by the in-vehicle voice system under test. The output speech signal is in the standard in-vehicle audio format and can be directly connected to the audio transmission module of the testing system to send signals to the in-vehicle voice system under test.

[0070] Test text and speech synthesis configuration parameters can be input into the preset speech synthesis model through its standard data input interface, after being formatted according to the model's supported format. Upon receiving the input data, the model performs a series of processing operations, such as text parsing, acoustic feature modeling, and noise superposition, according to the technical logic of speech synthesis. Finally, it outputs a speech signal that meets the parameter requirements, which is the command speech. For example, if the test text "navigate to the nearby shopping mall" and the speech synthesis configuration parameters "Chinese, medium speed, adult female voice, slight wind noise" are input into the preset speech synthesis model simultaneously, the corresponding speech audio signal will be output after the model has synthesized and processed. Similarly, if the test text "turn up the car volume" and the parameters "Chinese, fast, adult male voice, low engine noise" are input into the model, the resulting speech audio signal is the command speech generated by this operation.

[0071] By implementing the above embodiments, speech synthesis configuration parameters such as language, speech rate, timbre, and background noise are introduced, enabling the test speech to simulate different driving environments and different user characteristics, such as high-speed driving noise environments or different speech rate expressions. This can improve the realism and diversity of test scenarios, help expand the environmental coverage, and improve the adaptability and reference value of test results to actual vehicle usage scenarios.

[0072] In some embodiments, before sending the aforementioned command voice to the vehicle voice system under test, the vehicle voice interaction testing method may further include: obtaining a preset wake-up word associated with the vehicle voice system under test; generating a wake-up voice based on the preset wake-up word; and sending the wake-up voice to the vehicle voice system under test to wake up the vehicle voice system under test.

[0073] In some examples, the preset wake-up word is a fixed word or sentence pre-set by the vehicle-mounted voice system to be tested, which is used to trigger the system to switch from the sleep state to the interactive ready state. It is the exclusive wake-up identifier of the vehicle-mounted voice system and has a one-to-one corresponding relationship with the vehicle-mounted voice system to be tested. It can be directly extracted from the development configuration document, function description manual, and system parameter configuration library of the vehicle-mounted voice system to be tested, or the built-in wake-up word configuration information can be retrieved through the dedicated test interface of the vehicle-mounted voice system to be tested. For example, "Hello, Xiaolan" pre-set during the development of a certain vehicle-mounted voice system, "Hi, car" configured in another vehicle-mounted voice system, and "Xiaolan Xiaolan" exclusive to a certain brand of vehicle models can all be the preset wake-up words associated with the corresponding vehicle-mounted voice system to be tested. The wake-up voice is a voice audio signal generated based on the preset wake-up word associated with the vehicle-mounted voice system to be tested and can be accurately recognized by the vehicle-mounted voice system to be tested. It is the voice carrier that triggers the wake-up of the system to be tested. Its voice content is exactly the same as the preset wake-up word, and its acoustic characteristics meet the wake-up recognition requirements of the vehicle-mounted voice system to be tested. The preset wake-up word can be input into the voice synthesis model preset in this application, and the model performs voice synthesis processing on the preset wake-up word according to the acoustic recognition specifications of vehicle-mounted voice wake-up. The voice audio signal output by the model is the wake-up voice. For example, the corresponding voice audio signal generated based on the preset wake-up word "Hello, Xiaolan" and the voice audio signal that meets the vehicle-mounted wake-up recognition requirements synthesized based on "Hi, car" can both be the wake-up voice.

[0074] The generated wake-up voice can be sent to the exclusive voice receiving port of the vehicle-mounted voice system to be tested according to the general communication protocol of vehicle-mounted voice interaction, and wait for the vehicle-mounted voice system to be tested to complete the internal wake-up recognition and state switching, and detect the interactive ready signal feedback by the system to be tested, then the wake-up operation is completed. For example, sending the wake-up voice of "Hello, Xiaolan" to the vehicle-mounted voice system to be tested of the corresponding model to trigger the system to switch from the sleep state to the ready state to receive instructions; sending the wake-up voice of "Xiaolan Xiaolan" to the vehicle-mounted voice system to be tested of the exclusive brand and completing the state switching are all specific implementation examples of this operation.

[0075] Through the implementation of the above embodiments, adding the wake-up voice step before sending the command voice makes the test process consistent with the actual vehicle voice interaction process, can verify the effectiveness of the wake-up link, not only improves the test integrity, but also can count the wake-up success situation, thereby improving the authenticity of the test results and the overall evaluation accuracy.

[0076] In some embodiments, the aforementioned acquisition of the response voice returned by the vehicle voice system under test may include: detecting the audio stream from the vehicle voice system under test; if the detected signal energy of the audio stream is greater than a first energy threshold, and the duration of the state greater than the first energy threshold is greater than a first duration threshold, then determining a start node from the audio stream; if the detected signal energy of the audio stream is less than or equal to a second energy threshold, and the duration of the state less than or equal to the second energy threshold is greater than the second duration threshold, then determining an end node from the audio stream; and extracting the response voice from the audio stream based on the start node and the end node.

[0077] In some examples, the audio stream from the vehicle voice system under test is a continuous audio signal output through the audio output port of the vehicle voice system under test after receiving the command voice. This signal contains both the effective response voice of the vehicle voice system under test and background noise from the vehicle scene, and is the raw audio data collected. A real-time communication connection can be established with the audio output port of the vehicle voice system under test to collect and receive its output continuous audio signal in real time. For example, after receiving the command voice "turn on the air conditioner", the vehicle voice system under test outputs a continuous audio signal containing the effective voice "the air conditioner has been turned on for you" and slight engine noise. After receiving the navigation command, the vehicle voice system under test outputs a continuous audio signal containing route planning feedback voice and wind noise. Both of these are audio streams from the vehicle voice system under test.

[0078] Signal energy is a physical indicator that measures the strength of the audio signal at each time point in an audio stream. Its value is positively correlated with the volume of the audio signal and serves as the basis for determining whether there is a valid response speech in the audio stream and distinguishing valid speech from background noise. The signal energy value corresponding to each time point in the audio stream can be obtained by performing frame-by-frame calculations on the continuously acquired audio stream in real time. For example, the signal energy value corresponding to low noise from a vehicle engine is 25dB, the signal energy value corresponding to the response speech of the vehicle voice system under test is 65dB, and the signal energy value corresponding to slight conversation noise in the vehicle is 30dB. These can all be specific examples of signal energy values.

[0079] The first energy threshold is the critical value of the signal energy in the audio stream that indicates the start of a valid response speech. When the signal energy of the audio stream is higher than this value, it is determined that a valid response speech is suspected to have occurred, rather than weak background noise in the vehicle scenario. It can be calibrated through a large number of vehicle voice test experiments based on various background noise characteristics of the vehicle scenario and the speech output signal characteristics of the vehicle voice system under test. For example, after experimental calibration, the preset first energy threshold is 40dB. Only when the signal energy value of the audio stream is greater than 40dB will it enter the subsequent determination stage of valid speech start.

[0080] The first duration threshold is the critical value for determining the duration of a valid voice response start. It is used to avoid misjudging the start of valid voice due to transient noise in the vehicle. Only when the signal energy is higher than the first energy threshold for a sustained duration is it considered a genuine valid voice start. This threshold can be combined with the sustained characteristics of transient noise in the vehicle scenario. The longest duration of transient noise can be calibrated through a large number of experiments, and this value can be used as the first duration threshold. For example, after experimental calibration, the preset first duration threshold is 50 milliseconds. Only when the audio stream signal energy is greater than 40dB for a sustained duration of more than 50 milliseconds will it be determined that a valid voice is about to start.

[0081] The start node is the specific time point marked as the beginning of a valid response speech in the time dimension of the audio stream. It serves as the basis for subsequently extracting the starting position of the response speech from the audio stream. It can detect the signal energy and duration of the audio stream in real time. When the detected signal energy is greater than a first energy threshold and the duration of this state is greater than a first duration threshold, the start time point of this duration is marked as the start node of the audio stream. For example, if the signal energy of the audio stream reaches 45dB at 1.3 seconds and remains there, and meets the 50-millisecond duration requirement at 1.35 seconds, then the time point of 1.3 seconds is marked as the start node of the audio stream.

[0082] The second energy threshold is the critical value of the signal energy in the audio stream that determines the end of a valid speech response. When the signal energy of the audio stream is lower than or equal to this value, it is determined that the valid speech response is likely to end, and the audio stream returns to the state of in-vehicle background noise. It can be calibrated based on the average signal energy characteristics of background noise in the in-vehicle scenario through a large number of in-vehicle voice test experiments. Its value can be the same as or different from the first energy threshold. For example, after experimental calibration, the preset second energy threshold is 30dB. When the signal energy value of the audio stream drops to 30dB or below, it enters the subsequent determination stage of the end of valid speech.

[0083] The second duration threshold is a critical value for determining the duration of a valid voice response. It is used to avoid misjudging the end of a valid voice response due to brief silences in the voice response of the vehicle under test. Only when the signal energy is lower than or equal to the second energy threshold for a duration exceeding this threshold is the voice response considered to have officially ended. The second duration threshold can be determined by combining the sentence gap characteristics of the voice response of the vehicle under test with a large number of experiments to calibrate the longest duration of the speech gap. For example, after experimental calibration, the preset second duration threshold is 200 milliseconds. Only when the audio stream signal energy is ≤30dB for a duration exceeding 200 milliseconds will the voice response be considered to have officially ended.

[0084] The end node is a specific time point marked on the time dimension of the audio stream as the end of a valid response speech. It serves as the basis for subsequently extracting the end position of the response speech from the audio stream. After marking the start node, the audio stream can be continuously monitored. When the detected signal energy is less than or equal to a second energy threshold and the duration of this state is greater than a second duration threshold, the start time point of this duration is marked as the end node of the audio stream. For example, if the signal energy of the audio stream drops to 28dB at 3.8 seconds and remains there, and meets the 200ms duration requirement at 4.0 seconds, the test system will mark the 3.8-second time point as the end node of the audio stream.

[0085] Based on the marked start and end nodes, the acquired continuous audio stream can be truncated in the time dimension, retaining the audio segment between the start and end nodes and filtering out noise signals outside the two segments. The truncated audio segment is the response voice. For example, if the test system marks the start node of an audio stream as 1.3 seconds and the end node as 3.8 seconds, after truncating the audio stream, the audio segment between 1.3 seconds and 3.8 seconds is the response voice "The air conditioning has been turned on for you. The current temperature is 26 degrees" after removing the vehicle noise.

[0086] By implementing the above embodiments, the audio stream of the vehicle system is subjected to dual judgment of energy threshold and duration threshold, so as to achieve accurate interception of response speech and avoid the influence of environmental noise or instantaneous interference on the test results. In the vehicle test environment, the accuracy of response speech acquisition can be improved, providing reliable data for subsequent recognition and semantic verification, thereby improving the overall evaluation accuracy.

[0087] In some embodiments, step 104 may include: performing speech recognition on the response speech to obtain the target recognition text and recognition confidence; and encapsulating the target recognition text, recognition confidence, and speech duration of the response speech based on a preset structured encapsulation format to obtain the response text.

[0088] In some examples, the target recognized text is the textualized result obtained after automatic speech recognition processing of the response speech. It is the textual presentation of the response speech content and also the textual basis for subsequent semantic verification. Its content is highly consistent with the response speech content output by the in-vehicle voice system under test. The extracted clean response speech can be input into a preset automatic speech recognition model, which performs the entire process of audio feature extraction, speech signal matching, and text conversion. The text data that the model finally outputs and matches the response speech content is the target recognized text. For example, the corresponding text content obtained after recognizing the response speech "The car air conditioning has been turned on for you" and the corresponding text content obtained after recognizing the response speech "The optimal route to the nearby subway station has been planned for you" can both be considered target recognized text.

[0089] Recognition confidence is a quantitative evaluation metric given by the preset automatic speech recognition model on the accuracy of the generated target text. It is the core basis for judging the reliability of speech recognition results. This metric is presented in numerical form, and the higher the value, the higher the matching degree between the recognition result and the response speech. This metric is calculated by the preset automatic speech recognition model while completing speech recognition and generating target text. It combines multiple dimensions such as audio feature matching degree, speech signal similarity, and semantic fit through a built-in confidence calculation algorithm. The final quantitative value obtained is the recognition confidence. For example, the recognition confidence obtained after recognizing the noise-free response speech "turn off the car music" is 98%, and the recognition confidence obtained after recognizing the response speech "check the remaining fuel in the car" with slight wind noise is 93%. These are specific examples of recognition confidence.

[0090] The pre-defined structured encapsulation format is a standardized data format used to integrate three types of data: target recognition text, recognition confidence, and voice duration of the response speech. This format clarifies the presentation and arrangement of each type of data, enabling scattered test data to form a structured whole, facilitating rapid parsing and use by subsequent pre-defined large language models. A standardized data encapsulation format containing the three core data items—target recognition text, recognition confidence, and voice duration—can be developed to meet the subsequent semantic verification requirements of in-vehicle voice interaction testing, and this format can be pre-defined in the data analysis module of the test system. For example, the key-value pair format "{target recognition text: XXX, recognition confidence: XXX, voice duration: XXX seconds}" and the sequence format "target recognition text-XXX|recognition confidence-XXX|voice duration-XXX seconds" can both be pre-defined structured encapsulation formats.

[0091] First, the target recognition text and recognition confidence score can be extracted from the output of the automatic speech recognition model. Then, the specific speech duration can be obtained by calculating the time dimension of the response speech using an audio duration calculation algorithm. Subsequently, the three types of data are accurately filled into the corresponding data items according to the requirements of the preset structured encapsulation format, completing the integration and encapsulation of all data. The final structured data is the response text. For example, extracting the target recognition text "Two charging piles have been found nearby for you", recognition confidence score 95%, and response speech duration 1.7 seconds, and filling it into the preset structured encapsulation format in key-value pair form, we get "{Target recognition text: Two charging piles have been found nearby for you, recognition confidence score: 95%, speech duration: 1.7 seconds}", which is the response text obtained after encapsulation processing.

[0092] Through the implementation of the above embodiments, the response speech is recognized and structured by combining the recognition confidence and speech duration, so that the test results have quantifiable and statistical feature parameters, which facilitates unified analysis and comparison in batch vehicle testing; at the same time, it enhances the standardization of the results and improves the efficiency of test data management.

[0093] In some embodiments, the aforementioned speech recognition of the response speech to obtain the target recognition text and recognition confidence may include: performing speech recognition on the response speech using a preset speech recognition model to obtain the initial recognition text and recognition confidence; if the recognition confidence is less than a preset confidence threshold, then correcting the initial recognition text based on the test text corpus to obtain the target recognition text; if the recognition confidence is greater than or equal to the preset confidence threshold, then determining the initial recognition text as the target recognition text.

[0094] In some examples, the preset speech recognition model is an automatic speech recognition model that has been pre-trained and specifically adapted to in-vehicle voice interaction test scenarios. It has the ability to extract audio features from response speech in in-vehicle scenarios, match speech signals, and generate recognized text, while also outputting the recognition confidence of the corresponding recognition results. A general automatic speech recognition model can be selected as the base model, and the base model can be fine-tuned and trained using in-vehicle system response speech data in in-vehicle scenarios, speech recognition sample data under different noise environments, and acoustic feature data of in-vehicle voice interaction, so that the model can adapt to the speech recognition needs of complex in-vehicle environments. For example, an automatic speech recognition model that can accurately recognize in-vehicle system response speech with low engine noise after fine-tuning in in-vehicle scenarios, and an automatic speech recognition model that can effectively handle in-vehicle system feedback speech under wind noise interference, can both be used as preset speech recognition models.

[0095] The preset speech recognition model in this application embodiment is a scenario-adaptive model built on automatic speech recognition technology and specifically fine-tuned for in-vehicle voice interaction testing scenarios. It is an execution unit that realizes the speech-to-text conversion and outputs the recognition confidence score. The model has four core functional modules: audio feature extraction module, speech signal matching module, confidence score calculation module, and text correction triggering module. These modules are connected in a hierarchical manner. After the audio feature extraction module extracts features from the input audio data, it sends the generated acoustic feature data to the speech signal matching module. After the speech signal matching module completes the matching of speech and text, it synchronously outputs the initial recognized text. The confidence score is calculated based on features and other dimensions, and then transmitted to the text correction triggering module. This module determines the confidence score threshold and triggers the corresponding text processing logic. These modules work together to achieve accurate speech recognition and related data output in in-vehicle scenarios. The model can be implemented using general automatic speech recognition technologies such as deep speech recognition models (DeepSpeech series), self-supervised learning-based speech recognition models (Wav2Vec series), and convolutional Transformer architecture speech recognition models (Conformer series). The Speech Recognition (ASR) model was used as the base model for training. The training data was selected from the corpus specific to the in-vehicle voice interaction scenario, including various response speech audio-text pairing data of the in-vehicle voice system under test, speech recognition sample data with superimposed engine noise, wind noise, in-vehicle conversation noise and other typical in-vehicle noise, and speech sample data that are prone to recognition ambiguity in the in-vehicle scenario. The training adopted the fine-tuning method of transfer learning, with a training batch size of 16, a learning rate of 3e-5, and 80 training iterations. The loss function used was Connectionist Temporal Classification (CTC). During the training process, the focus was on optimizing the noise-resistant feature extraction module of the audio feature extraction module and optimizing the in-vehicle speech semantic matching logic of the speech signal matching module, so that the model could adapt to the speech recognition requirements of the complex noise environment of the in-vehicle and have the ability to accurately output the recognition confidence. After the training was completed, the model was deployed to the speech parsing processing unit of the in-vehicle voice interaction test system to support the real-time call of response speech recognition in the test process.In practical applications of in-vehicle voice interaction testing scenarios, this model forms a precise internal data correlation with each testing stage. Upstream, it receives clean response speech output from the audio processing module of the testing system. The input is single-channel audio data with a 16kHz sampling rate, normalized according to in-vehicle audio standards. The model first extracts Mel-spectrum acoustic features from the response speech using an audio feature extraction module. Then, a speech signal matching module matches the acoustic features with a text library to generate initial recognition text. Simultaneously, a confidence calculation module combines feature matching degree and signal similarity to quantify the recognition confidence. A text correction trigger module compares the recognition confidence with a preset confidence threshold. If the confidence is less than the threshold, it triggers the initial recognition text correction process based on the corresponding test text corpus, outputting the target recognition text after correction. If the confidence is greater than or equal to the threshold, the initial recognition text is directly determined as the target recognition text. Finally, the model outputs the target recognition text and its corresponding recognition confidence. The output data is encapsulated according to the testing system's preset format, allowing direct connection to subsequent response text generation stages based on a preset structured encapsulation format.

[0096] The initial recognized text is the raw text output by the preset speech recognition model after directly performing speech recognition on the response speech. This result has not undergone any correction processing and is the basic text data for speech recognition. Its accuracy can be affected by factors such as vehicle noise and speech signal clarity. You can extract clean response speech from the audio stream, format it according to the input specifications of the preset speech recognition model, and then input it into the model. After the model performs basic operations such as audio feature extraction, speech signal matching, and text conversion, the text data directly output is the initial recognized text. For example, the corresponding text content obtained by directly recognizing the response speech "Route to the high-speed rail station has been planned for you" and the text content obtained by directly recognizing the response speech "Space bar has been opened for you" with slight noise are both initial recognized text.

[0097] The preset confidence threshold is a quantitative critical value used to judge the reliability of the initial recognized text. It serves as the basis for triggering the text correction mechanism. This threshold is presented as a percentage value to distinguish whether the recognition result needs correction. It can be calibrated based on a large amount of in-vehicle speech recognition experimental data, combined with the recognition characteristics of different in-vehicle noise environments and the recognition accuracy requirements of in-vehicle voice interaction tests. The calibrated fixed percentage value is preset in the speech recognition processing module of the test system. For example, if the preset confidence threshold is 90% after experimental calibration, when the recognition confidence is lower than this value, the reliability of the initial recognized text is deemed insufficient, triggering the correction mechanism; when the recognition confidence is higher than or equal to this value, the recognition result is deemed reliable, and no correction is required.

[0098] This system enables precise correction of initial recognition text with a confidence level below a preset threshold. Using the corresponding test text corpus as semantic and content basis, it corrects and completes errors and missing content in the initial recognition text. The process involves first extracting the core semantics, key functional words, and interaction logic of the test text corpus corresponding to the current response speech. Then, this core information is semantically matched and textually compared with the low-confidence initial recognition text to identify erroneous characters and missing content. Finally, corrections are made based on the semantic and lexical information of the test text corpus. The text data that is completed and corrected is the target recognition text. For example, if the corresponding test text is "turn on the car air conditioner", the initial recognition text is "I have opened the empty bar for you" with a recognition confidence of 85%. Based on the test text, the incorrect character "bar" is corrected to "adjust", and the target recognition text "I have opened the air conditioner for you" is obtained. If the corresponding test text is "navigate to the nearby gas station", the initial recognition text is "I have found the nearby gas station for you" with a recognition confidence of 82%. Based on the test text, the missing word "station" is completed, and the target recognition text "I have found the nearby gas station for you" is obtained.

[0099] By implementing the above embodiments, setting a confidence threshold and correcting it in conjunction with test text corpus when the recognition confidence is low reduces the risk of misjudgment caused by recognition errors. This can improve the accuracy of response text in complex vehicle environments, thereby improving the reliability of semantic verification and enhancing the overall test evaluation accuracy.

[0100] In some embodiments, step 105 may include: structurally encapsulating the expected response result and response text based on a preset prompt word template to obtain a semantic verification prompt word; inputting the semantic verification prompt word into a preset large language model to obtain the semantic matching judgment result output by the preset large language model; generating a test result based on the semantic matching judgment result and the acquisition time information of the response speech, wherein the test result may include the wake-up success rate, command recognition accuracy, semantic matching pass rate determined according to the semantic matching judgment result, and the response delay determined according to the acquisition time information.

[0101] In some examples, the preset prompt word template is a standardized instruction template used to structurally encapsulate the expected response result and the response text. This template clarifies the task requirements of semantic verification and the location of the data to be filled, enabling the preset large language model to accurately understand the goal of semantic verification. This template can be combined with the semantic verification requirements of in-vehicle voice interaction testing to formulate a standardized text template containing semantic verification task instructions, expected response result filling positions, and response text filling positions, and preset this template in the prompt word generation module of the test system. For example, the formulated "Please compare and analyze the core semantics of the expected response result: [XXX] and the response text: [XXX], determine whether the two match, and only output the judgment result of matching or not matching", and the formulated "Please determine whether the expected response result: [XXX] and the response text: [XXX] conform to the same in-vehicle voice interaction intent, and directly output matching or not matching", can both be preset prompt word templates.

[0102] Semantic validation prompts are complete instruction texts obtained by precisely filling the expected response and response text into the corresponding positions according to a preset prompt template. This text is the core input data for the preset large language model to perform semantic validation tasks. It contains two sets of data: a clear validation task and data to be validated, and can be directly parsed by the preset large language model. The system can retrieve the preset prompt template from the prompt generation module and fill the corresponding positions in the template with the expected response obtained in step 101 and the response text generated in step 104. After integrating the data and template, the resulting complete instruction text is the semantic validation text. Prompt words; for example, after filling the expected response "Navigation route to the train station has been planned for you" and the response text "{Target recognition text: Route to the train station has been planned for you, recognition confidence: 97%, voice duration: 1.5 seconds}" into the preset template, the following text is obtained: "Please compare and analyze the core semantics of the expected response:

Navigation route to the train station has been planned for you

{Target recognition text: Route to the train station has been planned for you, recognition confidence: 97%, voice duration: 1.5 seconds}

[0103] First, the semantic matching judgment result output by the preset large language model can be extracted. At the same time, the acquisition time information of the response speech can be retrieved from the audio acquisition module and relevant time indicators can be calculated. Then, according to the preset indicator statistical rules, quantitative indicators such as wake-up success rate, command recognition accuracy rate, and semantic matching pass rate can be calculated in combination with the semantic matching judgment result. Finally, the various quantitative indicators are integrated with the qualitative conclusion of semantic matching to form a complete evaluation content, which is the test result. For example, the response delay of 1.8 seconds is calculated by combining the semantic matching judgment result "match" with the response speech acquisition time information. According to the rules, the semantic matching pass rate is 98%, the command recognition accuracy rate is 97%, and the wake-up success rate is 100%. The evaluation content that includes indicators and qualitative conclusions after integrating this information is the test result generated by this operation.

[0104] Four metrics—wake-up success rate, command recognition accuracy, semantic matching pass rate, and response latency—quantify the performance of an in-vehicle voice interaction system from different dimensions. Wake-up success rate is calculated by the ratio of successful wake-ups to the total number of wake-ups during the test. Command recognition accuracy is calculated by the ratio of correct voice recognition results to the total number of tests. Semantic matching pass rate is calculated by the ratio of matching results to the total number of tests. Response latency is calculated by the time difference between sending the command voice and receiving the response voice. For example, in a test with 100 total wake-ups and 99 successful wake-ups, the wake-up success rate is 99%; with 200 total tests and 195 correct recognitions, the command recognition accuracy rate is 97.5%; with 200 total tests and 192 semantic matches, the semantic matching pass rate is 96%; and with the command voice sent at 10:00:00 and the response voice received at 10:00:01.2, the response latency is 1.2 seconds.

[0105] Through the implementation of the above embodiments, the expected response result and the response text are encapsulated in a structured manner and input into a large language model for semantic matching judgment. Multi-dimensional indicators are generated in combination with the response time, so that the test results not only include semantic matching, but also indicators such as wake-up success rate, recognition accuracy and response latency. This enables multi-indicator quantitative evaluation and improves the comprehensiveness and objectivity of the test results.

[0106] In some embodiments, the aforementioned test results may further include the accuracy of voice region recognition; the aforementioned method may further include: obtaining the acoustic parameter configuration corresponding to the test voice region identifier; the aforementioned sending of command voice to the vehicle voice system under test may include: sending command voice to the vehicle voice system under test based on the aforementioned acoustic parameter configuration; the structured encapsulation of the expected response result and response text based on a preset prompt word template to obtain semantic verification prompt words may include: combining and encapsulating the test text corpus, expected response result, response text and test voice region identifier based on a preset prompt word template to obtain semantic verification prompt words.

[0107] In some examples, the voice region recognition accuracy is a quantitative indicator that measures the accuracy of the tested in-vehicle voice system in recognizing and semantically matching commands issued from different voice regions within the vehicle. It is an important new evaluation dimension in the test results, directly reflecting the multi-voice region voice interaction capability of the tested in-vehicle voice system. It can be statistically analyzed separately for each voice region, or the comprehensive accuracy of all voice regions can be calculated. The accuracy of a voice region can be calculated by comparing the number of tests in a specified test voice region where the semantic matching judgment result output by the preset large language model is a match with the total number of tests in that voice region. After integrating the test data of each voice region, the comprehensive voice region recognition accuracy of all voice regions can be calculated. For example, if 100 tests are conducted in the driver's voice region and 98 semantic matching results are a match, the accuracy of the driver's voice region recognition is 98%; if 100 tests are conducted in the passenger's voice region and 92 semantic matching results are a match, the accuracy of the passenger's voice region recognition is 92%; and if 80 tests are conducted in the rear voice region and 75 semantic matching results are a match, the accuracy of the rear voice region recognition is 93.75%.

[0108] Test zone identifiers are standardized and unique identifiers used to distinguish different voice interaction zones within the vehicle. They are the core identifiers for associating zone with corresponding acoustic parameter configurations and calculating the recognition accuracy of each zone. They have a one-to-one correspondence with the actual physical zones within the vehicle. By combining the zone recognition design logic of the in-vehicle voice system, typical physical voice interaction zones within the vehicle can be standardized and coded to generate exclusive test zone identifiers. The association between the identifiers and the corresponding actual zones is preset in the zone management module of the test system. For example, the driver's zone is coded as YQ001, the passenger's zone as YQ002, the rear left zone as YQ003, and the rear right zone as YQ004. YQ001, YQ002, YQ003, and YQ004 are all test zone identifiers.

[0109] Acoustic parameter configuration is a set of parameters that corresponds one-to-one with the test zone markers and is used to simulate the acoustic characteristics of speech propagation in different zones within the vehicle. It is the core basis for matching command speech with the corresponding zone's sound production and propagation characteristics, ensuring that the sent command speech conforms to the actual acoustic characteristics of speech in different locations within the vehicle. Based on the spatial acoustic characteristics, speech propagation laws, and sound wave attenuation characteristics of different physical sound zones within the vehicle, the acoustic parameters for each zone can be calibrated through acoustic simulation experiments and real-vehicle testing. The calibrated parameter set is then matched with the corresponding test zone markers. The acoustic parameters are bound and stored in the acoustic parameter library of the test system for retrieval during the test process. For example, the acoustic parameter configuration corresponding to the driver's side audio zone identifier YQ001 is "sound source distance 0.5 meters, volume 60dB, no sound wave reflection attenuation", the acoustic parameter configuration corresponding to the passenger side audio zone identifier YQ002 is "sound source distance 1.2 meters, volume 58dB, slight sound wave reflection attenuation", and the acoustic parameter configuration corresponding to the rear left side audio zone identifier YQ003 is "sound source distance 1.8 meters, volume 62dB, moderate sound wave reflection attenuation".

[0110] Based on a preset prompt word template, the process of combining and encapsulating test text corpus, expected response results, response text, and test audio region identifiers to obtain semantic verification prompt words is an extension and optimization of the semantic verification prompt word generation process. It adds two new data types—test text corpus and test audio region identifiers—to the original content, completing the structured combination and encapsulation of multi-dimensional data. This generates execution steps containing semantic verification prompt words with audio region information, allowing the semantic verification results of the preset large language model to be associated with specific test audio regions, providing a direct basis for subsequent statistical analysis of the recognition accuracy of each audio region. The preset prompt word template can be retrieved from the prompt word generation module, and the test text corpus, expected response results, response text, and test audio region identifiers corresponding to the current test item can be accurately filled into the corresponding preset prompt words in the template. After filling in the positions and completing the integration and structured encapsulation of all data, the resulting complete instruction text is the semantic verification prompt word. For example, the preset prompt word template is "Please combine the test voice region identifier [XXX], compare the expected response result [XXX] corresponding to the test text corpus [XXX] with the core semantics of the response text [XXX], determine whether the two match, and only output the judgment result of matching or not matching". After filling the test voice region identifier [YQ002], the test text corpus [turn on the car air conditioner], the expected response result [the car air conditioner has been turned on for you], and the response text [{target recognition text: the car air conditioner has been turned on for you, recognition confidence: 95%, voice duration: 1.2 seconds}] into the template, the resulting complete instruction text is the semantic verification prompt word generated by combination and encapsulation.

[0111] By implementing the above embodiments, the accuracy of sound zone recognition and the configuration of acoustic parameters are introduced, enabling the test to cover different in-vehicle sound zones (such as driver's seat, passenger seat, and rear seat) and verify the system's recognition ability in different spatial positions. This can improve the breadth of test coverage, solve the problem that traditional tests are difficult to distinguish the performance differences of sound zones, and improve the level of evaluation refinement.

[0112] In some embodiments, before performing semantic reasoning on the knowledge graph in the vehicle voice domain using a preset large language model to obtain test cases, the aforementioned vehicle voice interaction testing method may further include: obtaining the voice interaction specification document and functional requirement document of the target vehicle model; parsing and processing the voice interaction specification document and functional requirement document to obtain a knowledge graph, wherein the knowledge graph may include vehicle function nodes, interaction logic, and scene association relationships.

[0113] In some examples, the voice interaction specification document is a dedicated voice interaction design specification document for the target vehicle model, developed during the development phase of the in-vehicle voice system under test. It serves as one of the documents used to construct the in-vehicle voice domain knowledge graph for the target vehicle model. This document clarifies the core interaction rules of the target vehicle model's in-vehicle voice system, including the interaction style, wake-up word usage specifications, function response scripts, and semantic matching standards. This document can be retrieved from the target vehicle model's product development technical archives or the in-vehicle voice system developer's dedicated document library, uniquely matching the target vehicle model. For example, the "XX Brand B-Class Vehicle In-Vehicle Voice Interaction Design Specification V1.0" for a certain gasoline vehicle model and the "XX Pure Electric Vehicle Intelligent Voice Interaction Response Specification" for a certain new energy vehicle model are both voice interaction specification documents corresponding to the target vehicle models. These documents will specify the voice response scripts for the air conditioning function, the semantic matching standards for the navigation function, and other content.

[0114] Functional requirements documents are specific functional development requirements specifications for the target vehicle model during the development phase of the in-vehicle voice system under test. They serve as another document basis for constructing the knowledge graph of the in-vehicle voice domain for the target vehicle model. This document clarifies the core functional information of the in-vehicle voice system of the target vehicle model, such as the functional modules supported, the functional implementation logic, the functional triggering conditions, and the relationships between the functions. This document can be retrieved from the product requirements management library of the target vehicle model and the technical archives of the in-vehicle voice system development project to uniquely match the target vehicle model. For example, the "Functional Requirements Specification for In-vehicle Voice System of XX Off-road Vehicle Model" for a certain SUV model and the "Development Requirements Document for Intelligent Voice Function of XX Family Vehicle Model" for a certain sedan model are both functional requirements documents corresponding to the target vehicle models. Such documents will clearly specify the functional modules supported by the in-vehicle voice system, such as navigation, entertainment, and vehicle control, as well as the voice triggering logic of each function.

[0115] This system can perform a series of comprehensive processing operations, including text parsing, knowledge extraction, and structured association, on the voice interaction specification documents and functional requirement documents of a target vehicle model. During implementation, natural language understanding technology can be used to segment the two types of documents, extract core information, and semantically analyze them. Then, relying on knowledge graph construction technology, the extracted core information is structured and integrated according to a "node-relationship" graph construction logic, ultimately forming a knowledge graph in the field of in-vehicle voice. For example, by parsing the voice interaction specification documents and functional requirement documents of a certain vehicle model, information on functions such as navigation and air conditioning can be extracted. The interaction logic of "voice command - function execution - voice response" can be analyzed, and the usage rules of each function in scenarios such as driving, idling, and parking can be associated. This information is then structured and integrated into a graph. This series of operations constitutes the document parsing and processing.

[0116] Vehicle function nodes, interaction logic, and scenario relationships are the three core components of the knowledge graph in the in-vehicle voice domain of the target vehicle model. Together, they constitute a structured knowledge system in the in-vehicle voice domain and serve as the knowledge basis for generating test cases from the pre-set large language model. Corresponding core information can be extracted from the voice interaction specification document and functional requirement document of the target vehicle model using natural language understanding technology. Each functional unit supported by in-vehicle voice is designated as a vehicle function node, the voice triggering, execution, and response rules of each function are designated as interaction logic, and the adaptation rules of each function to different driving scenarios are designated as scenario relationships. For example, vehicle function nodes include navigation, air conditioning, entertainment, telephone, and vehicle control; interaction logic includes "the user issues a voice command 'turn on the air conditioning,' triggering the in-vehicle air conditioning function, and the vehicle system replies according to the specification 'the air conditioning has been turned on for you,'" and "the user issues a voice command 'next song,' triggering the in-vehicle entertainment system's song-skipping function, and the vehicle system does not respond with additional dialogue," etc.; scenario relationships include "in high-speed driving scenarios, the navigation function supports lane-level navigation voice prompts," and "in parking scenarios, the in-vehicle entertainment system supports voice-activated playback of high-definition video," etc.

[0117] For example, firstly, a dedicated voice interaction specification document and functional requirement document that perfectly match the target vehicle model can be retrieved from the product development technical archive of the target vehicle model, ensuring the relevance and accuracy of the document information. Next, natural language understanding technology is used to perform full-text parsing of both types of documents, extracting and organizing core information, separating key content such as vehicle functional units, voice interaction rules, and scenario adaptation requirements. Then, the extracted key content is structured, building vehicle functional nodes around various in-vehicle voice functions, organizing the voice triggering, execution, and response rules corresponding to each node to form interaction logic, and associating the adaptation relationships between each functional node and scenarios such as vehicle driving, idling, and parking to form scenario relationships. Finally, relying on knowledge graph construction technology, the vehicle functional nodes, interaction logic, and scenario relationships are structurally integrated according to the "node-relationship" logic to generate an in-vehicle voice domain knowledge graph adapted to the target vehicle model, providing a dedicated knowledge data foundation for subsequent large-scale language model parsing and processing to generate test cases.

[0118] By implementing the above embodiments, a knowledge graph is constructed by parsing the voice interaction specification document and functional requirement document of the target vehicle model. This ensures that the generated test cases are closely related to the specific vehicle model's functions and interaction logic, enhancing the relevance of the tests. In multi-vehicle testing scenarios, it can quickly adapt to different configurations, improve testing efficiency, and ensure that the test content is consistent with the actual functions of the vehicle, thereby improving the completeness of test coverage and the accuracy of evaluation.

[0119] Furthermore, as an implementation of the aforementioned method embodiments, this application also provides an in-vehicle voice interaction testing device for implementing the aforementioned method embodiments; this device embodiment corresponds to the aforementioned method embodiments. For ease of reading, this in-vehicle voice interaction testing device embodiment will not repeat the details of the aforementioned method embodiments one by one, but it should be clear that the device in this application embodiment can correspondingly implement all the contents of the aforementioned method embodiments. Figure 2As shown, the in-vehicle voice interaction testing device 20 includes: a test case acquisition unit 201, a voice generation unit 202, a feedback acquisition unit 203, a voice parsing unit 204, and a test verification unit 205. The test case acquisition unit 201 is used to parse and process the knowledge graph of the in-vehicle voice domain using a preset large language model to obtain test cases. The test cases may include test text corpus and expected response results. The voice generation unit 202 is used to perform voice synthesis processing on the test text corpus to obtain command voice. The feedback acquisition unit 203 is used to send command voice to the in-vehicle voice system under test and obtain the response voice returned by the in-vehicle voice system under test. The voice parsing unit 204 is used to perform voice parsing processing on the response voice to obtain response text. The test verification unit 205 is used to perform semantic verification on the expected response results and response text using a preset large language model to obtain test results for the in-vehicle voice system under test.

[0120] In some embodiments, the use case acquisition unit 201 is further configured to generate a first prompt word based on a knowledge graph, interaction style, and interaction complexity; input the first prompt word into a preset large language model so that the preset large language model outputs test text corpus that conforms to the interaction style and interaction complexity; generate a second prompt word based on the test text corpus; input the second prompt word into the preset large language model so that the preset large language model outputs the expected response result corresponding to the test text corpus; generate a third prompt word based on a preset use case arrangement template, test text corpus, and expected response result; and input the third prompt word into the preset large language model so that the preset large language model outputs test cases.

[0121] In some embodiments, the interaction style includes at least one of the following: imperative expression style, interrogative expression style, and casual expression style; the interaction complexity includes at least one of the following: first-level complexity, second-level complexity, and third-level complexity. The first-level complexity is used to represent the interaction text complexity containing a single user intent, the second-level complexity is used to represent the interaction text complexity containing multiple related user intents, and the third-level complexity is used to represent the interaction text complexity containing logical connectives and preset conditional clauses.

[0122] In some embodiments, the speech generation unit 202 is further configured to obtain speech synthesis configuration parameters, wherein the speech synthesis configuration parameters include at least one of language, speech rate, timbre and background noise type; and to perform speech synthesis processing on test text corpus to obtain instruction speech, including: inputting the speech synthesis configuration parameters and test text corpus into a preset speech synthesis model, and performing speech synthesis processing on the test text corpus through the preset speech synthesis model to obtain instruction speech.

[0123] In some embodiments, the feedback acquisition unit 203 is further configured to acquire a preset wake-up word associated with the vehicle voice system under test; generate a wake-up voice based on the preset wake-up word; and send the wake-up voice to the vehicle voice system under test to wake up the vehicle voice system under test.

[0124] In some embodiments, the feedback acquisition unit 203 is further configured to detect the audio stream from the vehicle voice system under test; if the detected signal energy of the audio stream is greater than a first energy threshold, and the duration of the state greater than the first energy threshold is greater than a first duration threshold, then a start node is determined from the audio stream; if the detected signal energy of the audio stream is less than or equal to a second energy threshold, and the duration of the state less than or equal to the second energy threshold is greater than a second duration threshold, then an end node is determined from the audio stream; and based on the start node and the end node, the response speech is extracted from the audio stream.

[0125] In some embodiments, the speech parsing unit 204 is further configured to perform speech recognition on the response speech to obtain the target recognition text and recognition confidence; and to encapsulate the target recognition text, recognition confidence, and speech duration of the response speech based on a preset structured encapsulation format to obtain the response text.

[0126] In some embodiments, the speech parsing unit 204 is further configured to perform speech recognition on the response speech using a preset speech recognition model to obtain the initial recognition text and recognition confidence; if the recognition confidence is less than a preset confidence threshold, the initial recognition text is corrected based on the test text corpus to obtain the target recognition text; if the recognition confidence is greater than or equal to the preset confidence threshold, the initial recognition text is determined as the target recognition text.

[0127] In some embodiments, the test verification unit 205 is further configured to: encapsulate the expected response result and the response text in a structured manner based on a preset prompt word template to obtain a semantic verification prompt word; input the semantic verification prompt word into a preset large language model to obtain the semantic matching judgment result output by the preset large language model; and generate test results based on the semantic matching judgment result and the acquisition time information of the response speech, wherein the test results include the wake-up success rate, command recognition accuracy, semantic matching pass rate determined according to the semantic matching judgment result, and the response delay determined according to the acquisition time information.

[0128] In some embodiments, the test results also include the accuracy of the sound region recognition; the feedback acquisition unit 203 is also used to acquire the acoustic parameter configuration corresponding to the test sound region identifier; based on the acoustic parameter configuration, send instruction voice to the vehicle voice system under test; the test verification unit 205 is also used to combine and encapsulate the test text corpus, expected response result, response text and test sound region identifier based on the preset prompt word template to obtain semantic verification prompt words.

[0129] In some embodiments, the use case acquisition unit 201 is further configured to acquire the voice interaction specification document and functional requirement document of the target vehicle model; and to parse and process the voice interaction specification document and functional requirement document to obtain a knowledge graph, wherein the knowledge graph includes vehicle function nodes, interaction logic and scene association relationships.

[0130] This application also provides a computer-readable storage medium storing computer-executable instructions or computer programs, which, when executed by a processor, will cause the processor to perform any step of the in-vehicle voice interaction testing method provided in this application.

[0131] In some embodiments, the computer-readable storage medium may be a random access memory (RAM), a read-only memory (ROM), flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); or it may be a variety of devices that include one or any combination of the above-mentioned memories.

[0132] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0133] In some embodiments, computer-executable instructions may, but do not necessarily, correspond to files in a file system, and may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0134] In some embodiments, computer-executable instructions may be deployed to execute on an electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0135] like Figure 3As shown, this application also provides an electronic device 30, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor. When the processor 320 executes the computer program 311, it implements any step of the above-described vehicle voice interaction test method.

[0136] This application also provides a computer program product comprising a computer program or computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer program or computer-executable instructions from the computer-readable storage medium and executes the computer program or computer-executable instructions, causing the electronic device to perform any step of the in-vehicle voice interaction testing method described above.

[0137] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for testing in-vehicle voice interaction, characterized in that, include: The knowledge graph of the in-vehicle voice domain is parsed and processed by a pre-set large language model to obtain test cases, wherein the test cases include test text corpus and expected response results; The test text corpus is processed by speech synthesis to obtain the instruction speech; Send the command voice to the vehicle voice system under test, and obtain the response voice returned by the vehicle voice system under test in response to the command voice; The response speech is processed by speech parsing to obtain the response text; The expected response result and the response text are semantically verified by the preset large language model to obtain the test result for the vehicle voice system under test.

2. The in-vehicle voice interaction testing method according to claim 1, characterized in that, The process involves parsing and processing the knowledge graph in the vehicle speech domain using a pre-set large language model to obtain test cases, including: The first prompt word is generated based on the knowledge graph, interaction style, and interaction complexity. The first prompt word is input into a preset large language model so that the preset large language model outputs the test text corpus that conforms to the interaction style and the interaction complexity; Generate a second prompt word based on the test text corpus; The second prompt word is input into a preset large language model so that the preset large language model outputs the expected response result corresponding to the test text corpus; A third prompt word is generated based on the preset test case arrangement template, the test text corpus, and the expected response result; The third prompt word is input into a preset large language model so that the preset large language model outputs the test case.

3. The in-vehicle voice interaction testing method according to claim 2, characterized in that, The interaction style includes at least one of the following: imperative expression style, interrogative expression style, and casual expression style; the interaction complexity includes at least one of the following: first-level complexity, second-level complexity, and third-level complexity. The first-level complexity is used to represent the complexity of interactive text containing a single user intent, the second-level complexity is used to represent the complexity of interactive text containing multiple related user intents, and the third-level complexity is used to represent the complexity of interactive text containing logical connectives and preset conditional clauses.

4. The in-vehicle voice interaction testing method according to claim 1, characterized in that, Before performing speech synthesis processing on the test text corpus to obtain the command speech, the in-vehicle voice interaction testing method further includes: Obtain speech synthesis configuration parameters, wherein the speech synthesis configuration parameters include at least one of language, speech rate, timbre, and background noise type; The step of performing speech synthesis processing on the test text corpus to obtain instruction speech includes: The speech synthesis configuration parameters and the test text corpus are input into a preset speech synthesis model. The preset speech synthesis model is used to process the test text corpus to obtain the instruction speech.

5. The in-vehicle voice interaction testing method according to claim 1, characterized in that, Before sending the command voice to the in-vehicle voice system under test, the in-vehicle voice interaction testing method further includes: Obtain the preset wake-up word associated with the vehicle voice system under test; A wake-up voice is generated based on the preset wake-up word and sent to the vehicle voice system under test to wake up the vehicle voice system under test.

6. The in-vehicle voice interaction testing method according to claim 1, characterized in that, The step of acquiring the response voice returned by the vehicle voice system under test includes: The audio stream from the vehicle-mounted voice system under test is detected; If the signal energy of the audio stream is detected to be greater than a first energy threshold, and the duration of the state greater than the first energy threshold is greater than a first duration threshold, then a starting node is determined from the audio stream. If the signal energy of the audio stream is detected to be less than or equal to a second energy threshold, and the duration of the state that is less than or equal to the second energy threshold is greater than a second duration threshold, then an end node is determined from the audio stream. The response speech is extracted from the audio stream based on the start node and the end node.

7. The in-vehicle voice interaction testing method according to claim 1, characterized in that, The step of performing speech parsing processing on the response speech to obtain the response text includes: The response speech is subjected to speech recognition to obtain the target recognition text and recognition confidence level; Based on a preset structured encapsulation format, the target recognition text, the recognition confidence level, and the speech duration of the response speech are encapsulated to obtain the response text.

8. The in-vehicle voice interaction testing method according to claim 1, characterized in that, The step of performing speech recognition on the response speech to obtain the target recognition text and recognition confidence score includes: The response speech is subjected to speech recognition using a preset speech recognition model to obtain the initial recognized text and the recognition confidence level. If the recognition confidence is less than a preset confidence threshold, the initial recognition text is corrected based on the test text corpus to obtain the target recognition text; If the recognition confidence level is greater than or equal to the preset confidence threshold, then the initial recognition text is determined as the target recognition text.

9. The in-vehicle voice interaction testing method according to claim 1, characterized in that, The step of performing semantic verification on the expected response result and the response text using the preset large language model to obtain the test results for the vehicle voice system under test includes: Based on a preset prompt word template, the expected response result and the response text are structurally encapsulated to obtain semantic verification prompt words; The semantic verification prompt is input into the preset large language model to obtain the semantic matching judgment result output by the preset large language model; Based on the semantic matching judgment result and the acquisition time information of the response speech, the test result is generated, wherein the test result includes the wake-up success rate, command recognition accuracy, semantic matching pass rate determined according to the semantic matching judgment result, and the response delay determined according to the acquisition time information.

10. The in-vehicle voice interaction testing method according to claim 9, characterized in that, The test results also include the accuracy of voice region recognition; the method also includes: Obtain the acoustic parameter configuration corresponding to the test zone identifier; Sending the command voice to the vehicle voice system under test includes: Based on the acoustic parameter configuration, the command voice is sent to the vehicle voice system under test; The step of structurally encapsulating the expected response result and the response text based on a preset prompt word template to obtain semantic verification prompt words includes: Based on the preset prompt word template, the test text corpus, the expected response result, the response text, and the test audio region identifier are combined and encapsulated to obtain the semantic verification prompt word.

11. The in-vehicle voice interaction testing method according to any one of claims 1 to 10, characterized in that, Before obtaining test cases by performing semantic reasoning on the knowledge graph of the in-vehicle voice domain through a pre-set large language model, the in-vehicle voice interaction testing method also includes: Obtain the voice interaction specification document and functional requirements document for the target vehicle model; The voice interaction specification document and the functional requirements document are parsed and processed to obtain the knowledge graph, wherein the knowledge graph includes vehicle function nodes, interaction logic and scene association relationships.

12. A vehicle-mounted voice interaction testing device, characterized in that, include: The test case acquisition unit is used to parse and process the knowledge graph in the field of vehicle voice through a preset large language model to obtain test cases, wherein the test cases include test text corpus and expected response results; The speech generation unit is used to perform speech synthesis processing on the test text corpus to obtain instruction speech; The feedback acquisition unit is used to send the instruction voice to the vehicle voice system under test and acquire the response voice returned by the vehicle voice system under test. A speech parsing unit is used to perform speech parsing processing on the response speech to obtain the response text; The test verification unit is used to perform semantic verification on the expected response result and the response text through the preset large language model, so as to obtain the test result for the vehicle voice system under test.

13. An electronic device, comprising: The memory and processor are characterized in that the processor, when executing a computer program stored in the memory, implements the steps of the in-vehicle voice interaction testing method as described in any one of claims 1 to 11.

14. A computer-readable storage medium having stored thereon computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or the computer program are executed by a processor, the steps of the in-vehicle voice interaction testing method as described in any one of claims 1 to 11 are implemented.

15. A computer program product comprising a computer program or computer-executable instructions, characterized in that, When the computer program or computer-executable instructions are executed by the processor, the steps of the in-vehicle voice interaction testing method as described in any one of claims 1 to 11 are implemented.

Images