Voice test method, system, device and medium
By testing the in-vehicle voice interaction model using a multi-level matching method and high-precision speech recognition technology, the problem of insufficient testing accuracy in existing technologies is solved, and the accuracy performance evaluation in complex driving scenarios is realized.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-19
- Publication Date
- 2026-04-03
AI Technical Summary
The accuracy of test results for existing in-vehicle voice interaction models is not good, especially in real driving scenarios such as high-speed noise, multi-dialect commands, and multi-task concurrency, which make it difficult to meet user needs.
A multi-level matching method is used to perform text matching, semantic matching, and intent matching on the voice response data of the vehicle-mounted voice interaction model. Combined with standardized voice test commands generated in the cloud and high-precision speech recognition technology, deep intent matching is performed through cosine similarity calculation and large language models to ensure the accuracy of the test results.
It improves the accuracy of test results for in-vehicle voice interaction models, enabling accurate evaluation of model performance in complex driving scenarios, reducing human interference, and enhancing test coverage and the objectivity of results.
Smart Images
Figure CN121789644A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a voice testing method, system, device and medium. Background Technology
[0002] Currently, with the continuous development of intelligent cockpits, in-vehicle voice interaction systems have evolved from simple command execution tools into complex scenario-based service entry points. Users have higher demands for the accuracy of their responses in real driving scenarios such as high-speed noise, multi-dialect commands, and multi-tasking concurrency.
[0003] However, the accuracy of test results in existing technologies for testing the performance of in-vehicle voice interaction models is not ideal. Summary of the Invention
[0004] The summary section introduces a series of simplified concepts, which will be further explained in detail in the detailed description section. This summary section is not intended to limit the key and essential technical features of the claimed technical solution, nor is it intended to determine the scope of protection of the claimed technical solution.
[0005] In a first aspect, embodiments of this application provide a voice testing method, the method comprising: Obtain a voice test command and input the voice test command into the vehicle voice interaction model so that the vehicle voice interaction model outputs voice response data; Obtain the expected response text corresponding to the voice test command and the speech recognition text corresponding to the voice response data; Perform a multi-level matching step, the multi-level matching step including: performing text matching on the speech recognition text and the expected response text to obtain a text matching result; if the text matching result is inconsistent, performing semantic matching on the speech recognition text and the expected response text to obtain a semantic matching result; if the semantic matching result is inconsistent, performing intent matching on the speech recognition text and the expected response text. Based on the matching results of the multi-level matching steps, the test results of the vehicle-mounted voice interaction model are determined.
[0006] In one embodiment of the present invention, the step of performing text matching between the speech-recognized text and the expected response text to obtain a text matching result includes: Determine whether the speech-recognized text and the expected response text are completely consistent. If the speech-recognized text and the expected response text are not completely consistent, the text matching result is that the text matching is inconsistent. If the speech-recognized text and the expected response text are completely identical, the text matching result is a text match.
[0007] In one embodiment of the present invention, the text matching of the speech-recognized text and the expected response text includes: Perform keyword matching between the speech recognition text and the expected response text; If the keywords in the speech-recognized text and the keywords in the expected response text do not match, the text matching result is a text mismatch. If the keywords in the speech-recognized text match the keywords in the expected response text, the text matching result is considered a text match.
[0008] In one embodiment of the present invention, the step of semantically matching the speech-recognized text and the expected response text to obtain a semantic matching result includes: The expected response text is encoded using sentence vector encoding to obtain a first vector; The speech recognition text is encoded into sentence vectors to obtain a second vector; The first similarity value is obtained by calculating the cosine similarity between the first vector and the second vector. If the first similarity value is less than a preset first similarity threshold, the semantic matching result is a semantic mismatch. If the first similarity value is greater than or equal to the preset first similarity threshold, the semantic matching result is a semantic match.
[0009] In one embodiment of the present invention, the intent matching of the speech-recognized text and the expected response text includes: The voice test command, the voice recognition text, and the expected response text are input into the large language model to obtain a second similarity value. If the second similarity value is less than the preset second similarity threshold, the matching result is a matching failure; If the second similarity value is greater than or equal to the preset second similarity threshold, the matching result is a successful match.
[0010] In one embodiment of the present invention, obtaining the voice test command includes: Obtain voice test instructions issued by the cloud, which are generated by the cloud through speech synthesis of the text content in the test cases.
[0011] In one embodiment of the present invention, after obtaining the expected response text corresponding to the voice test instruction and the speech recognition text corresponding to the voice response data, the process includes: Based on a preset vector database, errors in the speech recognition text are corrected.
[0012] Secondly, this application proposes a voice testing system, which includes: a data acquisition module, a text matching module, and a result determination module; The data acquisition module is configured to: acquire a voice test command, input the voice test command into the vehicle voice interaction model, so that the vehicle voice interaction model outputs voice response data; acquire the expected response text corresponding to the voice test command and the voice recognition text corresponding to the voice response data; The text matching module is configured to perform a multi-level matching step, which includes: performing text matching on the speech recognition text and the expected response text to obtain a text matching result; if the text matching result is inconsistent, performing semantic matching on the speech recognition text and the expected response text to obtain a semantic matching result; and if the semantic matching result is inconsistent, performing intent matching on the speech recognition text and the expected response text. The result determination module is configured to determine the test result of the vehicle-mounted voice interaction model based on the matching result of the multi-level matching steps.
[0013] Thirdly, an electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program stored in the memory to implement the steps of a voice testing method as described in any of the first aspects above.
[0014] Fourthly, this application also proposes a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of a voice testing method according to any one of the first aspects.
[0015] In summary, the voice testing method of this application performs multi-level matching on the voice response data output by the vehicle-mounted voice interaction model, making the test results of the vehicle-mounted voice interaction model more accurate, thereby helping to accurately evaluate the performance of the vehicle-mounted voice interaction model.
[0016] The speech testing method proposed in this application, along with other advantages, objectives, and features of this application, will be partly apparent from the following description and partly understood by those skilled in the art through study and practice of this application. Attached Figure Description
[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit this specification. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart illustrating a voice testing method provided in an embodiment of this application. Figure 2 This application provides a schematic diagram of the structure of a voice testing system. Figure 3 This is a schematic diagram of the structure of a voice testing electronic device provided in an embodiment of this application. Detailed Implementation
[0018] To better understand the technical solutions provided in the embodiments of this specification, the technical solutions of the embodiments of this specification will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this specification and the specific features in the embodiments are detailed descriptions of the technical solutions of the embodiments of this specification, rather than limitations on the technical solutions of this specification. In the absence of conflict, the embodiments of this specification and the technical features in the embodiments can be combined with each other.
[0019] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The term "two or more" includes two or more cases.
[0020] Please see Figure 1 This is a flowchart illustrating a voice testing method provided in an embodiment of this application, specifically including: S110. Obtain a voice test command and input the voice test command into the vehicle voice interaction model so that the vehicle voice interaction model outputs voice response data. For example, the voice test command text is first extracted from the test cases arranged in the test task. Then, using TTS (Text-To-Speech) technology, the voice test command text is converted into standardized audio, i.e., the voice test command. This audio is then sent to the vehicle's infotainment system via human speech. Before sending the voice test command, the system needs to be activated, for example, by sending a preset wake-up command to activate the system's voice interaction function. After ASR (Automatic Speech Recognition) confirms successful wake-up, the system enters the test state, triggering the vehicle's voice interaction model to respond and output the corresponding voice response data for the voice test command. The voice synthesis technology integrates corpora of more than 20 minority languages and dialects, supporting the automated synthesis of custom voice command text to generate standardized test voice. It also includes built-in noise libraries for traffic noise and environmental noise, which can be superimposed onto the synthesized voice at preset ratios to simulate the sound scenarios of real driving environments. By using TTS technology, test instructions can be generated automatically and in a standardized manner, avoiding test interference caused by differences in human pronunciation. It also supports noise superposition and multilingual or dialectal corpora, adapting to complex driving scenarios and solving the problems of non-standard manual instruction issuance and insufficient scenario coverage in traditional manual instruction issuance.
[0021] S120. Obtain the expected response text corresponding to the voice test command and the speech recognition text corresponding to the voice response data; For example, while issuing voice test commands, the expected response text corresponding to the voice test commands in the test cases is retrieved as a comparison benchmark. The VAD (Voice Activity Detection) audio acquisition module collects voice response data from the vehicle's infotainment system in real time. After extracting valid voice segments from the voice response data, these segments are input into the Sensevoice (high-precision speech recognition) model to complete automatic speech recognition, converting the voice response data into corresponding speech-recognized text. Simultaneously, the log acquisition module extracts voice interaction content from the vehicle's logs, forming a dual text data source. The VAD audio acquisition module can perform streaming audio acquisition, establishing a communication connection with the vehicle's infotainment system via HTTP (Hyper Text Transfer Protocol) service to collect voice response data in real time, supporting continuous streaming data transmission. The VAD module using FunASR (Fundamental Automatic Speech Recognition) also has audio monitoring capabilities, automatically detecting valid voice segments in the audio and eliminating meaningless noise segments to improve subsequent recognition efficiency.
[0022] S130. Perform a multi-level matching step, the multi-level matching step including: performing text matching on the speech recognition text and the expected response text to obtain a text matching result; if the text matching result is inconsistent, performing semantic matching on the speech recognition text and the expected response text to obtain a semantic matching result; if the semantic matching result is inconsistent, performing intent matching on the speech recognition text and the expected response text. For example, the text recognition text and the expected response text are first matched using either exact match or placeholder template match to obtain the text matching results, quickly filtering out scenarios that perfectly match the expectations. If the text matching results are inconsistent, a vector model is called to convert the speech recognition text and the expected response text into corresponding vectors. Based on the cosine similarity of the two vectors, a semantic matching result is obtained, thus completing the semantic matching. If the semantic matching results are inconsistent, the Qianwen 3-language model, the Wenxin Yiyan model, or the Zhipu Qingyan model are then used for fuzzy intent matching to determine whether the core intent is consistent, forming a progressive verification logic. This breaks through the limitations of traditional single-dimensional judgment. The three-level matching covers different scenarios of exact match, semantic similarity, and identical intent, avoiding misjudgments caused by differences in expression. Combining the synergistic advantages of rules, vector models, and large language models, it balances matching efficiency and accuracy, effectively covering implicit semantic defects and improving test coverage.
[0023] S140. Based on the matching results of the multi-level matching steps, determine the test results of the vehicle-mounted voice interaction model.
[0024] For example, if the text matching result is a text match, the test result of the vehicle-mounted voice interaction model is directly determined to be a pass. If the text matching result is a text mismatch, but the semantic matching result is a semantic match, the test result of the vehicle-mounted voice interaction model is also determined to be a pass. If both the text matching and semantic matching results are inconsistent, then the intent matching result needs to be considered. If the intent matching result is a successful match, the test result of the vehicle-mounted voice interaction model is also determined to be a pass. However, if the intent matching result is a failed match, the test result of the vehicle-mounted voice interaction model is determined to be a failure, and a test report is output, while test data and vehicle logs are retained to support subsequent review. Clearly defined and standardized result determination rules ensure that test results are repeatable and objective, eliminating subjective interference from manual judgment. The retention of test data and logs forms a closed loop, facilitating defect tracing and test plan optimization, improving the accuracy of test results, and effectively reducing customer complaint rates after product launch.
[0025] The cloud-based scheduling module enables automatic matching of test benches and test tasks, as well as collaborative scheduling of large and small models. Upon receiving a test task request from a client, it automatically matches suitable test bench resources and allocates computing power between large and small models based on task complexity, achieving a collaborative working mode where large models are responsible for fuzzy verification and small models are responsible for basic recognition and rule determination.
[0026] The client module supports plug-and-play functionality for engines, allowing for the combination and execution of test cases. It also features hot engine updates, enhancing system scalability. Employing a plug-in design, each test engine, such as speech synthesis, recognition, and verification engines, is encapsulated as a plugin, enabling plug-and-play operation. Users can combine different engines to execute test cases according to their testing needs, and hot engine updates are supported, allowing for feature upgrades without restarting the client.
[0027] A closed-loop test data system is built through functions such as natural language to test script conversion, automatic corpus generation, and generalized data generation. Test requirements described in natural language are transformed into structured test scripts, supporting direct invocation of various module functions. Diverse test corpora are automatically generated, including generalized data such as standard expressions and variant expressions, thus supporting comprehensive testing. Vehicle system logs are collected to provide data support for subsequent review and optimization.
[0028] In summary, the voice testing method proposed in this application performs multi-level matching on the voice response data output by the vehicle-mounted voice interaction model, making the test results of the vehicle-mounted voice interaction model more accurate, thereby helping to accurately evaluate the performance of the vehicle-mounted voice interaction model.
[0029] In some examples, the text matching of the speech-recognized text and the expected response text to obtain the text matching result includes: Determine whether the speech-recognized text and the expected response text are completely consistent. If the speech-recognized text and the expected response text are not completely consistent, the text matching result is that the text matching is inconsistent. If the speech-recognized text and the expected response text are completely identical, the text matching result is a text match.
[0030] For example, determining whether the speech recognition text and the expected response text are completely consistent is achieved through whole-word matching. The core of whole-word matching is to directly compare the character content of the speech recognition text and the expected response text. If the two textual expressions are completely consistent, such as having no character differences or word order differences, then the text matching result is considered to be consistent. As long as there are any differences at the textual level, regardless of whether the semantics are the same, then the text matching result is considered to be inconsistent. This is the most direct basic matching standard.
[0031] For example, if the expected response text is "For safety reasons, this operation is not supported in the current scenario", and the text recognized by the vehicle system is exactly the same as this text, the text matching result is a text match; if the text recognized by the voice system is "For driving safety, this operation is not supported in this scenario", although the semantics are similar, the wording is different, so the text matching result is a text mismatch.
[0032] When the text matching result is a text match, it indicates that the vehicle's voice response perfectly matches the expected requirements at the literal level. Based on this, the final result of the current test case will be determined as a pass, skipping subsequent semantic and intent matching steps, thus completing the verification with minimal computational cost. When the text matching result is a text mismatch, it indicates that the correctness of the response cannot be confirmed through simple character comparison. This text mismatch result will act as a trigger condition, automatically activating and entering the next level of semantic matching process. This provides a fast and efficient verification method, completing the judgment without complex calculations, saving testing time. Rigid standards ensure that scenarios that fully meet expectations are accurately identified, avoiding standard ambiguity caused by flexible matching and improving the rigor of test results.
[0033] In some examples, the text matching of the speech-recognized text and the expected response text includes: Perform keyword matching between the speech recognition text and the expected response text; If the keywords in the speech-recognized text and the keywords in the expected response text do not match, the text matching result is a text mismatch. If the keywords in the speech-recognized text match the keywords in the expected response text, the text matching result is considered a text match.
[0034] For example, the expected response text is first parsed to identify and mark the parts representing variable semantic fields, i.e., keywords are identified. Then, the speech recognition text is parsed to obtain the keywords corresponding to the speech recognition text. The text matching result is obtained by comparing the keywords in the speech recognition text and the expected response text. Specifically: First, the core keywords corresponding to the test scenario need to be identified, such as functional nouns, operational verbs, or key qualifiers. Then, these keywords are selected from both the speech recognition text and the expected response text. If the keywords of both sides match successfully, such as no missing or mismatched keywords, the text matching result is a text match. If there are missing or mismatched keywords, regardless of whether the non-keyword expressions are consistent, the text matching result is a text match disagreement.
[0035] For example, if the expected response text is "Air conditioning temperature dropped by 2℃", its core keywords are "air conditioning", "temperature", and "dropped by 2℃". The text recognized by the vehicle's ASR system is "The air conditioning temperature has been lowered by 2 degrees Celsius for you", and the extracted keywords are also "air conditioning", "temperature", and "dropped by 2℃". Since "lowered" and "dropped" have the same semantic meaning, and "2 degrees Celsius" and "2℃" have the same expression, the keyword match is successful, and the text matching result is a text match. However, if the voice recognition text is "The air conditioning fan mode has been turned on for you", the extracted keywords are "air conditioning" and "fan mode", which do not match the expected response keywords "temperature" and "dropped by 2℃". Therefore, the keyword match is considered unsuccessful, and the text matching result is a text match disagreement.
[0036] This ensures that the structure and key information of the expected response text are strictly validated for compliance, while allowing for reasonable changes to some content. It adapts to test scenarios with dynamic variables, resolving the issue of flexible expressions that rigid matching cannot cover; the templated design improves the reusability of test cases, eliminating the need to write multiple test cases for different expressions of the same function, thus reducing the cost of test case writing.
[0037] In some examples, the semantic matching of the speech-recognized text and the expected response text to obtain a semantic matching result includes: The expected response text is encoded using sentence vector encoding to obtain a first vector; The speech recognition text is encoded into sentence vectors to obtain a second vector; The first similarity value is obtained by calculating the cosine similarity between the first vector and the second vector. If the first similarity value is less than a preset first similarity threshold, the semantic matching result is a semantic mismatch. If the first similarity value is greater than or equal to the preset first similarity threshold, the semantic matching result is a semantic match.
[0038] For example, when the text matching step determines that the speech recognition text is inconsistent with the expected response text, the semantic matching process is automatically triggered. The first step of this process is sentence vector encoding: a preset vector model, such as the GTE (General Text Embedding) sentence vector model, is invoked to perform high-dimensional vector representation of both the expected response text and the speech recognition text. Specifically, sentence vector encoding is performed on the speech recognition text and the expected response text using Sentence Transformers. Through this encoding process, each of the two texts is converted into a fixed-dimensional numerical vector, resulting in a first vector representing the semantics of the expected response text and a second vector representing the semantics of the speech recognition text. Vector models trained on large-scale corpora can map semantically similar texts to positions close to each other in the vector space, thus laying the foundation for subsequent quantitative similarity comparison.
[0039] Subsequently, the similarity calculation and determination stage begins. After obtaining the first and second vectors, a cosine similarity algorithm is used to calculate the similarity between them. Specifically, the cosine of the angle between the first and second vectors in the spatial direction is calculated. This calculation result is the first similarity value, which represents the degree of semantic similarity between the two vectors and ranges from 0 to 1. This first similarity value is compared with a preset first similarity threshold used to determine whether the semantics match. For example, when the first similarity threshold is set to 0.85, if the calculated first similarity value is greater than or equal to 0.85, the semantic matching result is determined to be semantically consistent; conversely, if the first similarity value is less than 0.85, the semantic matching result is determined to be semantically inconsistent.
[0040] Transforming semantic matching into quantifiable numerical judgments avoids subjective judgment errors and improves the accuracy of semantic verification; capturing deep semantic relationships in text through vector calculation solves the problem of misjudgment when the textual expressions are different but the semantics are consistent, and expands the test coverage scenarios.
[0041] In some examples, the intent matching of the speech-recognized text and the expected response text includes: The voice test command, the voice recognition text, and the expected response text are input into the large language model to obtain a second similarity value. If the second similarity value is less than the preset second similarity threshold, the matching result is a matching failure; If the second similarity value is greater than or equal to the preset second similarity threshold, the matching result is a successful match.
[0042] For example, intent matching between the speech recognition text and the expected response text is the final matching step performed after text matching and semantic matching have both failed to confirm inconsistency. The principle is that when the first two levels of matching fail to confirm consistency, it indicates that the speech recognition text and the expected response text differ in both literal expression and general semantics. At this point, a large language model with deep language understanding and reasoning capabilities, such as Qianwen 3, is needed to determine intent. Specifically, the speech test command, the speech recognition text, and the expected response text are input into the large language model. Based on its understanding of natural language context, domain common sense, and logical relationships, the large language model comprehensively judges whether the core user intent or function execution result expressed by the speech recognition text and the expected response text is consistent, and outputs a quantified similarity score, i.e., the second similarity value.
[0043] After obtaining the second similarity value, the process proceeds to the judgment stage based on a preset second similarity threshold. The preset second similarity threshold is a predefined and configured numerical critical point used to objectively delineate the boundary between successful and failed intent matching; for example, the second similarity threshold can be set to 0.7. The judgment logic is as follows: the calculated second similarity value is compared with the second similarity threshold. If the second similarity value is greater than or equal to the second similarity threshold, the intent matching result is determined to be a successful match; conversely, if the second similarity value is less than the second similarity threshold, the matching result is determined to be a failed match. This ensures that even with flexible and varied expressions, as long as the core intent is determined to be highly consistent by the large language model, the verification will pass. It can accurately handle complex verification scenarios where expressions such as "turn on the coldest AC (Air Conditioning) mode," "turn on the rapid cooling mode," and "cooling mode has been switched to rapid cooling" have vastly different expressions but identical functional intents.
[0044] Breaking through the limitations of superficial differences in text and semantics, accurately identifying core meaning. Figure 1 This approach addresses specific scenarios and completely resolves misjudgments caused by differences in complex expressions. The intelligent analysis capabilities of the large language model can improve the level of intelligent matching, reduce manual intervention, and adapt to flexible response scenarios of in-vehicle voice interaction.
[0045] In some examples, obtaining the voice test command includes: Obtain voice test instructions issued by the cloud, which are generated by the cloud through speech synthesis of the text content in the test cases.
[0046] Exemplarily, based on the received test tasks, the cloud scheduling module extracts the corresponding test cases from the associated test case database, and parses the content of the voice command text defined in the test case and the test parameters bound thereto; the test parameters at least include at least one of the target language, dialect identifier, and the type and proportional coefficient of noise to be superimposed. Subsequently, the cloud invokes its integrated TTS voice synthesis service, which is built with pre-trained acoustic models and pronunciation dictionaries covering multiple minority languages and dialects, that is, corpora, and can select the corresponding model according to the target language or dialect identifier to synthesize the voice command text into a basic voice audio.
[0047] Next, according to the type and proportional coefficient of noise, from the noise library preset in the cloud, which contains various typical driving environment noise samples such as traffic wind noise, tire noise, and in-vehicle human voice interference, the corresponding noise sample is selected, and according to the specified proportional coefficient, the noise sample is accurately superimposed on the previously synthesized basic voice audio through a digital signal processing algorithm, so as to generate a voice test instruction simulating a real and complex acoustic environment. Finally, the cloud sends the generated voice test instruction to the specific test client that issued the request through a reliable network transmission protocol, and the local execution module of the client, such as playing it to the vehicle console through an artificial mouth audio playback device, thus completing the issuance of a standardized and scenario-based voice test instruction.
[0048] Implement the standardized generation of test instructions, avoid test interference caused by artificial pronunciation differences, and ensure the consistency of instructions in different test rounds and different test environments; centralized generation and issuance in the cloud facilitate batch management of test tasks and improve the standardization and efficiency of the test process.
[0049] In some examples, after obtaining the expected response text corresponding to the voice test instruction and the voice recognition text corresponding to the voice response data, it includes: Based on a preset vector database, correct the error text in the voice recognition text.
[0050] Exemplarily, first extract the standard text content corresponding to the voice test instruction from the collected voice logs as a reference benchmark for correct semantics. Subsequently, compare and analyze the recognition text output by the voice recognition model with the standard text content character by character or word by word, so as to accurately locate the characters or words with differences between the two. These marked difference points are potential recognition errors, such as misrecognizing the English proper noun "HIFI" as the homophonic Chinese word "嗨发".
[0051] The system corrects identified discrepancies in characters or words based on a pre-built vector database. This database is a pre-constructed and continuously optimized mapping knowledge base that stores numerous mapping pairs from "common recognition error forms" to "standard correct forms." The keys in these pairs are typical recognition error characters or words, and the values are their corresponding standard characters or words. Using the discrepancy character or word as the query key, the system searches the vector database. If a match is found, its mapped standard character or word is retrieved, and this standard character or word is used to directly replace the corresponding discrepancy in the recognized text, generating a corrected and more accurate speech recognition text. If no match is found, the discrepancy is recorded and marked as a new error pattern requiring manual review. After manual confirmation of its correct standard form, the new error and its corresponding standard are fed back as training data and added to the vector database.
[0052] Correcting errors in the ASR recognition process improves the accuracy of speech-to-text recognition, providing reliable data support for subsequent matching processes; reducing test misjudgments caused by recognition errors improves the overall accuracy of test data, reduces manual review costs, and indirectly improves test efficiency.
[0053] like Figure 2 As shown, this application proposes a voice testing system, which includes: a data acquisition module 21, a text matching module 22, and a result determination module 23; The data acquisition module 21 is configured to: acquire a voice test command, input the voice test command into the vehicle voice interaction model so that the vehicle voice interaction model outputs voice response data; acquire the expected reply text corresponding to the voice test command and the voice recognition text corresponding to the voice response data; The text matching module 22 is configured to perform a multi-level matching step, the multi-level matching step including: performing text matching on the speech recognition text and the expected response text to obtain a text matching result; if the text matching result is inconsistent, performing semantic matching on the speech recognition text and the expected response text to obtain a semantic matching result; if the semantic matching result is inconsistent, performing intent matching on the speech recognition text and the expected response text. The result determination module 23 is configured to determine the test result of the vehicle voice interaction model based on the matching result of the multi-level matching steps.
[0054] The effects of applying the aforementioned method in the above system can be found in the description of the aforementioned method embodiments, and will not be repeated here.
[0055] like Figure 3As shown, this application embodiment also provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor. When the processor 320 executes the computer program 311, it implements the steps of any of the above-described voice testing methods.
[0056] Since the electronic device described in this embodiment is the device used to implement a voice testing device in the embodiments of this application, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this embodiment based on the method described in the embodiments of this application. Therefore, how the electronic device implements the method in the embodiments of this application will not be described in detail here. Any device used by those skilled in the art to implement the method in the embodiments of this application falls within the scope of protection of this application.
[0057] In practical implementation, when the computer program 311 is executed by the processor, it can achieve the following: Figure 1 Any of the corresponding implementation methods in the embodiments.
[0058] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.
[0059] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-readable program code.
[0060] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0061] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0062] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0063] This application also provides a computer program product, which includes computer software instructions that, when executed on a processing device, cause the processing device to execute the LDPC decoding method of a solid-state drive controller.
[0064] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).
[0065] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0066] In the several embodiments provided in this application, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.
[0067] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0068] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0069] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0070] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
[0071] Although preferred embodiments have been described in this specification, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this specification.
[0072] Obviously, those skilled in the art can make various modifications and variations to this specification without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims and their equivalents, this specification is also intended to include such modifications and variations.
Claims
1. A voice testing method, characterized in that, The method includes: Obtain a voice test command and input the voice test command into the vehicle voice interaction model so that the vehicle voice interaction model outputs voice response data; Obtain the expected response text corresponding to the voice test command and the speech recognition text corresponding to the voice response data; Perform a multi-level matching step, the multi-level matching step including: performing text matching on the speech recognition text and the expected response text to obtain a text matching result; if the text matching result is inconsistent, performing semantic matching on the speech recognition text and the expected response text to obtain a semantic matching result; if the semantic matching result is inconsistent, performing intent matching on the speech recognition text and the expected response text. Based on the matching results of the multi-level matching steps, the test results of the vehicle-mounted voice interaction model are determined.
2. The voice testing method according to claim 1, characterized in that, The step of performing text matching between the speech recognition text and the expected response text to obtain the text matching result includes: Determine whether the speech-recognized text and the expected response text are completely consistent. If the speech-recognized text and the expected response text are not completely consistent, the text matching result is that the text matching is inconsistent. If the speech-recognized text and the expected response text are completely identical, the text matching result is a text match.
3. The voice testing method according to claim 1, characterized in that, The text matching of the speech-recognized text and the expected response text includes: Perform keyword matching between the speech recognition text and the expected response text; If the keywords in the speech-recognized text and the keywords in the expected response text do not match, the text matching result is a text mismatch. If the keywords in the speech-recognized text match the keywords in the expected response text, the text matching result is considered a text match.
4. The voice testing method according to claim 1, characterized in that, The step of semantically matching the speech-recognized text and the expected response text to obtain a semantic matching result includes: The expected response text is encoded using sentence vector encoding to obtain a first vector; The speech recognition text is encoded into sentence vectors to obtain a second vector; The first similarity value is obtained by calculating the cosine similarity between the first vector and the second vector. If the first similarity value is less than a preset first similarity threshold, the semantic matching result is a semantic mismatch. If the first similarity value is greater than or equal to the preset first similarity threshold, the semantic matching result is a semantic match.
5. The voice testing method according to claim 1, characterized in that, The intent matching of the speech-recognized text and the expected response text includes: The voice test command, the voice recognition text, and the expected response text are input into the large language model to obtain a second similarity value. If the second similarity value is less than the preset second similarity threshold, the matching result is a matching failure; If the second similarity value is greater than or equal to the preset second similarity threshold, the matching result is a successful match.
6. The voice testing method according to claim 1, characterized in that, The process of obtaining the voice test command includes: Obtain voice test instructions issued by the cloud, which are generated by the cloud through speech synthesis of the text content in the test cases.
7. The voice testing method according to claim 1, characterized in that, After obtaining the expected response text corresponding to the voice test command and the speech recognition text corresponding to the voice response data, the process includes: Based on a preset vector database, errors in the speech recognition text are corrected.
8. A voice testing system, characterized in that, The system includes: a data acquisition module, a text matching module, and a result determination module; The data acquisition module is configured to: acquire a voice test command, input the voice test command into the vehicle voice interaction model, so that the vehicle voice interaction model outputs voice response data; acquire the expected response text corresponding to the voice test command and the voice recognition text corresponding to the voice response data; The text matching module is configured to perform a multi-level matching step, which includes: performing text matching on the speech recognition text and the expected response text to obtain a text matching result; if the text matching result is inconsistent, performing semantic matching on the speech recognition text and the expected response text to obtain a semantic matching result; and if the semantic matching result is inconsistent, performing intent matching on the speech recognition text and the expected response text. The result determination module is configured to determine the test result of the vehicle-mounted voice interaction model based on the matching result of the multi-level matching steps.
9. An electronic device, comprising: The memory and processor are characterized in that the processor is used to implement the steps of a voice testing method as described in any one of claims 1-7 when executing a computer program stored in the memory.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of a voice testing method as described in any one of claims 1-7.