Voice processing link test method and device, equipment and storage medium

Abstract

The invention relates to a voice processing link test method and device, equipment and a storage medium. The method comprises the following steps: acquiring a test case set obtained by executing preprocessing operation according to the type of the test case set; according to the test case set obtained after the preprocessing operation is executed, different verifications are sequentially carried out on different links of the voice processing links, and verification of each link of the different links depends on an output result of verification of the previous link. According to the scheme provided by the invention, the whole link of the voice processing link can be verified completely, the complexity of verifying the voice processing link is reduced, and the link with a problem is positioned more accurately.

Description

Technical Field

[0001] The present application relates to the field of automated testing and verification technology, and in particular to a testing method, device, equipment and storage medium for a voice processing link. Background Art

[0002] With the continuous development of electric vehicles, various intelligent functions can be provided to users, such as voice interaction. In order to provide users with better voice interaction functions, the vehicle's voice processing link needs to be tested.

[0003] The voice testing tools of related technologies are often limited to technical verification of a single module of the voice processing link, and the verification process is complex and incomplete. For example, the modules of the voice processing link such as Automatic Speech Recognition (ASR), Natural Language Understanding (NLU), and Text To Speech (TTS) are verified as independent modules. This segmented verification method ignores the complex interactive relationship between ASR, NLU and TTS, resulting in a lack of integrity in the verification of the entire voice processing link. This incomplete verification not only increases the complexity of the verification process, but also makes it difficult to accurately trace the specific link where the problem occurs when problems occur in actual applications, thereby increasing the difficulty of troubleshooting.

[0004] Therefore, the voice automation testing method of the related art lacks completeness in verifying the entire voice processing link, and it is difficult to accurately track the specific link where the problem occurs. Summary of the invention

[0005] In order to solve or partially solve the problems existing in the related technology, the present application provides a testing method, device, equipment and storage medium for a voice processing link, which can completely verify the entire link of the voice processing link, reduce the complexity of verifying the voice processing link, and more accurately locate the link where the problem occurs.

[0006] A first aspect of the present application provides a method for testing a voice processing link, the method comprising: Obtain a test case set obtained after performing a preprocessing operation according to the type of the test case set; According to the test case set obtained after performing the preprocessing operation, different verifications are performed on different links of the voice processing link in turn, wherein each link verification of the different links depends on the output result of the previous link verification.

[0007] Preferably, the test case set obtained after performing the preprocessing operation is used to sequentially perform different verifications on different links of the voice processing link, wherein each link verification of the different links depends on the output result of the previous link verification, including: Using the test case set obtained after performing the preprocessing operation, the ASR link of the speech processing link is verified to obtain an output text data set; Verify the NLU link of the speech processing link according to the text data set to obtain a feedback audio data set; The TTS link of the speech processing link is verified according to the feedback audio data set.

[0008] Preferably, the test case set obtained after performing the preprocessing operation is used to verify the ASR link of the speech processing link to obtain an output text data set, including: Obtaining a text data set output by the ASR link of the speech processing link according to the test case set obtained after performing the preprocessing operation; The ASR link of the speech processing link is evaluated by one or a combination of the character accuracy rate, word error rate, and sentence accuracy rate of the text data set.

[0009] Preferably, the verifying the NLU link of the speech processing link according to the text data set includes: Obtaining the dialogue control algorithm, rule engine, and instruction information of the NLU link in processing the text data set; The NLU link of the speech processing link is evaluated by comparing the conversation control algorithm, rule engine, and instruction information of the NLU link in processing the text data set with the expected conversation control algorithm, expected rule engine, and expected instruction information annotated in the test case set.

[0010] Preferably, the verifying the TTS link of the speech processing link according to the feedback audio data set includes: Acquiring speech parameters through a visual interface, and adjusting the feedback audio data set according to the speech parameters, wherein the speech parameters include one or more of sound quality, intonation, speech speed, and volume; The TTS link of the speech processing link is verified using a feedback audio data set for adjusting speech parameters.

[0011] Preferably, the verification of the TTS link of the speech processing link by using the feedback audio data set for adjusting speech parameters includes: Convert the feedback audio dataset of adjusting speech parameters into a feedback text dataset; The TTS link of the speech processing link is evaluated by comparing the feedback text dataset with the expected feedback text dataset annotated by the test case set.

[0012] Preferably, before obtaining the test case set obtained after performing a preprocessing operation according to the type of the test case set, the method includes: Get the test parameters entered through the visual interface; A test case set is loaded according to the test parameters.

[0013] Preferably, the method further comprises: generating a test report evaluating the speech processing link; The test report is sent to the user's terminal.

[0014] Preferably, the test report includes the test case pass rate and / or field pass rate; wherein, According to the test cases in the test case set that pass the voice processing link, the test case pass rate is obtained, wherein the test case pass rate is the ratio of the number of test cases in the test case set that pass the voice processing link to the total number of test cases in the test case set; and / or, According to the test cases that have passed the field in the test case set, the field pass rate is obtained, wherein the field pass rate is the ratio of the number of test cases that have passed the field in the test case set to the total number of test cases in the field in the test case set, and the field corresponds to a function or module of the vehicle.

[0015] Preferably, obtaining the test case pass rate according to the test cases in the test case set that pass the voice processing link further includes: Converting the feedback audio data in the feedback audio data set into feedback text data; If the feedback text data is consistent with the expected feedback text data annotated by the test case in the test case set, determining that the test case in the test case set is a test case in the test case set that passes the voice processing link; and / or, Obtaining a pass rate of a field according to the test cases in the test case set that pass the field, further comprising: Converting the feedback audio data in the feedback audio data set into feedback text data; If the feedback text data is consistent with the expected feedback text data annotated on the test case in the test case set, it is determined that the test case in the test case set is a test case in the passed field in the test case set.

[0016] Preferably, the method further comprises: If the feedback text data is inconsistent with the expected feedback text data annotated by the test case in the test case set, determining that the test case in the test case set is a failed test case; The failed test case is notified to the domain person in charge corresponding to the domain of the failed test case.

[0017] Preferably, the test report also includes actual log information when verifying the ASR link, the NLU link, and the TTS link; The method further comprises: Compare the actual log information of the failed test case with the expected log information marked by the failed test case to generate a failed test case highlighted log information library.

[0018] Preferably, the method further comprises: The voice processing link is tested using the failed test case.

[0019] Preferably, performing preprocessing operations according to the type of the test case set includes: If the test case set is of audio type, directly use it as the test case set; or, Since the type of the test case set is a text type, a preset tool is used to convert the text type test case set into an audio type test case set as the test case set.

[0020] A second aspect of the present application provides a test device for a voice processing link, the device comprising: A preprocessing module, used to obtain a test case set obtained after performing a preprocessing operation according to the type of the test case set; The verification module is used to perform different verifications on different links of the voice processing link in turn according to the test case set obtained after the preprocessing operation is performed by the preprocessing module, wherein each link verification of the different links depends on the output result of the previous link verification.

[0021] A third aspect of the present application provides a testing device, comprising: Processor; and The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method as described above.

[0022] A fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of a test device, the processor is caused to execute the method as described above.

[0023] The technical solution provided by this application may have the following beneficial effects: The technical solution of the present application performs different verifications on different links of the voice processing link in turn according to the test case set obtained after executing the preprocessing operation, wherein each link verification of different links depends on the output result of the previous link verification; the test case set is used to verify the ASR link of the voice processing link; the ASR link is used to verify the NLU link of the voice processing link based on the text data set output by the test case set; the voice processing link is used to verify the TTS link of the voice processing link based on the feedback audio data set fed back by the test case set; by verifying the ASR link-NLU link-TTS link of the voice processing link, the full-link verification of the voice processing link is realized, the effective coordination of each link in the voice processing link is ensured, and the problem of complex and incomplete voice processing link verification process in related technologies is solved; the full-link of the voice processing link can be fully verified, the complexity of verifying the voice processing link can be reduced, the link where the problem occurs can be more accurately located, the accuracy and stability of the entire voice processing link can be ensured, and the application effect of the voice processing link in the actual voice interaction business can be improved. It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0025] Figure 1 It is a flowchart of a method for testing a voice processing link shown in an embodiment of the present application; Figure 2 is a flow chart of a method for testing a voice processing link according to another embodiment of the present application; Figure 3 is a flow chart of a method for testing a voice processing link according to another embodiment of the present application; Figure 4 It is a structural schematic diagram of a test device for a voice processing link shown in an embodiment of the present application; Figure 5 is a structural schematic diagram of a test device for a voice processing link shown in another embodiment of the present application; Figure 6 It is a schematic diagram of the structure of the test equipment shown in the embodiment of the present application. DETAILED DESCRIPTION

[0026] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0027] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0028] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0029] With the continuous development of electric vehicles, various intelligent functions can be provided to users, such as voice interaction. The voice interaction function of electric vehicles can realize human-computer interaction through the on-board voice processing link. Users can not only control the on-board software such as navigation and music by voice, but also control the hardware in the car such as windows and air conditioning. In order to provide users with better voice interaction functions, it is necessary to test the vehicle's voice processing link.

[0030] The voice testing tools of related technologies are often limited to technical verification of a single module of the voice processing link, and the verification process is complex and incomplete. For example, the modules of the voice processing link such as Automatic Speech Recognition (ASR), Natural Language Understanding (NLU), and Text To Speech (TTS) are verified as independent modules. This segmented verification method ignores the complex interactive relationship between ASR, NLU and TTS, resulting in a lack of integrity in the verification of the entire voice processing link. This incomplete verification not only increases the complexity of the verification process, but also makes it difficult to accurately trace the specific link where the problem occurs when problems occur in actual applications, thereby increasing the difficulty of troubleshooting.

[0031] Therefore, the voice automation testing method of the related art lacks completeness in verifying the entire voice processing link, and it is difficult to accurately track the specific link where the problem occurs.

[0032] In response to the above problems, an embodiment of the present application provides a testing method for a voice processing link, which can completely verify the entire link of the voice processing link, reduce the complexity of verifying the voice processing link, and more accurately locate the link where the problem occurs.

[0033] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0034] Figure 1 It is a flow chart of a method for testing a voice processing link shown in an embodiment of the present application.

[0035] See also Figure 1 , a method for testing a voice processing link, comprising: Step 101, obtaining a test case set obtained after performing a preprocessing operation according to the type of the test case set.

[0036] Step 102, according to the test case set obtained after performing the preprocessing operation, different links of the voice processing link are verified in turn, wherein each link verification of different links depends on the output result of the previous link verification.

[0037] In one embodiment, the type of the test case set can be obtained, and according to the type of the test case set, A preprocessing operation is performed on the test case set so that the test case set obtained after the preprocessing operation meets the requirements of testing the voice processing link.

[0038] In one embodiment, the test case set obtained after the preprocessing operation can be input into the ASR link of the voice processing link to verify the ASR link of the voice processing link; the ASR link is evaluated based on the text data set output by the test case set.

[0039] In one embodiment, the text data set can be input into the NLU link of the speech processing link to verify the NLU link; the NLU link is evaluated based on the dialogue control algorithm, rule engine, and instruction information of the NLU link processing the text data set.

[0040] In one embodiment, a feedback audio data set of a speech processing link based on feedback from a test case set can be obtained; the TTS link of the speech processing link is verified based on the feedback audio data set, and the TTS link of the speech processing link is evaluated by comparing the feedback text data set with the expected feedback text data set annotated by the test case set.

[0041] In one embodiment, a test report for evaluating the speech processing link may be generated based on evaluations of the ASR link, the NLU link, and the TTS link of the speech processing link, respectively.

[0042] The testing method of the voice processing link of the embodiment of the present application performs different verifications on different links of the voice processing link in turn according to the test case set obtained after performing the preprocessing operation, wherein each link verification of different links depends on the output result of the previous link verification; the ASR link of the voice processing link is verified by the ASR link based on the text data set output by the test case set; the TTS link of the voice processing link is verified by the voice processing link based on the feedback audio data set fed back by the test case set; by verifying the ASR link-NLU link-TTS link of the voice processing link, the full-link verification of the voice processing link is realized, the effective coordination of each link in the voice processing link is ensured, and the problem of complex and incomplete voice processing link verification process in related technologies is solved; the full-link of the voice processing link can be fully verified, the complexity of verifying the voice processing link is reduced, the link where the problem occurs can be more accurately located, the accuracy and stability of the entire voice processing link can be ensured, and the application effect of the voice processing link in the actual voice interaction business is improved.

[0043] Figure 2 is a flow chart of a method for testing a voice processing link according to another embodiment of the present application; Figure 3 It is a flowchart of a method for testing a voice processing link according to another embodiment of the present application. Figure 2 and Figure 3 Relative to Figure 1 The technical solution of the present application is described in more detail.

[0044] See also Figure 2 and Figure 3 , a method for testing a voice processing link, comprising: Step 201, loading a predefined test case set.

[0045] In one embodiment, the test device may obtain test parameters input through the visual interface; Load a test case suite based on test parameters.

[0046] In one embodiment, the testing device may test the voice processing links including the ASR link, the NLU link, and the TTS link according to the test parameters input by the user; wherein the user's test parameters may be parameters for testing whether the voice processing link meets the design requirements under various application scenarios (various voices).

[0047] In the voice processing link test of the related art, the input generation of the test case set depends on manual operation, which increases the workload of the voice processing link test and reduces the efficiency of the voice processing link test. In the embodiment of the present application, by providing a visual interface to the user, obtaining the test parameters input through the visual interface; loading the test case set according to the test parameters; and realizing the automatic input of the test case set, the workload of the voice processing link test can be reduced and the efficiency of the voice processing link test can be improved.

[0048] In one embodiment, the test device may provide a visual interface to the user for the user to input test parameters; the test device obtains the test parameters input by the user through the visual interface, and loads a predefined test case set corresponding to the test parameters for the test voice processing link by calling a function; wherein the predefined test case set may be a test case set stored in a cloud online document, a local document, or a default test case set. For example, the test device may load a predefined test case set corresponding to the test parameters provided by default according to the test parameters; or, the test device may load a predefined test case set corresponding to the test parameters stored in an excel file in a local document according to the test parameters; or, the test device may load a predefined test case set corresponding to the test parameters provided by a third-party application software (e.g., a chat social software) according to the test parameters.

[0049] In one embodiment, the predefined test case set is predefined text type data or audio type data that can test the ASR link, NLU link, and TTS link of the voice processing link. The test device can provide the user with a visual interface that implements the test case set version management function, so that the user can manage the predefined test case set through the visual interface. The user can establish a test case set named with a version number, establish a use case item associated with the version number of the test case set, and distinguish the field of the test case set, the model and type of the applicable vehicle, and other information. The user manages the version, field, model and type of the applicable vehicle of the predefined test case set through the visual interface to ensure version control and track changes to the predefined test case set.

[0050] In one embodiment, a domain corresponds to a function or module of a vehicle, and a function of a vehicle may correspond to a domain, such as turning on the air conditioner, automatic parking, etc. A module of a vehicle may also correspond to a domain, such as the vehicle's air conditioner, windows, seats, automatic parking, XGPT (large natural language model), third-party applications, music, navigation, etc.

[0051] In one embodiment, the user can input test parameters through the visual interface of the test device, and the test parameters include vehicle model (vehicle model), environment (the environment in which the vehicle is located, for example, the vehicle is indoors, the vehicle is outdoors), large model or small model (model, indicating the processing capability of the voice processing link), voice version (version of the test case set), and voice processing link version. Through the visual interface of the test device, it is possible to support multiple vehicle models and / or multiple versions of voice processing links for parallel execution, thereby enhancing the flexibility and adaptability of voice processing link testing and meeting the needs in different scenarios.

[0052] In one embodiment, the test device can provide a visual interface to the user, so that the user can specify the test case set for testing the voice processing link through the visual interface. The user specifies the field, version number, model and type of applicable vehicles of the test case set through the visual interface, and the test device can quickly locate and execute a specific type of test case set, thereby greatly improving the test efficiency of the voice processing link test.

[0053] Step 202, determine whether the test case set is an audio type test case set; if yes, execute step 204; if not, execute step 203.

[0054] In one embodiment, the type of the test case set may be a text type or an audio type test case set. The test device may detect the type of the test case set, perform a preprocessing operation on the test case set according to the type of the test case set, and obtain the test case set obtained after the preprocessing operation is performed according to the type of the test case set.

[0055] In one embodiment, the testing device can detect the type of the test case set and determine whether the test case set is an audio type test case set; if the test case set is an audio type test case set, execute step 204; if the test case set is not an audio type test case set but a text type test case set, execute step 203.

[0056] Step 203: Use a preset tool to convert the text-type test case set into an audio-type test case set as the test case set.

[0057] In one embodiment, if the test device determines that the type of the test case set is text type based on the fact that the test case set is text type, a preset tool is used to convert the text type test case set into an audio type test case set as the test case set.

[0058] In one embodiment, if the test case set is not an audio type test case set but a text type test case set, the testing device may convert the text type test case set into an audio type test case set and use the audio type test case set as the test case set.

[0059] In one embodiment, if the test case set is not an audio type test case set but a text type test case set, the testing device can convert the text type test case set into an audio type test case set according to the text-to-audio engine selected by the user through the visual interface (for example, pyttsx3 (Python text-to-speech library), gTTS (Google Text-to-Speech, a Python library that converts text to speech), coqui-tts (text-to-speech (TTS)) generation library, Microsoft tts), and use the audio type test case set as the test case set.

[0060] In one embodiment, the test device can convert the text-type test case set into the audio-type test case set according to the text-to-audio engine selected by the user through the visual interface, and can obtain the audio-type test case set corresponding to the text-type test case set and for different application scenarios. For example, the audio-type test case set of different genders, and / or different ages, and / or different speech speeds, and / or different speech rates, and / or different speech volumes.

[0061] Step 204: Use an audio type test case set to verify the ASR link of the speech processing link to obtain an output text data set.

[0062] In one embodiment, the test device may use the test case set obtained after performing the preprocessing operation to verify the ASR link of the speech processing link to obtain an output text data set.

[0063] In one embodiment, during the verification of the ASR link, the test device may directly use the audio type test case set as the test case set based on the type of the test case set being an audio type. The ASR link of the voice processing link is verified using the audio type test case set, and a text data set output by the ASR link based on the audio type test case set is obtained. The test device may input the audio type test case set into the ASR link of the voice processing link. The ASR link receives the audio type test case set input by the test device, outputs text data corresponding to each test case of the test case set based on the audio type test case set, and obtains a text data set composed of text data corresponding to each test case.

[0064] In one embodiment, the test device can obtain a text data set output by the ASR link of the speech processing link according to the test case set obtained after performing a preprocessing operation; evaluate the ASR link of the speech processing link through one or a combination of the word accuracy rate, word error rate, and sentence accuracy rate of the text data set. The test device can compare the text data output by the ASR link and corresponding to each test case in the test case set according to the expected text data annotated by each test case in the test case set, and obtain evaluation indicators such as the word accuracy rate, word error rate, and sentence accuracy rate of the text data set; evaluate the ASR link of the speech processing link through one or a combination of the word accuracy rate, word error rate, and sentence accuracy rate of the text data set to comprehensively evaluate the accuracy of the ASR link output, and through a detailed comparison of the expected text data annotated by each test case in the test case set and the text data output by the ASR link and corresponding to each test case in the test case set, so as to provide users with a clear understanding of the performance of the ASR link, and provide strong data support for the subsequent optimization and improvement of the ASR link.

[0065] In one embodiment, the testing device may calculate one or more of the word accuracy rate, word error rate (WER), and sentence accuracy rate (S.Corr) of the text data corresponding to each test case in the test case set output by the ASR link based on the expected text data annotated for each test case in the test case set and the text data corresponding to each test case in the test case set output by the ASR link, and obtain one or more of the evaluation indicators such as the word accuracy rate, word error rate, and sentence accuracy rate of the text data set; evaluate the accuracy of the ASR link output through one or more of the word accuracy rate, word error rate, and sentence accuracy rate of the text data set, so that the user can clearly understand the performance of the ASR link, and provide strong data support for the subsequent optimization and improvement of the ASR link.

[0066] In one embodiment, in the process of verifying the ASR link of the voice processing link using an audio type test case set, the test equipment can obtain the duration of the ASR link from receiving each test case in the test case set and converting each test case in the test case set into text data corresponding to each test case in the test case set according to each test case in the test case set; and evaluate the real-time performance of the ASR link according to the duration of each test case in the test case set.

[0067] In one embodiment, during the verification of the ASR link, the test device may add support for a specific language or dialect, or add a custom evaluation index for evaluating the ASR link through a visual interface.

[0068] In one embodiment, during the verification of the ASR link, the test equipment may verify the ASR link using a test case set including test cases of different genders, or a test case set including test cases of different ages, or a test case set including test cases of different dialects, or a test case set including test cases of different languages, and output evaluation indicators for evaluating the accuracy of the ASR link output to expand the application scope of the test equipment, so as to more accurately evaluate the performance of the voice processing link, and facilitate the subsequent precise positioning of the vehicle's customer base based on the evaluation indicators of the ASR link and the voice processing link.

[0069] In one embodiment, the test device can use an open source audio processing library (for example, Ffmpeg (a set of open source computer programs for recording, converting digital audio and video, and converting them into streams), Librosa (a Python library for audio and music analysis)) or utilize a cloud-based TTS service (Text-to-Speech Service) to convert test case sets of different audio types into test case sets of audio types in wav format, increase support for more audio formats, enhance the flexibility and functionality of the test device, and provide rich pre-processing options.

[0070] In one embodiment, the testing device may use VAD (Voice activity detection) to pre-process the test case set in WAV format (sound file format), detect whether there is a voice signal in the test case set in WAV format, perform noise reduction processing on the test case set in WAV format, and / or adjust the volume of the test case set in WAV format to meet the needs of different users.

[0071] Step 205: Use the ASR link to verify the NLU link of the speech processing link based on the text data set output by the test case set to obtain a feedback audio data set.

[0072] In one embodiment, the test device can verify the NLU link of the speech processing link according to the text data set output by the ASR link according to the test case set, and obtain a feedback audio data set.

[0073] In one embodiment, the testing device can input the text data set output by the ASR link based on the test case set into the NLU link; the NLU link outputs semantic understanding and semantic execution based on the text data set output by the ASR link; the testing device can check the semantic understanding and semantic execution output by the NLU link based on the text data set, and verify the NLU link.

[0074] In one embodiment, the test device can obtain the dialogue control algorithm, rule engine, and instruction information of the NLU link processing the text data set process; by comparing the dialogue control algorithm, rule engine, and instruction information of the NLU link processing the text data set process with the expected dialogue control algorithm, expected rule engine, and expected instruction information annotated in the test case set, the NLU link of the voice processing link is evaluated.

[0075] In one embodiment, the NLU link outputs the semantic understanding and semantic execution (instruction information) corresponding to each text data in the text data set output by the ASR link according to the dialogue control algorithm and the rule engine. The test equipment can check the dialogue control algorithm, rule engine, and instruction information of the semantic understanding and semantic execution process output by the NLU link, check the dialogue control algorithm (action and arguments) and rule engine (actionName, actionArguments) and instruction information (cmd, nlgId, tts) of the semantic understanding and semantic execution output by the NLU link, and compare the dialogue control algorithm, rule engine, and instruction information of the semantic understanding and semantic execution process corresponding to each text data in the text data set output by the NLU link with the semantic understanding, semantic execution, and dialogue control algorithm annotated for each test case in the test case set. The NLU link is verified according to the differences between the dialogue control algorithm, rule engine, and instruction information of the semantic understanding and semantic execution process corresponding to each text data in the text dataset output by the NLU link and the semantic understanding, semantic execution, dialogue control algorithm, rule engine, and instruction information annotated in each test case in the test case set; wherein each test case in the test case set corresponds to each text data in the text dataset output by the ASR link, and each text data in the text dataset output by the ASR link corresponds to the semantic understanding and semantic execution output by the NLU link and the dialogue control algorithm, rule engine, and instruction information of the semantic understanding and semantic execution process output by the NLU link.

[0076] Step 206 , verify the TTS link of the speech processing link according to the feedback audio data set fed back by the speech processing link according to the test case set.

[0077] In one embodiment, for a test case in a test case set, a voice processing link obtains the input audio data of the test case, performs VAD processing and signal processing on the audio data of the test case, and obtains the voice signal of the audio data; the ASR link extracts the features of the voice signal according to voice algorithms with different functions, obtains the feature extraction results and caches them; according to the preset configuration parameters of each voice algorithm, the feature extraction results in the cache are called, the feature extraction results are decoded (decoder) to obtain the corresponding decoding results; the corresponding acoustic model (AM), dictionary (lexicon), and language model (LM) are called, and the corresponding acoustic calculation results (text data) are obtained according to the corresponding decoding results; the NLU link outputs the corresponding semantic understanding and semantic execution according to the acoustic calculation results, and sends the semantic understanding and semantic execution to the vehicle; the vehicle executes the corresponding action according to the semantic understanding and semantic execution, and feeds back the corresponding action; the TTS link outputs feedback audio data (feedback voice) corresponding to the feedback according to the feedback.

[0078] In one embodiment, the test device can provide a visual interface to the user, so that the user can input the voice parameters of the feedback audio data set fed back by the test case set to adjust the voice processing link through the visual interface. The test device obtains the voice parameters through the visual interface, and adjusts the feedback audio data set according to the voice parameters, wherein the voice parameters include one or more of sound quality, intonation, speech speed, and volume; and verifies the TTS link of the voice processing link using the feedback audio data set with the adjusted voice parameters.

[0079] In one embodiment, the test device can provide a visual interface to the user, so that the user can set the voice parameters of each feedback audio data in the feedback audio data set through the visual interface; the test device receives the voice parameters input by the user through the visual interface, and adjusts the voice parameters of each feedback audio data in the feedback audio data set.

[0080] In one embodiment, the test device may convert a feedback audio data set for adjusting speech parameters into a feedback text data set; and evaluate the TTS link of the speech processing link by comparing the feedback text data set with the expected feedback text data set annotated by the test case set.

[0081] In one embodiment, the test device may perform STT (Speech-to-Text) conversion on the feedback audio data set to obtain a feedback text data set corresponding to the feedback audio data set, wherein each piece of feedback text data in the feedback text data set corresponds to each piece of feedback audio data in the feedback audio data set, and compare the feedback text data set with the expected feedback text data set annotated by the test case set, and check whether each piece of feedback text data in the feedback text data set conforms to the expected feedback text data annotated by each test case in the test case set, and verify the TTS link.

[0082] Step 207: Generate a test report for evaluating the speech processing link based on the verification of the ASR link, the NLU link, and the TTS link of the speech processing link.

[0083] In one embodiment, the test device can generate a test report for evaluating the voice processing link through a test report generation tool based on the evaluation of the ASR link, NLU link, and TTS link of the voice processing link, and send the test report to the user's terminal.

[0084] In one embodiment, the test report may include one or a combination of a word accuracy rate, a word error rate, and a sentence accuracy rate of a text data set output by an ASR link of the speech processing link.

[0085] In one embodiment, the test report may include the dialogue control algorithm, rule engine, and instruction information of the semantic understanding and semantic execution process corresponding to each text data in the text data set output by the NLU link, and the difference between the semantic understanding, semantic execution, dialogue control algorithm, rule engine, and instruction information annotated for each test case in the test case set.

[0086] In one embodiment, the test report may include the difference between each piece of feedback text data in the feedback text data set of the TTS link of the evaluation speech processing link and the expected feedback text data annotated by each test case in the test case set.

[0087] In one embodiment, the test device may send the test report to a user's terminal, such as a mobile phone of the user, through a communication tool.

[0088] In one embodiment, the test device may integrate the test report into the CI / CD (Continuous Integration / Continuous Deployment) process to improve the test efficiency of the voice processing link.

[0089] In one embodiment, the test report includes a test case pass rate and / or a field pass rate.

[0090] In one embodiment, a test case pass rate can be obtained based on the test cases in the test case set that pass the voice processing link, wherein the test case pass rate is the ratio of the number of test cases in the test case set that pass the voice processing link to the total number of test cases in the test case set.

[0091] In one embodiment, the test case pass rate W of the test cases in the test case set that pass the voice processing link can be calculated based on the number S1 of the test cases in the test case set that pass the voice processing link and the total number S0 of the test cases in the test case set. T1 , W T1 = S1 / S0.

[0092] In one embodiment, a domain pass rate can be obtained based on the test cases that pass the domain in the test case set, wherein the domain pass rate is the ratio of the number of test cases that pass the domain in the test case set to the total number of test cases in the domain in the test case set, and the domain corresponds to a function or module of the vehicle.

[0093] In one embodiment, the number of test cases in the test case set that pass the domain S can be calculated. L1 , the total number of test cases in this field in the test case set S L10 , calculate the domain pass rate W of the test cases that pass this domain in the test case set L1 , WL1 = S L1 / S L10 .

[0094] In one embodiment, the feedback audio data in the feedback audio data set can be converted into feedback text data; if the feedback text data is consistent with the expected feedback text data annotated by the test case in the test case set, the test case in the test case set is determined to be a test case that passes the voice processing link in the test case set.

[0095] In one embodiment, if the feedback text data is consistent with the expected feedback text data annotated on the test case in the test case set, then the test case in the test case set is determined to be a test case in the test case set that has passed the domain.

[0096] In one embodiment, the testing device can compare the test case pass rate of the current version of the voice processing link with the test case pass rate of the historical version of the voice processing link based on the test case pass rate of the voice processing link test report, and generate a quality comparison chart of the voice processing link to help users efficiently judge the quality of the current version of the voice processing link.

[0097] Step 208: Notify the domain manager corresponding to the domain of the failed test case of the failed test case.

[0098] In one embodiment, if the feedback text data is inconsistent with the expected feedback text data annotated on the test case in the test case set, the test case in the test case set is determined to be a failed test case.

[0099] In one embodiment, each test case in the test case set is marked with a corresponding field. The test device can search for the field person in charge of the failed test case in the field person in charge database according to the field corresponding to the failed test case and the corresponding relationship between the field and the field person in the field person in charge database, and send the failed test case to the field person in charge and / or the terminal of the test device user through a third-party communication tool (office software or chat software) in an expedited manner, so that the field person in charge and / or the test device user can obtain the test results of the voice processing link in a timely manner.

[0100] In one embodiment, each test case in the test case set is annotated with the corresponding expected log information. The test report generated by the test device may also include the actual log information when the ASR link, NLU link, and TTS link are verified. When the test device verifies the voice processing link, it can obtain the actual log information of each test case in the ASR link, NLU link, and TTS link processing test case set of the voice processing link; compare the actual log information of the failed test case with the expected log information annotated by the failed test case, and generate a failed test case highlight log information library. The failed test case highlight log information library can be a highlight failure information tool library, which compares and displays the actual log information of the failed test case with the expected log information annotated by the failed test case, and annotates and displays the error log information of the actual log information of the failed test case. The failed test case highlight log information library can facilitate the test device user and / or the neighborhood person in charge to quickly locate and intuitively understand the reasons for the failure of the test case, and can help the test device user and / or the neighborhood person in charge to fully understand the accuracy and reliability of the test results, and provide strong data support for the subsequent voice processing link algorithm optimization and voice processing link development.

[0101] In one embodiment, the log information may include, but is not limited to, the input information of each algorithm and each module called by the ASR link, NLU link, and TTS link according to each test case in the test case set, and the output information of each algorithm and each module.

[0102] In one embodiment, the testing device may count the failed fields and the number of failed fields of the field person in charge of the failed test case according to the field corresponding to the failed test case and the field person in charge corresponding to the field of the failed test case.

[0103] In one embodiment, if there are failed test cases in the test case set, the test device can continue to execute step 201, load the failed test cases, use the failed test cases to test the voice processing link, and use the failed test cases as new test cases to perform a new round of verification on the voice processing link.

[0104] For example, the test case set TS includes test case A "turn on the air conditioner", test case B "turn on the music", and test case C "turn off the music".

[0105] The ASR link outputs text data A1 corresponding to test case A, text data B1 corresponding to test case B, and text data C1 corresponding to test case C based on test case A, test case B, and test case C respectively.

[0106] The NLU link outputs the semantic understanding and semantic execution A2 corresponding to the text data A1, the semantic understanding and semantic execution B2 corresponding to the text data B1, and the semantic understanding and semantic execution C2 corresponding to the text data C1.

[0107] The TTS link outputs feedback audio data A3 corresponding to test case A, feedback audio data B3 corresponding to test case B, and feedback audio data C3 corresponding to test case C. The feedback audio data A3, feedback audio data B3, and feedback audio data C3 are converted into corresponding feedback text data, respectively, to obtain feedback text data A4 corresponding to feedback audio data A3, feedback text data B4 corresponding to feedback audio data B3, and feedback text data C4 corresponding to feedback audio data C3.

[0108] If the expected feedback text data A0 marked by the test case A "turn on the air conditioner" is consistent with the feedback text data A4, then the test case A "turn on the air conditioner" is determined to be a test case in the test case set that passes the voice processing link.

[0109] The domain of test case A "Turn on the air conditioner" is "Air conditioning". If the expected feedback text data A0 marked by test case A "Turn on the air conditioner" is consistent with the feedback text data A4, then test case A "Turn on the air conditioner" is determined to be a test case in the test case set that passes the domain "Air conditioning".

[0110] If the expected feedback text data A0 and feedback text data A4 marked by test case A "Turn on the air conditioner" are consistent, the expected feedback text data B0 and feedback text data B4 marked by test case B "Turn on the music" are inconsistent, and the expected feedback text data C0 and feedback text data C4 marked by test case C "Turn off the music" are inconsistent, the test case that passes the voice processing link in the test case set TS is only test case A "Turn on the air conditioner", and the number is 1. The voice processing link is verified using the test case set TS including test cases A, B, C, and a total of 3 test cases. The test case pass rate in the test report for evaluating the voice processing link is 1 / 3.

[0111] The domains of test case B "Turn on music" and test case C "Turn off music" are both "Music". If the expected feedback text data B0 and feedback text data B4 marked by test case B "Turn on music" are consistent, and the expected feedback text data C0 and feedback text data C4 marked by test case C "Turn off music" are inconsistent, the number of test cases that pass the domain "Music" in the test case set TS is only 1 for test case B "Turn on music", and the number of test cases in the domain "Music" in the test case set TS is 2. The voice processing link is verified using the test case set TS including test cases A, B, C, and the number of test cases is 3. The domain pass rate of the "Music" domain in the test report for evaluating the voice processing link is 1 / 2.

[0112] If the expected feedback text data A0 and the feedback text data A4 marked for test case A “turn on the air conditioner” are inconsistent, test case A “turn on the air conditioner” is determined to be a failed test case.

[0113] Test case A "Turn on the air conditioner", the ASR link of the voice processing link receives the audio data of test case A "Turn on the air conditioner", and the voice processing link processes the audio data of test case A "Turn on the air conditioner" in sequence through the ASR link, NLU link, and TTS link, and outputs the feedback audio data "The air conditioner is turned on". The test equipment converts the feedback audio data "The air conditioner is turned on" into the corresponding feedback text data through the voice processing link, and compares the feedback text data corresponding to the feedback audio data "The air conditioner is turned on" with the expected feedback text data annotated by test case A "Turn on the air conditioner". If the feedback text data corresponding to the feedback audio data "The air conditioner is turned on" is consistent with the expected feedback text data annotated by test case A "Turn on the air conditioner", then test case A "Turn on the air conditioner" is a test case that passes the field "air conditioner" in the test case set and a test case that passes the voice processing link in the test case set. If the feedback text data corresponding to the feedback audio data output by the voice processing link based on test case A "Turn on the air conditioner" is inconsistent with the expected feedback text data marked by test case A "Turn on the air conditioner", then test case A "Turn on the air conditioner" is a test case that has not passed the domain "air conditioner" in the test case set, and is a test case that has not passed the voice processing link in the test case set, and test case A "Turn on the air conditioner" is a failed test case.

[0114] The test method of the voice processing link of the embodiment of the present application performs different verifications on different links of the voice processing link in turn according to the test case set obtained after performing the preprocessing operation, wherein each link verification of different links depends on the output result of the previous link verification; the ASR link of the voice processing link is verified by using the test case set; the NLU link of the voice processing link is verified by using the ASR link based on the text data set output by the test case set; the TTS link of the voice processing link is verified by using the voice processing link based on the feedback audio data set fed back by the test case set; based on the verification of the ASR link, the NLU link and the TTS link, a test report for evaluating the voice processing link is generated and pushed; by integrating multiple disciplines such as computer science, artificial intelligence, and software engineering, through automation means and user The friendly visual interface can perform efficient and accurate full-link testing and verification of the core components of the voice processing link, such as the ASR link, NLU link, and TTS link, to achieve full-link verification of the voice processing link, ensure the effective coordination of each link in the voice processing link, and solve the problem of complex and incomplete voice processing link verification process in related technologies; it can fully verify the full link of the voice processing link, reduce the complexity of verifying the voice processing link, facilitate and fully verify the full link of the voice processing link, improve the efficiency of voice processing link verification, and more accurately locate the link where the problem occurs, so as to ensure the stability and reliability of the ASR link, NLU link, TTS link and other components of the voice processing link in actual applications, and improve the application effect of the voice processing link in the actual business of voice interaction.

[0115] Furthermore, the testing method of the voice processing link in the embodiment of the present application adopts a test case set to verify the ASR link of the voice processing link; adopts the ASR link to verify the NLU link of the voice processing link based on the text data set output by the test case set; adopts the voice processing link to verify the TTS link of the voice processing link based on the feedback audio data set feedback by the test case set; adopts different inputs to verify different links of the voice processing link (ASR link, NLU link, TTS link), can realize the use of multiple different threads to use different inputs to verify different links of the voice processing link, can process the test tasks of multiple different links at the same time, realize parallel processing of voice processing link tests, can significantly improve the test efficiency of the voice processing link, and shorten the test cycle.

[0116] Furthermore, the voice processing link test method of the embodiment of the present application introduces an automated test report push and precise notification function, establishes an automated test report push and precise notification mechanism, sends the test report to the user's terminal; notifies the field person in charge of the field of the failed test case of the failed test case; ensures that the voice processing link problem can be quickly fed back to the relevant person in charge, and improves the timeliness and efficiency of solving the voice processing link problem. In addition, by accurately corresponding to the person in charge of each field in the voice processing link or test case, the difficulty of tracking and solving the voice processing link problem is reduced.

[0117] Furthermore, the testing method of the voice processing link in the embodiment of the present application adopts a retry mechanism and a fault-tolerant mechanism, and adopts a failed test case to test the voice processing link, thereby achieving a "re-run" of the failed test case, and re-verifying the voice processing link using the failed test case, thereby ensuring the stability, accuracy and reliability of the verification of the voice processing link, and improving the application effect of the voice processing link in the actual business of voice interaction.

[0118] Furthermore, the test method of the voice processing link of the embodiment of the present application provides a user with a friendly visual interface, through which the voice parameters of the feedback audio data set of the voice processing link based on the feedback of the test case set are adjusted, wherein the voice parameters include one or more of sound quality, intonation, speech speed, and volume; the TTS link of the voice processing link is verified by using the feedback audio data set with adjusted voice parameters, which can enhance the user's experience of the voice processing link and enhance the personalization of the voice processing link. The voice parameters are obtained through the visual interface, and the feedback audio data set is adjusted according to the voice parameters. It can automatically generate corresponding audio data based on the case information and user options input by the user, integrating multiple test case types and multiple audio generation forms, thereby reducing the workload of users testing the voice processing link, providing great convenience for users to test the voice processing link, and improving the efficiency of testing the voice processing link.

[0119] Furthermore, the testing method of the voice processing link in the embodiment of the present application obtains test parameters input through a visual interface; loads a test case set according to the test parameters, and uses the test case set to verify the voice processing link. Through a highly integrated visual interface, the user does not need to frequently switch between multiple tools or platforms, and can easily complete the configuration of the test parameters of the voice processing link and the real-time monitoring of the test process.

[0120] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides a voice processing link test device, test equipment and corresponding embodiments.

[0121] Figure 4It is a structural schematic diagram of a test device for a voice processing link shown in an embodiment of the present application.

[0122] See also Figure 4 , a testing device for a voice processing link includes a preprocessing module 401 and a verification module 402.

[0123] The preprocessing module 401 is used to obtain a test case set obtained after performing a preprocessing operation according to the type of the test case set.

[0124] The verification module 402 is used to perform different verifications on different links of the voice processing link in turn according to the test case set obtained after the preprocessing operation is performed by the preprocessing module 401, wherein each link verification of different links depends on the output result of the previous link verification.

[0125] In one embodiment, the preprocessing module 401 may obtain the type of the test case set, and perform a preprocessing operation on the test case set according to the type of the test case set, so that the test case set obtained after the preprocessing operation meets the requirements of testing the voice processing link.

[0126] In one embodiment, the verification module 402 can input the test case set obtained after the preprocessing operation of the preprocessing module 401 into the ASR link of the voice processing link to verify the ASR link of the voice processing link; the ASR link is based on the text data set output by the test case set; and the ASR link is evaluated based on the text data set.

[0127] In one embodiment, the verification module 402 may input the text data set into the NLU link of the speech processing link to verify the NLU link; and evaluate the NLU link according to the dialogue control algorithm, rule engine, and instruction information of the NLU link processing the text data set.

[0128] In one embodiment, the verification module 402 can obtain a feedback audio data set of the speech processing link based on the test case set feedback; verify the TTS link of the speech processing link based on the feedback audio data set, and evaluate the TTS link of the speech processing link by comparing the feedback text data set with the expected feedback text data set annotated by the test case set.

[0129] In one embodiment, the verification module 402 may generate a test report for evaluating the speech processing link based on the evaluation of the ASR link, the NLU link, and the TTS link of the speech processing link.

[0130] The technical solution of the embodiment of the present application performs different verifications on different links of the voice processing link in turn according to the test case set obtained after performing the preprocessing operation, wherein each link verification of different links depends on the output result of the previous link verification; the ASR link of the voice processing link is verified by the ASR link based on the text data set output by the test case set; the TTS link of the voice processing link is verified by the voice processing link based on the feedback audio data set fed back by the test case set; by verifying the ASR link-NLU link-TTS link of the voice processing link, the full-link verification of the voice processing link is realized, the effective coordination of each link in the voice processing link is ensured, and the problem of complex and incomplete voice processing link verification process in related technologies is solved; the full-link of the voice processing link can be fully verified, the complexity of verifying the voice processing link is reduced, the link where the problem occurs can be more accurately located, the accuracy and stability of the entire voice processing link can be ensured, and the application effect of the voice processing link in the actual voice interaction business is improved.

[0131] Figure 5 It is a structural schematic diagram of a test device for a voice processing link shown in an embodiment of the present application.

[0132] See also Figure 5 , a testing device for a voice processing link includes a preprocessing module 401, a verification module 402, a parameter acquisition module 501, a loading module 502, a report generation module 503, and a push module 504.

[0133] The parameter acquisition module 501 is used to acquire the test parameters input through the visual interface.

[0134] The loading module 502 is used to load the test case set according to the test parameters obtained by the parameter obtaining module 501.

[0135] In one embodiment, the verification module 402 is used to use the test case set obtained after the preprocessing module 401 performs the preprocessing operation to verify the ASR link of the speech processing link and obtain an output text data set; verify the NLU link of the speech processing link based on the text data set; obtain the feedback audio data set; and verify the TTS link of the speech processing link based on the feedback audio data set.

[0136] In one embodiment, the loading module 502 can load a failed test case. The verification module 402 uses the failed test case loaded by the loading module 502 to test the voice processing link.

[0137] The report generation module 503 is used to generate a test report for evaluating the speech processing link according to the verification of the ASR link, NLU link, and TTS link of the speech processing link by the verification module 402.

[0138] In one embodiment, the report generation module 503 may compare the actual log information of the failed test case with the expected log information marked by the failed test case, and generate a failed test case highlighted log information library.

[0139] The push module 504 is used to send the test report to the user's terminal.

[0140] In one embodiment, the push module 504 may notify the domain manager corresponding to the domain of the failed test case of the failed test case.

[0141] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0142] Figure 6 It is a schematic diagram of the structure of the test equipment shown in the embodiment of the present application.

[0143] See also Figure 6 , the testing device 1000 includes a memory 1010 and a processor 1020 .

[0144] The processor 1020 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0145] The memory 1010 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 1020 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 1010 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 1010 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0146] The memory 1010 stores executable codes, and when the executable codes are processed by the processor 1020 , the processor 1020 can execute part or all of the methods described above.

[0147] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0148] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-temporary machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of a test device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0149] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for testing a voice processing link, characterized in that: include: Obtain a test case set obtained after performing a preprocessing operation according to the type of the test case set; According to the test case set obtained after performing the preprocessing operation, different verifications are performed on different links of the voice processing link in turn, wherein each link verification of the different links depends on the output result of the previous link verification.

2. The method according to claim 1, characterized in that The test case set obtained after performing the preprocessing operation is sequentially verified on different links of the voice processing link, wherein each link verification of the different links depends on the output result of the previous link verification, including: Using the test case set obtained after performing the preprocessing operation, the ASR link of the speech processing link is verified to obtain an output text data set; Verify the NLU link of the speech processing link according to the text data set to obtain a feedback audio data set; The TTS link of the speech processing link is verified according to the feedback audio data set.

3. The method according to claim 2, characterized in that The test case set obtained after performing the preprocessing operation is used to verify the ASR link of the speech processing link to obtain an output text data set, including: Obtaining a text data set output by the ASR link of the speech processing link according to the test case set obtained after performing the preprocessing operation; The ASR link of the speech processing link is evaluated by one or a combination of the character accuracy rate, word error rate, and sentence accuracy rate of the text data set.

4. The method according to claim 2, characterized in that: The verifying the NLU link of the speech processing link according to the text data set includes: Obtaining the dialogue control algorithm, rule engine, and instruction information of the NLU link in processing the text data set; The NLU link of the speech processing link is evaluated by comparing the conversation control algorithm, rule engine, and instruction information of the NLU link in processing the text data set with the expected conversation control algorithm, expected rule engine, and expected instruction information annotated by the test case set.

5. The method according to claim 2, characterized in that: The verifying the TTS link of the speech processing link according to the feedback audio data set includes: Acquire speech parameters through a visual interface, and adjust the feedback audio data set according to the speech parameters, wherein the speech parameters include one or more of sound quality, intonation, speech speed, and volume; The TTS link of the speech processing link is verified using a feedback audio data set for adjusting speech parameters.

6. The method according to claim 5, characterized in that The verifying the TTS link of the speech processing link by using the feedback audio data set for adjusting speech parameters includes: Convert the feedback audio dataset of adjusting speech parameters into a feedback text dataset; The TTS link of the speech processing link is evaluated by comparing the feedback text dataset with the expected feedback text dataset annotated by the test case set.

7. The method according to claim 2, characterized in that The method further comprises: Generating a test report for evaluating the speech processing link according to the verification of the ASR link, the NLU link, and the TTS link; The test report is sent to the user's terminal.

8. The method according to claim 7, characterized in that The test report includes the test case pass rate and / or field pass rate; wherein, According to the test cases in the test case set that pass the voice processing link, the test case pass rate is obtained, wherein the test case pass rate is the ratio of the number of test cases in the test case set that pass the voice processing link to the total number of test cases in the test case set; and / or, According to the test cases that have passed the field in the test case set, the field pass rate is obtained, wherein the field pass rate is the ratio of the number of test cases that have passed the field in the test case set to the total number of test cases in the field in the test case set, and the field corresponds to a function or module of the vehicle.

9. The method according to claim 8, characterized in that The obtaining the test case pass rate according to the test cases in the test case set that pass the voice processing link also includes: Converting the feedback audio data in the feedback audio data set into feedback text data; If the feedback text data is consistent with the expected feedback text data annotated by the test case in the test case set, determining that the test case in the test case set is a test case in the test case set that passes the voice processing link; and / or, Obtaining a pass rate of a field according to the test cases in the test case set that pass the field, further comprising: Converting the feedback audio data in the feedback audio data set into feedback text data; If the feedback text data is consistent with the expected feedback text data annotated on the test case in the test case set, it is determined that the test case in the test case set is a test case in the passed field in the test case set.

10. A test device for a voice processing link, characterized in that: include: A preprocessing module, used to obtain a test case set obtained after performing a preprocessing operation according to the type of the test case set; The verification module is used to perform different verifications on different links of the voice processing link in turn according to the test case set obtained after the preprocessing operation is performed by the preprocessing module, wherein each link verification of the different links depends on the output result of the previous link verification.

11. A testing device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: Executable codes are stored thereon, and when the executable codes are executed by a processor of a test device, the processor is caused to execute the method according to any one of claims 1 to 9.

Images