Speech recognition test method and device, computer equipment and speech recognition test system

Abstract

The invention discloses a voice recognition test method and device, computer equipment and a voice recognition test system. Comprising the following steps: controlling a player to play a test voice of a test text so as to enable a voice recognition device to recognize the test voice, and displaying a recognition text of the test voice on a UI interface; a recognition text of the voice recognition device for the test voice is screened from operation information fed back by the voice recognition device in the connected state, and the operation information comprises UI interface elements and / or log files; and comparing the test text with the recognition text, and determining a test result of the voice recognition equipment. According to the embodiment of the invention, based on automatic voice playing and recognition text extraction, the recognition result of the voice recognition equipment can be obtained without manual intervention, and then automatic ASR testing is realized. Not only is the workload of voice technicians for checking the voice recognition accuracy reduced, but also the accuracy of recognition result evaluation of the voice recognition equipment is ensured.

Description

Technical Field

[0001] The present application relates to the field of speech recognition technology, and in particular to a speech recognition test method, device, computer equipment and speech recognition test system. Background Art

[0002] With the development of technology, smart devices with new interactive modes of voice assistants are widely used, such as smart phones, smart watches, smart speakers, smart home devices, learning machines, etc. Automatic speech recognition (ASR) technology is one of the key technologies of voice assistants. In the relevant testing technology of the technical step of converting the voice recognized by the voice assistant into text,

[0003] In the related art, for voice assistant devices with UI interfaces, that is, smart devices that users can see the text recognized and converted by the voice assistant on the device screen during use, testers are usually required to dictate. After the voice assistant recognizes, it converts the voice into text and displays it. The tester records the text on the screen, counts the number of correct words / incorrect words, and finally calculates the accuracy of the voice assistant. This method is time-consuming, inefficient, and prone to errors. If it is a voice assistant device without a UI interface, the recognition accuracy cannot be tested. Summary of the invention

[0004] In view of this, the present application provides a speech recognition test method, apparatus, computer equipment and speech recognition test system to solve the problems described in the background technology.

[0005] According to one aspect of the present application, a speech recognition test method is provided, comprising:

[0006] Control the player to play the test voice of the test text, so that the voice recognition device recognizes the test voice and displays the recognized text of the test voice on the UI interface;

[0007] Filtering the recognition text of the test voice by the voice recognition device from the operation information fed back by the voice recognition device in the connected state, wherein the operation information includes UI interface elements and / or log files;

[0008] The test text and the recognition text are compared to determine the test result of the speech recognition device.

[0009] Further, the filtering of the recognition text of the test voice by the voice recognition device from the operation information fed back by the voice recognition device in the connected state comprises:

[0010] Filtering the recognition text of the test speech by the speech recognition device from the UI interface elements according to the text attributes; and / or,

[0011] The log file is matched according to a preset format, and the recognized text in the log file matching the preset format is extracted.

[0012] Furthermore, the speech recognition test method further comprises:

[0013] If the recognition text corresponding to the test text is not obtained within the preset time interval, the player is controlled to repeatedly play the test voice until the recognition text is obtained or the number of times the test voice is played is greater than or equal to the preset number of times corresponding to the test text.

[0014] Furthermore, the number of the test texts is multiple, and the controlling player to play the test voice of the test texts includes:

[0015] If the recognition text corresponding to the current test text is obtained or the number of times the current test voice is played is greater than or equal to the preset number of times corresponding to the current test text, the player is controlled to play the test voice of the next test text.

[0016] Furthermore, the speech recognition test method further comprises:

[0017] Periodically detecting status information of the speech recognition device;

[0018] If the voice recognition device is in a screen-off state, sending a wake-up command to the voice recognition device;

[0019] If the speech recognition device is in a non-debugging mode, output a first prompt message;

[0020] If the voice recognition device is in a disconnected state, stop controlling the player to play the test voice of the test text and output a second prompt message.

[0021] Furthermore, comparing the test text with the recognition text to determine the test result of the speech recognition device includes:

[0022] Comparing the test text with the recognized text, determining the number of correct text characters, the number of deleted erroneous characters, the number of inserted erroneous characters, and the number of replaced erroneous characters between the test text and the recognized text;

[0023] Determining a text recognition rate of the test text according to the number of correct text characters, the number of deleted erroneous characters, the number of inserted erroneous characters and / or the number of replaced erroneous characters;

[0024] Performing weighted processing on the text recognition rate of at least one of the test texts according to the weight value of the test text, and calculating the recognition score of the speech recognition device;

[0025] The test result is determined based on a comparison result between the recognition score and a preset scoring standard.

[0026] Furthermore, the speech recognition test method further comprises:

[0027] Determining the recognition difficulty level of the test text according to the number of correct text characters and the similarity of speech features between adjacent characters in the test text;

[0028] The weight value is matched according to the recognition difficulty level.

[0029] Furthermore, the speech recognition test method further comprises:

[0030] In response to a test instruction for a speech recognition device, loading a configuration file carried by the test instruction, wherein the configuration file includes a test text;

[0031] The test speech of the test text is generated by speech synthesis technology.

[0032] Furthermore, the speech recognition test method further comprises:

[0033] Adding the test text, the recognized text, the text recognition rate and the test result to a table template according to a preset format to form a test report of the speech recognition device;

[0034] The test report is sent to a user device associated with the speech recognition device.

[0035] According to another aspect of the present application, a speech recognition test device is provided, comprising:

[0036] A control module, used to control the player to play the test voice of the test text, so that the voice recognition device recognizes the test voice and displays the recognized text of the test voice on the UI interface;

[0037] An acquisition module, used for filtering the recognition text of the test voice by the voice recognition device from the operation information fed back by the voice recognition device in a connected state, wherein the operation information includes UI interface elements and / or log files;

[0038] The test module is used to compare the test text with the recognition text to determine the test result of the speech recognition device.

[0039] Furthermore, the acquisition module is specifically used to filter the recognition text of the test voice by the voice recognition device from the UI interface elements according to text attributes.

[0040] Furthermore, the acquisition module is specifically used to match the log file according to a preset format, and extract the recognition text in the log file matching the preset format.

[0041] Furthermore, the control module is also used to control the player to repeatedly play the test voice if the recognition text corresponding to the test text is not obtained within a preset time interval until the recognition text is obtained or the test voice is played more than or equal to the preset number of times corresponding to the test text.

[0042] Furthermore, there are multiple test texts, and the control module is specifically used to control the player to play the test voice of the next test text if the recognition text corresponding to the current test text is obtained or the number of times the current test voice is played is greater than or equal to the preset number of times corresponding to the current test text.

[0043] Furthermore, the speech recognition test device also includes:

[0044] A status detection module, used for periodically detecting status information of the speech recognition device;

[0045] An exception handling module is used to send a wake-up instruction to the voice recognition device if the voice recognition device is in a screen-off state; and if the voice recognition device is in a non-debugging mode, output a first prompt message; and if the voice recognition device is in a disconnected state, stop controlling the player to play the test voice of the test text and output a second prompt message.

[0046] Furthermore, the test module is specifically used to compare the test text and the recognized text, determine the number of correct text characters, the number of deleted error characters, the number of inserted error characters and the number of replaced error characters between the test text and the recognized text; determine the text recognition rate of the test text according to the number of correct text characters, the number of deleted error characters, the number of inserted error characters and / or the number of replaced error characters; perform weighted processing on the text recognition rate of at least one of the test texts according to the weight value of the test text, and calculate the recognition score of the speech recognition device; determine the test result according to the comparison result between the recognition score and the preset scoring standard.

[0047] Furthermore, the test module is also used to determine the recognition difficulty level of the test text based on the number of correct text characters and the similarity of speech features between adjacent characters in the test text; and match the weight value according to the recognition difficulty level.

[0048] Furthermore, the speech recognition test device also includes:

[0049] A report generation module, used for adding the test text, the recognition text, the text recognition rate and the test result to a table template according to a preset format to form a test report of the speech recognition device;

[0050] The communication module is used to send the test report to the user equipment associated with the speech recognition device.

[0051] Furthermore, the speech recognition test device also includes:

[0052] A configuration module, configured to load a configuration file carried by a test instruction for a speech recognition device in response to the test instruction, wherein the configuration file includes a test text;

[0053] A speech synthesis module is used to generate the test speech of the test text through speech synthesis technology.

[0054] According to another aspect of the present application, a computer-readable storage medium is provided, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the above-mentioned speech recognition test method are implemented.

[0055] According to another aspect of the present application, a computer device is provided, comprising at least one processor, the processor being coupled to a memory, the memory storing a computer program running on the processor, characterized in that the processor implements the steps of the above-mentioned speech recognition test method when executing the program.

[0056] According to yet another aspect of the present application, there is provided a speech recognition test system, comprising a player, the player being arranged at a position at a preset distance from a speech recognition device;

[0057] A noise generator, the noise generator is arranged around the speech recognition device, and the noise generator is used to generate background noise;

[0058] The computer device in the above embodiment is a computer device, and the computer device is respectively connected to the player and the speech recognition device for communication, and the computer device executes the steps of the above speech recognition test method when executing a program.

[0059] By means of the above technical solution, the test voice is played in advance according to the set test text, and the system obtains the recognition text of the test voice by the voice recognition device by reading the operation information of the voice recognition device. The ASR test result of the voice recognition device is obtained by comparing the difference between the test text and the recognition text. In this way, based on the automated voice playback and recognition text extraction, the recognition result of the voice recognition device can be obtained without human intervention, thereby realizing automated ASR testing. This not only reduces the workload of voice technicians in verifying the accuracy of voice recognition, but also ensures the accuracy of the evaluation of the recognition results of the voice recognition device.

[0060] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0062] Figure 1 A schematic diagram of the structure of a speech recognition test system provided in an embodiment of the present application is shown;

[0063] Figure 2 A flow chart of a speech recognition test method provided in an embodiment of the present application is shown;

[0064] Figure 3 The structural block diagram of the speech recognition test device provided in an embodiment of the present application is shown.

[0065] Reference numerals:

[0066] 110 player, 120 noise generator, 130 computer device, 140 speech recognition device. DETAILED DESCRIPTION

[0067] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0068] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0069] Those skilled in the art will appreciate that, unless expressly stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, or there may be intermediate elements. In addition, the "connection" or "connection" used herein may include wireless connection or wireless fusion. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0070] Now, exemplary embodiments according to the present application will be described in more detail with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in a variety of different forms and should not be interpreted as being limited to the embodiments set forth herein. It should be understood that these embodiments are provided to make the disclosure of the present application thorough and complete, and to fully convey the concepts of these exemplary embodiments to those of ordinary skill in the art.

[0071] Figure 1 A speech recognition test system is provided for the present application. In the process of testing the speech recognition function of a speech recognition device 140 with a UI interface, the scheme of the present application is applied for testing. Specifically, the speech recognition test system includes a player 110, a noise generator 120, and a computer device 130. The player 110 is arranged at a preset distance from the speech recognition device 140. The noise generator 120 is arranged around the speech recognition device 140 to generate background noise. The computer device 130 is respectively connected to the player 110 and the speech recognition device 140 for communication, and together implements the speech recognition test method of the present application, so as to achieve the purpose of accurately and automatically evaluating the speech recognition ability of the speech recognition device.

[0072] It is understandable that the computer device 130 and the voice recognition device 140 can be connected to each other by means of a USB data cable, Bluetooth, WIFI, etc. At the same time, the debugging mode of the voice recognition device 140 needs to be turned on so that the computer device 130 can successfully access the voice recognition device 140 and obtain relevant data. The player 110 can be independently arranged outside the computer device 130, or it can be integrated into the computer device 130, for example, a computer speaker.

[0073] It is worth mentioning that the preset distance can be reasonably set according to the commonly used setting position of the voice recognition device. For example, the preset distance of a smart watch can be set to 10cm~50cm, and the preset distance of a smart TV can be set to 1m~3m, so as to simulate the voice reception distance of the voice recognition device in daily use scenarios, so that the decibel of the received sound is about 60dB. The noise value generated by the noise generator can be reasonably set according to the application scenario. For example, the noise value in the outdoor scene is greater than the noise value in the indoor scene, so that there is daily noise around the voice recognition device for better environmental simulation.

[0074] The specific implementation of the speech recognition test method can refer to the following embodiments.

[0075] In this embodiment, a speech recognition test method is provided, which is described by taking a computer device as the execution subject. Figure 2 As shown, the method includes:

[0076] Step 201, controlling the player to play the test voice of the test text.

[0077] It should be noted that the test voice can be audio pre-recorded by the user, or can be voice synthesized from a test text.

[0078] In this embodiment, the test text is used to automatically control the player to broadcast the test voice. After receiving the test voice, the voice recognition device will call the automatic voice recognition module to recognize the test voice and display the recognition text of the test voice on the UI interface, thereby triggering the automatic voice recognition function of the voice recognition device.

[0079] It is understandable that the number of speech recognition devices can be one or more. By broadcasting the test voice through the player, the test work of multiple speech recognition devices can be completed at the same time, and the test efficiency is higher. The number of test texts can be one or more, specifically, it can be one or more sentences or one or more paragraphs, which is not specifically limited in the embodiments of the present application. For example, "What time is it now", "What's the weather like today", "What day is today", etc.

[0080] In a specific application scenario, when there are multiple test texts, step 201, that is, controlling the player to play the test voice of the test text, specifically includes the following steps: if the recognition text corresponding to the current test text or the number of times the current test voice is played is greater than or equal to the preset number of times corresponding to the current test text, control the player to play the test voice of the next test text.

[0081] In this embodiment, if different test voices are played at a uniform preset time interval by default, a certain sentence may be recognized but the next voice needs to wait until the preset time interval ends, or the playback stops directly after repeated playback for a preset number of times. In order to save test time, the system can control the player to play the test voice of the next test text when the recognition text corresponding to the current test text is obtained or the number of times the current test voice is played is greater than or equal to the preset number of times corresponding to the current test text. This ensures that the recognition text is obtained, improves the effectiveness and accuracy of the test, and can also achieve the continuity of voice playback, thereby improving the efficiency of the device voice recognition test.

[0082] The preset number of times and the preset time interval can be adjusted according to different test texts and environments.

[0083] In one embodiment, before step 201, the speech recognition test method further includes: in response to a test instruction to the speech recognition device, loading a configuration file carried by the test instruction; and generating a test speech of the test text by using speech synthesis technology.

[0084] The configuration file includes a test text. If there are multiple test texts, the configuration file may also include a preset time interval between the multiple test texts to ensure that the speech recognition device has sufficient time to recognize the current test speech.

[0085] In this embodiment, the computer device responds to the user's test instruction to the speech recognition device, loads the configuration file carried by the test instruction, and performs ASR test configuration. After completing the ASR test configuration, the test speech is generated by using speech synthesis technology with reference to the test text. Thus, the automated test text speech synthesis and automatic audio playback functions are completed.

[0086] Furthermore, the text-to-speech (TTS) technology may adopt the technology in the prior art, which will not be described in detail in the embodiments of the present application.

[0087] Furthermore, the voice sources for speech synthesis can use male and female voices of different ages, and the test corpus text can also be converted into multiple dialects, such as male-Mandarin-children's voices, female-Sichuan dialect-adult voices, etc. Therefore, by converting the test corpus text into male and female voices of different ages and multiple dialects, the test environment is closer to the actual application scenario, and the credibility of the test results is improved.

[0088] Step 202: Filter the recognition text of the test speech by the speech recognition device from the operation information fed back by the speech recognition device in the connected state.

[0089] Among them, the operation information includes UI interface elements and / or log files. UI interface elements refer to a series of elements that meet the user interaction requirements contained in the software or system interface that can meet the interaction requirements. For example, text, graphics (cursor, button), menu, window, etc. Different UI interface elements have different properties. Log files refer to text files automatically generated by voice recognition devices, which are used to record events, operations, errors, warnings and other important information.

[0090] In a specific application scenario, step 202 may be implemented as follows:

[0091] Method 1: Filter the recognition text of the test voice by the voice recognition device from the UI interface elements according to the text attribute.

[0092] In this embodiment, the UI interface elements of the speech recognition device that has established a connection with the computer device can be captured through the UI (User Interface) automated testing framework. After the speech recognition device recognizes the test speech, a text view will pop up on the UI interface to display the converted recognition text, and different UI interface elements have different attributes. The recognition text of the speech recognition device for the test speech can be filtered out from the UI interface elements of the speech recognition device according to the text attributes.

[0093] Method 2: matching the log files according to a preset format and extracting the recognition text in the log files matching the preset format.

[0094] In this embodiment, for the situation where the recognized text cannot be extracted 100% from the UI interface elements due to display failure or rapid changes in UI interface elements, and for voice recognition devices that do not have a UI interface. Since the voice recognition device will print a log after recognizing the voice, the system can read the log file of the voice recognition device and automatically extract the text recognized by the voice assistant from the log file according to the preset format related to voice recognition. This solves the problem that UI interface elements sometimes fail to be obtained, and ensures that the voice recognition device can smoothly perform voice recognition testing. In addition, automated voice recognition testing can also be implemented for voice recognition devices that do not have a UI interface.

[0095] In one embodiment, the voice recognition test method also includes: periodically detecting the status information of the voice recognition device; if the voice recognition device is in a screen-off state, sending a wake-up command to the voice recognition device; if the voice recognition device is in a non-debugging mode, outputting a first prompt message; if the voice recognition device is in a disconnected state, stopping controlling the player to play the test voice of the test text, and outputting a second prompt message.

[0096] In this embodiment, after the test system is started, the computer device periodically detects the status information of the voice recognition device to ensure that the voice recognition device can maintain the ability to obtain data during the test. Specifically, if it is detected that the voice recognition device is in a screen-off state, that is, the voice recognition device screen will not display the recognized text, at this time, the computer device sends a wake-up instruction to the voice recognition device to wake up the voice recognition device to ensure that the system can extract UI interface elements. If the voice recognition device is in a non-debugging mode, the voice recognition device may close the access rights of certain functions at this time, so that the computer device cannot successfully access all the data of the voice recognition device, then output a first prompt message to prompt the staff to confirm or adjust the voice recognition device mode. If it is detected that the voice recognition device is in a disconnected state, the computer device will not be able to obtain data from the voice recognition device at this time, then stop controlling the player to play the test voice of the test text, and output a second prompt message to prompt the staff to connect the two in time, and play the test voice of the test text again after the connection is restored. It can be understood that in the case of testing multiple voice recognition devices at the same time, only when all voice recognition devices are in a disconnected state will the player stop playing the test voice, ensuring that other voice recognition devices in a connected state can continue to perform ASR testing.

[0097] Step 203, comparing the test text and the recognition text to determine the test result of the speech recognition device.

[0098] The speech recognition test method provided in the embodiment of the present application plays the test speech in advance according to the set test text, and the system obtains the recognition text of the test speech by the speech recognition device by reading the operation information of the speech recognition device. The ASR test result of the speech recognition device is obtained by comparing the difference between the test text and the recognition text. In this way, based on the automated speech playback and recognition text extraction, the recognition result of the speech recognition device can be obtained without human intervention, thereby realizing automated ASR testing. This not only reduces the workload of speech technicians in verifying the accuracy of speech recognition, but also ensures the accuracy of the evaluation of the recognition results of the speech recognition device.

[0099] In a specific application scenario, step 203, that is, comparing the test text and the recognition text to determine the test result of the speech recognition device, specifically includes the following steps:

[0100] Step 203-1, compare the test text and the recognition text, determine the number of correct text characters, the number of deleted erroneous characters, the number of inserted erroneous characters and the number of replaced erroneous characters between the test text and the recognition text.

[0101] Among them, the number of correct text characters refers to the number of characters in the test text, deleting error characters refers to the text characters missing compared to the correct text characters, inserting error characters refers to the redundant text characters compared to the correct text characters, and replacing error characters refers to replacing the correct text characters with other text characters.

[0102] Step 203-2, determining the text recognition rate of the test text according to the number of correct text characters, the number of deleted erroneous characters, the number of inserted erroneous characters and / or the number of replaced erroneous characters.

[0103] Assuming that the number of correct text characters is M, the number of deleted error characters is Dc, the number of inserted error characters is Ic, and the number of replaced error characters is Sc, the text recognition rate calculation formula is:

[0104] Substitution error rate Ser = (Sc / M) × 100%;

[0105] Insertion error rate Ier = (Ic / M) × 100%;

[0106] Deletion error rate Der = (Dc / M) × 100%;

[0107] Word error rate Cer = ((Sc + Ic + Dc) / M) × 100%;

[0108] Text recognition rate Ccr=100%-Cer.

[0109] Step 203-3: weighting the text recognition rate of at least one test text according to the weight value of the test text to calculate the recognition score of the speech recognition device.

[0110] Specifically, the recognition difficulty level of the test text is determined according to the number of correct text characters and the similarity of speech features between adjacent characters in the test text; and the weight value is matched according to the recognition difficulty level. The corresponding relationship between the number of correct text characters, the similarity of speech features between adjacent characters in the test text, the recognition difficulty level and the weight value can be established in advance based on experience, so as to match the appropriate weight value. For example, the greater the number of correct text characters, the higher the recognition difficulty level, and the greater the weight value matched. The more adjacent characters with high speech feature similarity there are, the more character groups with similar pronunciation but different words there are, and the more difficult it is to distinguish. The higher the difficulty level, the greater the weight value matched.

[0111] In this embodiment, the text recognition rate of the speech recognition device for at least one different test text is determined based on the difference between the test text and the recognition text. The text recognition rates of the speech recognition device for different test texts are weighted according to the weight values ​​of the test texts, thereby quantifying the complexity of different test texts, so that the final recognition score can balance the overall evaluation indicators, effectively avoid errors caused by simple recognition rate calculations, help improve the accuracy of the speech recognition device's recognition evaluation, and thus improve the credibility of the test results.

[0112] Step 203-4, determining the test result according to the comparison result between the recognition score and the preset scoring standard.

[0113] Among them, the preset scoring standard can be reasonably set according to the test accuracy, and this embodiment does not make any specific limitation.

[0114] In this embodiment, by comparing the recognition score with the preset standard, the actual performance of the speech recognition device can be clearly evaluated, and its performance in a specific task can be understood to obtain objective test results, avoid errors caused by human subjective judgment, and further improve the accuracy of the device speech recognition result evaluation.

[0115] In one embodiment, the speech recognition test method also includes: if the recognition text corresponding to the test text is not obtained within a preset time interval, controlling the player to repeatedly play the test voice until the recognition text is obtained or the number of times the test voice is played is greater than or equal to the preset number of times corresponding to the test text.

[0116] In this embodiment, if the recognition text corresponding to the test text is not obtained within the preset time interval of controlling the playback of the test voice, the player is controlled to play the same test voice again until the recognition text is obtained or the number of playbacks of the test voice is greater than or equal to the preset number of playbacks corresponding to the test text. This ensures that when the recognition result is not obtained, the system can automatically process and ensure that the automated ASR test can be successfully completed.

[0117] For example, Figure 1 Take the test system in as an example. The speech recognition test system has built-in different types of test corpus texts. These test texts are maintained in the configuration file. Before the automated test is run, the system will first perform speech synthesis on the test corpus text to generate the corresponding audio file and store it in the test data set directory. When the automated test is running, the test data set directory will be traversed and played in a loop through the audio.

[0118] Step 2: The system uses the Python playsound module to play the test text. After playing one line, it waits for 1s to 3s (preset time interval) to automatically obtain the text data converted by the ASR speech recognition device and record it. The detailed steps are as follows:

[0119] (1) Play the test corpus: After playing, wait for the voice recognition device to recognize the voice. This process takes about 1s to 3s. This time is maintained in the configuration file and can be easily modified. At the same time, this automation system provides some common tools and methods, such as automatically turning on the screen when the voice recognition device is detected to be off, and automatically stopping the program and giving an error prompt when the voice recognition device is detected to be shut down due to insufficient battery.

[0120] (2) Obtain the recognized text after ASR speech conversion: The system is compatible with speech recognition devices with UI interfaces and speech recognition devices without UI interfaces. Generally, UI interface elements will be given priority. If they cannot be obtained, they will be obtained through logs. UI interface elements are obtained through the UI interface element acquisition module of this system. Specifically, when the speech recognition device recognizes the speech, a text box will pop up on the UI interface to display the converted text. The system can locate the element through uiautomator2 and obtain the text attribute, that is, the text is obtained. Logs are obtained through the system's log extraction and analysis module. Specifically, when the speech recognition device recognizes the speech, the log will be printed. The system can obtain the log through abdlogcat|grep'asrContent', format the log, and retain the text.

[0121] (3) If the recognition text is successfully obtained, it is recorded and the next test text voice is played; if the acquisition fails, the playback of this test text voice is repeated and the recognition text is obtained again.

[0122] Step 3: After the system loops through all the test texts, it automatically executes a python script to calculate the recognition rate, obtain the final result, and save it in an Excel file in a certain format. The Excel file not only has the accuracy value, but also lists the result records of each test corpus, including the original text, recognized text, text recognition rate, sentence accuracy, etc. The system will also send this Excel file via email as a report notification.

[0123] It should be noted that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0124] Furthermore, if Figure 3 As shown, as a specific implementation of the above-mentioned speech recognition test method, an embodiment of the present application provides a speech recognition test device 300, and the speech recognition test device 300 includes: a control module 301, an acquisition module 302, and a test module 303.

[0125] The control module 301 is used to control the player to play the test voice of the test text, so that the voice recognition device recognizes the test voice and displays the recognized text of the test voice on the UI interface;

[0126] The acquisition module 302 is used to filter the recognition text of the test speech by the speech recognition device from the operation information fed back by the speech recognition device in the connected state, wherein the operation information includes UI interface elements and / or log files;

[0127] The test module 303 is used to compare the test text and the recognition text to determine the test result of the speech recognition device.

[0128] Furthermore, the acquisition module 302 is specifically used to filter the recognition text of the test voice by the voice recognition device from the UI interface elements according to the text attributes.

[0129] Furthermore, the acquisition module 302 is specifically configured to match the log file according to a preset format and extract the recognition text in the log file matching the preset format.

[0130] Furthermore, the control module 301 is also used to control the player to repeatedly play the test voice if the recognition text corresponding to the test text is not obtained within a preset time interval until the recognition text is obtained or the number of times the test voice is played is greater than or equal to the preset number of times corresponding to the test text.

[0131] Furthermore, there are multiple test texts, and the control module 301 is specifically used to control the player to play the test voice of the next test text if the recognition text corresponding to the current test text or the number of times the current test voice is played is greater than or equal to the preset number of times corresponding to the current test text.

[0132] Furthermore, the speech recognition testing device 300 also includes: a status detection module (not shown in the figure), which is used to periodically detect the status information of the speech recognition device; an exception handling module (not shown in the figure), which is used to send a wake-up instruction to the speech recognition device if the speech recognition device is in a screen-off state; and, if the speech recognition device is in a non-debugging mode, output a first prompt message; and, if the speech recognition device is in a disconnected state, stop controlling the player to play the test voice of the test text, and output a second prompt message.

[0133] Furthermore, the test module 303 is specifically used to compare the test text and the recognized text, determine the number of correct text characters, the number of deleted error characters, the number of inserted error characters and the number of replaced error characters between the test text and the recognized text; determine the text recognition rate of the test text based on the number of correct text characters, the number of deleted error characters, the number of inserted error characters and / or the number of replaced error characters; perform weighted processing on the text recognition rate of at least one test text according to the weight value of the test text, and calculate the recognition score of the speech recognition device; determine the test result based on the comparison result between the recognition score and the preset scoring standard.

[0134] Furthermore, the test module 303 is also used to determine the recognition difficulty level of the test text according to the number of correct text characters and the similarity of speech features between adjacent characters in the test text; and match the weight value according to the recognition difficulty level.

[0135] Furthermore, the speech recognition testing device 300 further includes:

[0136] A report generation module (not shown in the figure), which is used to add the test text, recognition text, text recognition rate and test results to the table template according to a preset format to form a test report of the speech recognition device;

[0137] A communication module (not shown in the figure), the communication module is used to send the test report to the user equipment associated with the voice recognition device.

[0138] Furthermore, the speech recognition testing device 300 further includes:

[0139] A configuration module (not shown in the figure), the configuration module is used to respond to the test instruction of the speech recognition device and load the configuration file carried by the test instruction, wherein the configuration file includes the test text;

[0140] A speech synthesis module (not shown in the figure) is used to generate a test speech of a test text by using speech synthesis technology.

[0141] For the specific definition of the speech recognition test device, please refer to the definition of the speech recognition test method above, which will not be repeated here. Each module in the above-mentioned speech recognition test device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0142] Based on the above Figure 2 The method shown in the embodiment of the present application accordingly provides a readable storage medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned Figure 2The speech recognition test method shown.

[0143] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0144] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages and saves the hardware and software resources of the computer device, and supports the operation of information processing programs and other software and / or programs. The network communication module is used to realize communication between the components inside the storage medium, and communication with other hardware and software in the physical device.

[0145] Based on the above Figure 2 The method shown, and Figure 3 In order to achieve the above-mentioned purpose, the embodiment of the present application further provides a computer device, which can be a personal computer, a server, a network device, a vehicle-mounted device, etc. The computer device includes at least one processor; the processor is coupled to a memory, and the memory stores a computer program running on the processor; the processor executes the computer program to achieve the above-mentioned Figure 2 The speech recognition test method shown.

[0146] Optionally, the computer device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a Bluetooth interface, a WI-FI interface), etc.

[0147] Those skilled in the art will appreciate that the computer device structure provided in this embodiment does not limit the computer device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0148] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware platforms, or by hardware to control the player to play the test voice of the test text, so that the voice recognition device recognizes the test voice and displays the recognition text of the test voice on the UI interface; the recognition text of the voice recognition device for the test voice is screened from the operation information fed back by the voice recognition device in the connected state, wherein the operation information includes UI interface elements and / or log files; the test text and the recognition text are compared to determine the test result of the voice recognition device. In the embodiment of the present application, the test voice is played in advance according to the set test text, and the system obtains the recognition text of the voice recognition device for the test voice by reading the operation information of the voice recognition device. The ASR test result of the voice recognition device is obtained by comparing the difference between the test text and the recognition text. In this way, based on the automated voice playback and recognition text extraction, the recognition result of the voice recognition device can be obtained without manual intervention, thereby realizing automated ASR testing. It not only reduces the workload of voice technicians to verify the accuracy of voice recognition, but also ensures the accuracy of the evaluation of the recognition result of the voice recognition device.

[0149] Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present application. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple submodules.

[0150] The above serial numbers of this application are only for description and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of this application, but this application is not limited to them, and any changes that can be thought of by technicians in this field should fall within the scope of protection of this application.

Claims

1. A speech recognition test method, characterized in that: The method comprises: Control the player to play the test voice of the test text, so that the voice recognition device recognizes the test voice and displays the recognized text of the test voice on the UI interface; Filtering the recognition text of the test voice by the voice recognition device from the operation information fed back by the voice recognition device in the connected state, wherein the operation information includes UI interface elements and / or log files; The test text and the recognition text are compared to determine the test result of the speech recognition device.

2. The speech recognition test method according to claim 1, characterized in that: The step of filtering the recognition text of the test voice by the voice recognition device from the operation information fed back by the voice recognition device in the connected state comprises: Filtering the recognition text of the test speech by the speech recognition device from the UI interface elements according to the text attributes; and / or, The log file is matched according to a preset format, and the recognized text in the log file matching the preset format is extracted.

3. The speech recognition test method according to claim 1, characterized in that: The method further comprises: If the recognition text corresponding to the test text is not obtained within the preset time interval, control the player to repeatedly play the test voice until the recognition text is obtained or the number of times the test voice is played is greater than or equal to the preset number of times corresponding to the test text; The number of the test texts is multiple, and the controlling player to play the test voice of the test texts includes: If the recognition text corresponding to the current test text is obtained or the number of times the current test voice is played is greater than or equal to the preset number of times corresponding to the current test text, the player is controlled to play the test voice of the next test text.

4. The speech recognition test method according to claim 1, characterized in that: The method further comprises: Periodically detecting status information of the speech recognition device; If the voice recognition device is in a screen-off state, sending a wake-up command to the voice recognition device; If the speech recognition device is in a non-debugging mode, output a first prompt message; If the voice recognition device is in a disconnected state, stop controlling the player to play the test voice of the test text and output a second prompt message.

5. The speech recognition test method according to any one of claims 1 to 4, characterized in that: The comparing the test text and the recognition text to determine the test result of the speech recognition device includes: Comparing the test text with the recognized text, determining the number of correct text characters, the number of deleted erroneous characters, the number of inserted erroneous characters, and the number of replaced erroneous characters between the test text and the recognized text; Determining a text recognition rate of the test text according to the number of correct text characters, the number of deleted erroneous characters, the number of inserted erroneous characters and / or the number of replaced erroneous characters; Performing weighted processing on the text recognition rate of at least one of the test texts according to the weight value of the test text, and calculating the recognition score of the speech recognition device; The test result is determined based on a comparison result between the recognition score and a preset scoring standard.

6. The speech recognition test method according to claim 5, characterized in that: The method further comprises: Determining the recognition difficulty level of the test text according to the number of correct text characters and the similarity of speech features between adjacent characters in the test text; The weight value is matched according to the recognition difficulty level.

7. The speech recognition test method according to any one of claims 1 to 4, characterized in that: The method further comprises: In response to a test instruction for a speech recognition device, loading a configuration file carried by the test instruction, wherein the configuration file includes a test text; The test speech of the test text is generated by speech synthesis technology.

8. A speech recognition test device, characterized in that: The device comprises: A control module, used to control the player to play the test voice of the test text, so that the voice recognition device recognizes the test voice and displays the recognized text of the test voice on the UI interface; an acquisition module, used to acquire UI interface elements of the voice recognition device in a connected state; and Filtering the recognition text of the test speech by the speech recognition device from the UI interface elements according to the text attributes; The test module is used to compare the test text with the recognition text to determine the test result of the speech recognition device.

9. A computer device comprising at least one processor, the processor being coupled to a memory, the memory storing a computer program running on the processor, characterized in that: When the processor executes the program, the speech recognition test method according to any one of claims 1 to 7 is implemented.

10. A speech recognition test system, characterized in that: include: A player, wherein the player is arranged at a preset distance from the speech recognition device; A noise generator, the noise generator is arranged around the speech recognition device, and the noise generator is used to generate background noise; A computer device, wherein the computer device is respectively connected to the player and the voice recognition device for communication, and the computer device performs the following steps when executing a program: controlling the player to play a test voice of a test text so that the voice recognition device recognizes the test voice and displays the recognized text of the test voice on a UI interface; obtaining a UI interface element of the voice recognition device in a connected state; The recognition text of the test voice by the voice recognition device is filtered from the UI interface elements according to the text attributes; and the test text and the recognition text are compared to determine the test result of the voice recognition device.

Images