Speech recognition rate automatic test method and system

Abstract

The invention discloses a voice recognition rate automatic test method and system. The method comprises the following steps: S1, corpus voice entry content confirmation is carried out; s2, if the to-be-tested corpus in the corpus is incomplete, corpus collection and normalization processing need to be carried out until the to-be-tested corpus is complete; s3, after the corpus to be tested completely exists in the voice library, configuring a test process, and adding all items needing to be tested into the test process in advance; s4, configuring an identification configuration file according to the vocabulary entry to be tested; s5, opening PC end test software, selecting a correct baud rate and a correct serial port number to connect the equipment and the PC end test software, and clicking to start the test; s6, the PC end test software reads information in the test process file line by line, and a test scene is built; and S7, if the Hall sensor judges that the equipment is in place, the manual mouth begins to play vocabulary entries in the voice library, and the playing rule is that line-by-line searching is carried out according to the sequence in the identification configuration file and a certain standard audio in a certain corpus is randomly played.

Description

Technical Field

[0001] The present invention relates to the technical field of automated testing of functions such as speech recognition rate of intelligent products, and in particular to a speech recognition rate automated testing method and system. Background Art

[0002] Nowadays, there are more and more smart products, and products with voice functions are becoming more popular. However, in the testing phase of voice smart products, it is necessary to consider how to produce test results under a constant and standard condition. In the past, voice tests were performed manually by inputting voice commands and observing the device status to statistically test the test results. This is a large number of repetitive tests, and because it is performed manually, it is easy to cause a series of subjective differences due to manual operation, affecting the test results. Summary of the invention

[0003] The purpose of the present invention is to provide a method and system for automatically testing speech recognition rate, in order to solve the technical problems existing in the background technology.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] A method for automatically testing speech recognition rate, comprising the following steps:

[0006] S1: Confirm the content of the corpus speech entries to check whether the corpus to be tested exists completely in the corpus;

[0007] S2: If the corpus to be tested is incomplete, it is necessary to collect and normalize the corpus until the corpus to be tested is complete;

[0008] S3: After the corpus to be tested is completely present in the speech database, configure the test process and add all the items to be tested to the test process in advance;

[0009] S4: Configure the recognition configuration file according to the test entry;

[0010] S5: Open the PC test software, select the correct baud rate and serial port number to connect the device and the PC test software, and click to start the test;

[0011] S6: The PC test software reads the information in the test process file line by line and builds the test scenario;

[0012] S7: If the Hall sensor determines that the device is in place, the artificial mouth starts to play the words in the speech library. The playing rule is to search line by line according to the order in the recognition configuration file and randomly play a standard audio in a certain corpus;

[0013] S8: Every time the artificial mouth plays a corpus entry, the PC-side test software will compare the key log information through the serial port log of the device to determine whether the device successfully recognizes the content of the relevant entry. If the log comparison is successful, the correct count will increase by 1, otherwise the failure count will increase by 1, until the current test item content is fully played and the test result is recorded.

[0014] In some embodiments, the S4: configures a recognition configuration file according to the test entry, including: if an offline recognition test is performed, the format of the recognition configuration file is A$B, wherein: A is the file name of the corpus in the voice library that the artificial mouth needs to play, and B is the log keyword of successful recognition in the serial port printing on the device side; if an online recognition test is performed, the format of the recognition configuration file is A$B$C, wherein: A is the file name of the corpus in the voice library that the artificial mouth needs to play, B is the log keyword of successful ASR recognition in the serial port printing on the device side, and C is the log keyword of successful NLP semantic recognition in the serial port printing on the device side.

[0015] In some embodiments, the S6: PC-side test software reads the information in the test process file line by line and builds a test scene, including: 5) deploying the device in a speech laboratory, the PC-side test software reads the configuration content of the Excel file line by line, builds a test scene for each test item, and sends corresponding information to the WiFi module; the WiFi module sends information such as the test distance to the motor module, the motor module controls the track to move the artificial mouth to a preset position, and the Hall sensor determines whether the device is in place.

[0016] In some embodiments, the corpus collection and normalization processing includes:

[0017] A sort out the entry list that needs to be collected;

[0018] B. Prepare the necessary recording environment;

[0019] C Once the environment is ready, you can start collecting audio;

[0020] D. After collecting the audio, perform editing and segmentation processing;

[0021] E Rename the edited audio;

[0022] F adds pink noise to each folder with one click through the script file;

[0023] G The renamed audio is normalized using normalization software;

[0024] H. Second manual confirmation to see if the audio meets the usage standards;

[0025] IAdd compliant audio to the voice library and classify it for management.

[0026] The present application also discloses a speech recognition rate automatic testing system, which is used to implement any of the methods described above, including:

[0027] A test parameter configuration module, which is used to include a test process configuration file and an identification configuration file;

[0028] The speech library resource module is used to place all corpora in different folders according to their actual contents;

[0029] Track operation module, used to interact with the PC test software and move the artificial mouth to the specified point through the track;

[0030] The voice broadcast module reads the information in the recognition configuration file through the PC test software and selects the corpus in the voice library resource module for broadcasting;

[0031] Data comparison and analysis module: the device and PC test software are connected through the serial port to obtain the device log, and the key log information of the device is captured for data comparison and analysis, and the data results are updated;

[0032] The data display module automatically generates and saves the data results of each item after all test items are tested to display and analyze the results.

[0033] The present invention provides a method and system for automatically testing speech recognition rate, which has beneficial effects including but not limited to:

[0034] The present invention aims to invent a method and system for automatic testing of speech recognition rate. Through standardized collection of corpus, and after normalization processing and screening according to standards, it is uniformly stored in a voice library file, which is convenient for subsequent multiple cycles. The automatic testing method of speech recognition rate is to first configure the relevant process and recognition file, and then let the artificial mouth run to the specified position, play the corpus at a preset volume, establish communication between the PC-side test software and the WiFi module, and judge whether the recognition of the corpus is correct by grabbing the log information, and finally count the results and record the logs and abnormal information. This method and system improves labor efficiency, reduces manpower and time costs, and ensures that the machine is used for testing under a standard environment, so that the results are more reliable and avoids subjective differences caused by manual testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Corpus collection flow chart.

[0036] Figure 2 Speech automation testing flow chart.

[0037] Figure 3 Functional module diagram of automated testing system.

[0038] Figure 4Schematic diagram of competitive product comparison test. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described in more detail below in conjunction with the drawings in the preferred embodiments of the present application. In the drawings, the same or similar reference numerals throughout represent the same or similar parts or parts with the same or similar functions. The described embodiments are part of the embodiments of the present application, not all of the embodiments. The embodiments described below with reference to the drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limitations on the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0040] The following will be combined Figure 1-4 It is worth noting that the following embodiments are only used to explain the present application and do not constitute a limitation on the present application.

[0041] Example 1

[0042] The present invention provides a method and system for standard corpus collection and processing and automatic testing of speech recognition rate function, which collects corpus and processes it uniformly in the early stage of automatic testing of speech function, thereby producing standard corpus content, and performs a series of system tests such as speech recognition function in a standard speech laboratory. In the same environment, a standard corpus is used to test speech intelligent products, and the results are more real, standard and reliable. The present invention is fast and convenient, and greatly saves human resources and time costs, and has a good application scenario.

[0043] like Figure 1 The figure shows the process of corpus collection, starting from the sorting of the corpus entry table, through a series of steps, finally adding qualified corpus to the existing corpus to achieve repeated use. The relevant process description is as follows:

[0044] A sorts out the word list that needs to be collected; the corpus collector enters the speech laboratory to record and collect the corpus; the speech laboratory sets up a recording environment, which needs to meet the requirements of a quiet environment (the indoor precision sound pressure meter collects the environmental noise of 25-30db), and the main hardware facilities include sound cards, microphone amplifiers, headphone amplifiers, sound cards and other equipment;

[0045] B. Prepare the necessary recording environment (the volume of the voice laboratory environment should be measured by a precise sound pressure meter and maintained at 25-30db);

[0046] C Once the environment is ready, you can start collecting audio;

[0047] D. After collecting the audio, perform editing and segmentation processing;

[0048] E Rename the edited audio;

[0049] F adds pink noise to each folder with one click through the script file;

[0050] G The renamed audio is normalized using normalization software;

[0051] H. Second manual confirmation to see if the audio meets the usage standards;

[0052] IAdd compliant audio to the voice library and classify it for management.

[0053] like Figure 2 The voice automation test flow chart includes:

[0054] S1: Confirm the content of the corpus speech entries to check whether the corpus to be tested exists completely in the corpus;

[0055] S2: If the corpus to be tested is incomplete, a series of processes such as corpus collection and normalization are required until the corpus to be tested is complete;

[0056] S3: After the corpus to be tested is completely present in the speech database, configure the test process and add all the items to be tested to the test process in advance;

[0057] S4: According to the test entry, configure the recognition configuration file. If an offline recognition test is performed, the format of the recognition configuration file is A$B (where A is the file name of the corpus in the voice library that the artificial mouth needs to play, and B is the log keyword of successful recognition in the serial port printing on the device side); if an online recognition test is performed, the format of the recognition configuration file is A$B$C (where A is the file name of the corpus in the voice library that the artificial mouth needs to play, B is the log keyword of successful ASR recognition in the serial port printing on the device side, and C is the log keyword of successful NLP semantic recognition in the serial port printing on the device side);

[0058] S5: Open the PC test software, select the correct baud rate and serial port number to connect the device and the PC test software, and click to start the test;

[0059] S6: The PC test software reads the information in the Excel file (test process file) line by line, automatically adjusts the volume of the artificial mouth to the preset value, and sends the artificial mouth position information to the motor module. The motor module controls the track to move the artificial mouth to the preset position, and the Hall sensor determines whether the device is in place;

[0060] S7: If the Hall sensor determines that the device is in place, the artificial mouth starts to play the entries in the speech library (the speech library has multiple folders classified by corpus content). The playing rule is to search line by line according to the order in the recognition configuration file and randomly play a standard audio in a certain corpus;

[0061] S8: Every time the artificial mouth plays a corpus entry, the PC-side test software will compare the key log information through the serial port log of the device to determine whether the device successfully recognizes the content of the relevant entry. If the log comparison is successful, the correct count will increase by 1, otherwise the failure count will increase by 1 until the current test item content is fully played, and the test results will be recorded in the test record. All device-side logs will be recorded in the log file;

[0062] S9: After all test processes are executed, all test results are automatically generated in the test record file, and failed data will be recorded in the unidentified file to facilitate subsequent analysis of problems and results.

[0063] like Figure 3 The functional module diagram of the automated test system includes:

[0064] 1: Test parameter configuration module, which mainly includes test process configuration files and identification configuration files, and is the test parameter information required before testing;

[0065] 2: Voice library resource module, which contains all test corpus information. All corpus are placed in different folders according to the actual content of the corpus. That is, each folder contains multiple audio corpora with different voices but the same content. During the test phase, they can be randomly selected and played in the corresponding folder;

[0066] 3: Track operation module, which mainly interacts with the PC test software to move the artificial mouth to the specified point through the track;

[0067] 4: Voice broadcast module, read the information in the recognition configuration file through the PC test software, and select the corpus in the voice library resource module for broadcast;

[0068] 5: Data comparison and analysis module: the device and PC test software are connected through the serial port to obtain the device log, and the key log information of the device is captured for data comparison and analysis, and the data results are updated;

[0069] 6: Data display module: after all test items are tested, the data results of each item are automatically generated and saved to display and analyze the results.

[0070] like Figure 4As shown in the principle diagram of competitive product comparison test, our current complete set of methods and systems can not only support voice testing of our own products, but also benchmark against competitors and conduct comparative tests on the voice functions of competing products. Under the same environment and using the same voice library samples, the output conclusions are reliable.

[0071] 1. Manual screening is required to compare the common function terms of the product and the competitor's product, and select the function terms common to the two products;

[0072] 2. Perform different configurations on the test process files of the PC test software (for your own products: you need to indicate the recognition rate test in the test process file; for competing products: you need to configure the competing product recognition test in the test process file);

[0073] 3. Our own products can complete the test fully automatically, obtain the voice test results by printing log information, and automatically complete the data statistics; the competitive product test adopts semi-automatic testing, automatically plays the audio, and manually records the data results according to the actual response of the device;

[0074] 4. Compare the automatically generated results of your own products with the manually recorded test results of competing products to draw corresponding conclusions and make the data more reliable.

[0075] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for automated testing of speech recognition rate, characterized in that: The following steps are involved: S1: Confirm the content of the corpus speech entries to check whether the corpus to be tested exists completely in the corpus; S2: If the corpus to be tested is incomplete, it is necessary to collect and normalize the corpus until the corpus to be tested is complete; S3: After the corpus to be tested is completely present in the speech database, configure the test process and add all the items to be tested to the test process in advance; S4: Configure the recognition configuration file according to the test entry; S5: Open the PC test software, select the correct baud rate and serial port number to connect the device and the PC test software, and click to start the test; S6: The PC test software reads the information in the test process file line by line and builds the test scenario; S7: If the Hall sensor determines that the device is in place, the artificial mouth starts to play the words in the speech library. The playing rule is to search line by line according to the order in the recognition configuration file and randomly play a standard audio in a certain corpus; S8: Every time the artificial mouth plays a corpus entry, the PC-side test software will compare the key log information through the serial port log of the device to determine whether the device successfully recognizes the content of the relevant entry. If the log comparison is successful, the correct count will increase by 1, otherwise the failure count will increase by 1, until the current test item content is fully played and the test result is recorded.

2. The method for automatic testing of speech recognition rate according to claim 1, characterized in that: S4: configuring a recognition configuration file according to the test entry, including: if an offline recognition test is performed, the format of the recognition configuration file is A$B, wherein: A is the file name of the corpus in the voice library that the artificial mouth needs to play, and B is the log keyword of successful recognition in the serial port printing on the device side; if an online recognition test is performed, the format of the recognition configuration file is A$B$C, wherein: A is the file name of the corpus in the voice library that the artificial mouth needs to play, B is the log keyword of successful ASR recognition in the serial port printing on the device side, and C is the log keyword of successful NLP semantic recognition in the serial port printing on the device side.

3. The method for automatic testing of speech recognition rate according to claim 1, characterized in that: The S6: PC-side test software reads the information in the test process file line by line and builds a test scenario, including: 5) deploying the device in the voice laboratory, the PC-side test software reads the configuration content of the Excel file line by line, builds a test scenario for each test item, and sends corresponding information to the WiFi module; the WiFi module sends information such as the test distance to the motor module, the motor module controls the track to move the artificial mouth to a preset position, and the Hall sensor determines whether the device is in place.

4. The method for automatic testing of speech recognition rate according to claim 1, characterized in that: The corpus collection and normalization processing includes: A sort out the entry list that needs to be collected; B. Prepare the necessary recording environment; C Once the environment is ready, you can start collecting audio; D. After collecting the audio, perform editing and segmentation processing; E Rename the edited audio; F adds pink noise to each folder with one click through the script file; G The renamed audio is normalized using normalization software; H. Second manual confirmation to see if the audio meets the usage standards; IAdd compliant audio to the voice library and classify it for management.

5. A speech recognition rate automated testing system, characterized in that: The method for implementing any one of claims 1 to 4 comprises: A test parameter configuration module, which is used to include a test process configuration file and an identification configuration file; The speech library resource module is used to place all corpora in different folders according to their actual contents; Track operation module, used to interact with the PC test software and move the artificial mouth to the specified point through the track; The voice broadcast module reads the information in the recognition configuration file through the PC test software and selects the corpus in the voice library resource module for broadcasting; Data comparison and analysis module: the device and PC test software are connected through the serial port to obtain the device log, and the key log information of the device is captured for data comparison and analysis, and the data results are updated; The data display module automatically generates and saves the data results of each item after all test items are tested to display and analyze the results.

Images