Optimization method of speech recognition system, electronic device and storage medium

Abstract

The present invention discloses an optimization method, electronic device, and storage medium for a speech recognition system. In this method, an initial data set is obtained, the initial data set comprising multiple audio data and corresponding matching recognition texts; a first data subset and a second data subset are determined based on the initial data set; wherein the recognition confidence level of the first data subset is greater than the recognition confidence level of the second data subset; based on the first data subset and the second data subset, a speech recognition system is subjected to mixed supervised training, wherein the first data subset is used for supervised training of the speech recognition system, and the second data subset is used for unsupervised training of the speech recognition system. Thus, a data subset with a high recognition confidence level is used for supervised training, and a data subset with a low recognition confidence level is used for unsupervised training, thereby fully tapping the learning value of the data set and effectively improving the optimization effect of the speech recognition system.

Description

Technical Field

[0001] The present invention belongs to the technical field of speech processing, and in particular relates to an optimization method, electronic equipment and storage medium for a speech recognition system. Background Art

[0002] Speech recognition technology has been integrated into various aspects of our lives and work, such as smart homes and voice navigation. With the continuous advancement of speech technology and device technology, people have placed higher demands on the recognition performance of speech recognition systems, making efficient system optimization solutions a key research direction for major speech recognition service providers.

[0003] Currently, when optimizing a system, most companies or individuals generally manually and randomly select some data from online data, then manually send it for labeling to obtain labeled supervised audio data and labeled text, and then conduct supervised speech recognition acoustic model training and language model training before optimizing, testing, and launching the system.

[0004] However, manual training is time-consuming and inefficient, preventing rapid iterations, resulting in poor recognition of recent trending content. Furthermore, labeling randomly selected online data is relatively expensive, while a significant portion of the data is of low value. Furthermore, the value of data is often strongly correlated with the model currently in use. Furthermore, the use of only supervised data for model training results in relatively limited optimization of the voice system.

[0005] In response to the above problems, the industry has not yet provided a better solution. Summary of the Invention

[0006] The embodiments of the present invention provide a method for optimizing a speech recognition system, an electronic device, and a storage medium, which are used to solve at least one of the above-mentioned technical problems.

[0007] In a first aspect, an embodiment of the present invention provides an optimization method for a speech recognition system, comprising: obtaining an initial data set, wherein the initial data set includes multiple audio data and corresponding matching recognition texts; determining a first data subset and a second data subset based on the initial data set; wherein the recognition confidence level of the first data subset is greater than the recognition confidence level of the second data subset; and performing mixed supervised training on the speech recognition system based on the first data subset and the second data subset, wherein the first data subset is used for supervised training of the speech recognition system, and the second data subset is used for unsupervised training of the speech recognition system.

[0008] In a second aspect, an embodiment of the present invention provides an electronic device comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the above method.

[0009] In a third aspect, an embodiment of the present invention provides a storage medium, in which one or more programs including execution instructions are stored. The execution instructions can be read and executed by an electronic device (including but not limited to a computer, a server, or a network device, etc.) to execute the steps of the above-mentioned method of the present invention.

[0010] In a fourth aspect, an embodiment of the present invention further provides a computer program product, which includes a computer program stored on a storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the steps of the above method.

[0011] The beneficial effects of the embodiments of the present invention are at least:

[0012] Data classification is performed based on the credibility of the data set. A data subset with a high recognition credibility level is used for supervised training, and a data subset with a low recognition credibility level is used for unsupervised training. This makes full use of the data in the data set and fully explores the training value of the data set, which can effectively improve the optimization effect of the speech recognition system. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0014] Figure 1 A flowchart showing an example of a method for optimizing a speech recognition system according to an embodiment of the present invention is shown;

[0015] Figure 2 Shown Figure 1 A flowchart of an example of a specific operation process of step 120;

[0016] Figure 3 Shown Figure 2 A flowchart of an example of a specific operation process of step 220;

[0017] Figure 4 Shown Figure 1A flowchart of an example of a specific operation process of step 130;

[0018] Figure 5 A schematic diagram of an operation flow for optimizing a speech recognition system based on active learning according to an embodiment of the present invention is shown;

[0019] Figure 6 A schematic diagram of a process for generating cluster labels for unsupervised learning data according to an embodiment of the present invention is shown;

[0020] Figure 7 An operational flow chart illustrating an example of a self-training module for a language model;

[0021] Figure 8 A flowchart of an example of a speech recognition method according to an embodiment of the present invention is shown;

[0022] Figure 9 FIG. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0024] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other.

[0025] The present invention may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0026] In the present invention, "module", "system" and the like refer to related entities applied to computers, such as hardware, a combination of hardware and software, software or software in execution, etc. Specifically, for example, an element can be, but is not limited to, a process, a processor, an object, an executable element, an execution thread, a program and / or a computer running on a processor. In addition, an application or script program running on a server, or a server can all be an element. One or more elements can be in an execution process and / or thread, and an element can be localized on a computer and / or distributed between two or more computers, and can be operated by various computer-readable media. An element can also communicate by local and / or remote processes based on a signal with one or more data packets, for example, a signal from a network that interacts with a local system, another element in a distributed system, and / or data that interacts with other systems via a signal on the Internet.

[0027] Finally, it should be noted that, in this document, the terms "include" and "comprising" include not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, the elements defined by the phrase "include..." do not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the elements.

[0028] Currently, the need to manually complete the optimization of the entire speech recognition system and the consistency of the online process is a pain point in R&D in the industry. Due to the limitations of labeling costs and labeling capacity, large-scale supervised labeling for model optimization training is often not feasible. In this case, the optimization of training based on the current mainstream supervised model often has limited effects, and a large amount of unsupervised data and semi-supervised data with existing recognition results cannot be used.

[0029] In view of this, Figure 1 A flowchart of an example of a method for optimizing a speech recognition system according to an embodiment of the present invention is shown.

[0030] like Figure 1 As shown, in step 110, an initial data set is obtained. The initial data set includes multiple audio data and corresponding matching recognition texts. Here, the initial data set can be set or changed. For example, the initial data set can be continuously updated to achieve iterative optimization of the system.

[0031] In some implementations, the initial dataset can be constructed based on online operational data from the speech recognition system. Furthermore, online audio data and corresponding recognized text can be collected at a predetermined collection period (e.g., T+1), and the initial dataset can be supplemented and improved based on the new online audio data and corresponding recognized text, thereby achieving continuous iterative optimization of the speech recognition system.

[0032] In step 120, a first data subset and a second data subset are determined based on the initial data set. Here, the recognition confidence level of the first data subset is greater than the recognition confidence level of the second data subset. For example, each piece of data in the initial data set records the recognition confidence level of the audio data relative to the matching recognition text, and recognition confidence rating can be performed based on the recognition confidence level. Furthermore, if the initial data set contains audio data with expert-annotated text, such data should have the highest recognition confidence level.

[0033] In step 130, a speech recognition system is subjected to mixed supervised training based on the first data subset and the second data subset. Here, the first data subset is used for supervised training of the speech recognition system, and the second data subset is used for unsupervised training of the speech recognition system.

[0034] Through the embodiments of the present invention, the data in the data set is fully utilized, a data subset with a high recognition confidence level is used for supervised training, and a data subset with a low recognition confidence level is used for unsupervised training. Mixed supervised training is used to fully explore the learning value of the data set, which can further improve the optimization effect of the speech recognition system.

[0035] Figure 2 Shown Figure 1 FIG. 1 is a flowchart of an example of a specific operation process of step 120 in FIG.

[0036] In step 210, invalid data in the initial data set is identified and filtered out. Here, invalid data mainly refers to data that does not have a positive benefit on the performance improvement of the speech recognition system. These data mainly include audio data without voice or very little voice information, such as pure silence data and noise data, thereby distinguishing "invalid audio data" from "audio data with learning value" in the initial data set. Exemplarily, a neural network model is used, combined with information such as the acoustic model, language model and intermediate results of the decoding process of the recognition system, to perform a binary classification on each audio, dividing it into "invalid audio data" or "audio data with learning value". The classification criteria are more inclined to distinguish whether the audio itself is pure noise, pure silence or human voice that cannot be labeled.

[0037] In step 220 , the recognition confidence level corresponding to each data in the filtered initial data set is obtained, and a first data subset and a second data subset are determined based on the recognition confidence level.

[0038] By identifying and filtering invalid data, data valuable to the speech recognition system is output, thereby determining the training data for the speech recognition system. Therefore, by filtering out invalid data, the high value and efficiency of the data used to train the model can be guaranteed, resulting in faster model optimization speed and shorter optimization cycles.

[0039] Figure 3 Shown Figure 2 FIG. 1 is a flowchart of an example of a specific operation process of step 220 in FIG.

[0040] In step 310, for each piece of audio data in the filtered initial dataset, the recognition confidence of the audio data relative to the matching recognition text is obtained, and the recognition confidence is compared with the recognition confidence intervals [a1], [a2], and [a3] corresponding to different recognition confidence levels, where [a1] < [a2] < [a3]. Thus, the learning value of the audio data is quantified based on the recognition confidence.

[0041] Specifically, a neural network model can be used, combined with the recognition system's acoustic model, language model, and intermediate results of the decoding process, to predict the confidence level of each audio transcription result. Here, confidence is a real number less than 1. The higher the confidence level, the higher the probability that the current transcription result of the recognition system is correct, and the lower the learning value of the audio for the recognition system in subsequent hybrid supervised training. Furthermore, all "learnable audio data" can be quantified as a real number less than 1 and sorted by confidence level.

[0042] In step 320 , a second data subset is determined based on the audio data corresponding to the recognition confidence interval [a2].

[0043] In step 330 , corresponding annotation text is configured for the audio data corresponding to the recognition confidence interval [a1] to obtain corresponding annotated audio data.

[0044] In step 340, a first data subset is determined based on the labeled audio data and the audio data corresponding to the recognition confidence interval [a3]. Here, the data within the recognition confidence interval [a3] is treated as semi-supervised data, the labeled audio data is treated as supervised data, and the second data subset is treated as unsupervised data. Data valuable to the speech recognition system for learning is divided into three categories based on confidence, and these data are output together to form "audio data valuable for learning" for use in training and optimizing the model.

[0045] Exemplarily, two percentages p and q can be given, and the generated ranking index for confidence can be divided into three subsets: the first p% data (0%-p%) with the highest confidence is considered to have lower learning value and more accurate transcription results. It does not need to be labeled by experts to generate positive benefits for the recognition system in subsequent mixed supervision training. This part of the data is used as the screened semi-supervised data; the last q% data (q%-100%) with the lowest confidence is considered to have higher learning value and a higher error rate in transcription results. It needs to be labeled by experts before it can generate a larger positive benefit for the recognition system in subsequent mixed supervision training. This part of the data is used as the screened supervised data; limited by the labeling budget under objective conditions, the data with a confidence ranking of p%-q% also has certain learning value. Although the corresponding labeled text cannot be obtained, the audio information of this part of the data itself can still generate certain positive benefits for the recognition system in mixed supervision training. This part of the data is used as the screened unsupervised data to expand the second data subset. Based on the above operation examples, active learning selects three types of audio data with different learning values, so that the corresponding data can be used for model optimization operations during mixed supervised training.

[0046] Figure 4 Shown Figure 1 FIG. 1 is a flowchart of an example of a specific operation process of step 130 in FIG.

[0047] like Figure 4 As shown, in step 410, the acoustic model is subjected to hybrid supervised training based on the first data subset and the second data subset. Specifically, the acoustic model is trained in a supervised manner based on the first data subset, and unsupervised training is performed on the acoustic model based on the second data subset. Through hybrid supervised training, data of different learning values ​​are comprehensively considered to maximize the optimization effect of the acoustic model.

[0048] In step 420 , a language model is trained based on the annotated text corresponding to the annotated audio data.

[0049] In an embodiment of the present invention, the speech recognition system includes an acoustic model and a language model to recognize sound signals (acoustic features) into sentences (word strings), wherein the sound signals are recognized into acoustic units such as phonemes based on the acoustic model (and the probability is given), and the recognition results of acoustic units such as phonemes are further screened based on the language model according to grammar and lexical knowledge (the part that conforms to grammatical rules is selected).

[0050] Figure 5 A schematic diagram of an operation flow for optimizing a speech recognition system based on active learning according to an embodiment of the present invention is shown.

[0051] The following describes the process optimization solution in conjunction with the process node identifiers in the figure, assuming that "online audio data to be screened" for a certain period of time (such as one day) has been stored in the database.

[0052] In process node 1, the active learning data screening module is regularly activated to screen all "online audio data to be screened." The active learning data screening module regularly screens all "online audio data to be screened," identifies invalid data, and filters it out, thereby obtaining data with learning value.

[0053] At process node 2.1, the data with learning value is divided into three groups. The specific classification operations can be partially referred to in the description of other embodiments above and will not be repeated here. Among them, the "audio data to be annotated by experts" is selected audio data that is considered to be worthy of manual fine-grained annotation to obtain accurate text content annotation. Due to the high cost of manual fine-grained annotation, the data volume of this part is relatively small.

[0054] In process node 2.2, "Semi-supervised audio data and recognized text" are selected audio data and its corresponding recognized text, which are considered to have relatively reliable recognition results and can help improve the subsequent training of the speech recognition acoustic model. This data does not undergo detailed manual annotation, so there is no manual annotation cost. It is called semi-supervised data. The data volume of this data is generally larger than that of "Audio data to be annotated by experts" but smaller than that of "Unsupervised audio data."

[0055] In process node 2.3, "unsupervised audio data" is selected as audio data that is considered to be able to improve the subsequent training of the speech recognition acoustic model through unsupervised training principles. Because the recognition results of this data may be relatively inaccurate, its recognized text will not be used to assist in training. Furthermore, because this data is not manually annotated, there is no manual annotation cost, and it is therefore called unsupervised audio data.

[0056] In process node 3, the "audio data to be annotated by experts" is pushed to the data annotation platform system, generating a manual annotation task for speech recognition data.

[0057] In process node 4, annotation experts with data annotation experience perform data annotation and quality inspection on the "audio data to be annotated by experts" through the annotation platform system.

[0058] Through active learning data screening, data with high manual annotation value (as well as semi-supervised and unsupervised data with learning value for mixed supervised training) can be effectively screened out. Compared with traditional random screening schemes, less data is required to achieve the same recognition effect optimization, which can effectively reduce the data annotation costs caused by optimization needs. Because the amount of data used for model optimization training is smaller, the model optimization speed is faster and the cycle is shorter.

[0059] In process node 5, the data annotated in the previous step will generate manually annotated text. At this time, we call this part of the data supervised audio data and supervised annotated text.

[0060] At process node 6, the supervised audio data and annotated text generated in node 5, along with the semi-supervised audio data, recognized text, and unsupervised audio data, are used for hybrid supervised acoustic model optimization training to generate an optimized acoustic model. Generally speaking, acoustic model training can be subdivided into supervised training, unsupervised training, and semi-supervised training based on the data labels. In this embodiment of the present invention, it is proposed that these three training methods, supervised training, unsupervised training, and semi-supervised training, can be integrated to perform hybrid supervised training on the acoustic model.

[0061] When performing hybrid supervised acoustic model optimization training operations, the supervised training data will be sent for labeling through the automatic optimization system, and after the data group completes the labeling, it will be directly returned to the data set platform for subsequent model training. For semi-supervised audio data, the acoustic model can also be trained based on the labeled audio data (supervised audio data) to obtain the teacher acoustic model, and then the matching recognition text corresponding to the audio data (i.e., semi-supervised audio data) corresponding to the recognition confidence interval [a1] is calibrated based on the teacher acoustic model to update the first data subset. Based on the updated first data subset, the acoustic model is trained, thereby integrating the training processes of supervised audio and semi-supervised audio data to achieve better model optimization effects.

[0062] In some examples of the embodiments of the present invention, supervised data (i.e., manually accurately labeled audio and text data, denoted as D super ) and semi-supervised data (denoted as D semi ) are all regarded as labeled data (denoted as D two ) is used to perform preliminary training optimization on the acoustic model, and the unsupervised data (denoted as D self ) is used to perform advanced training optimization on the acoustic model. Specifically, multiple data training units are determined based on the first data subset and the second data subset, for example, D two With D selfThe training data with a data volume ratio of 1:N (N>1) is used as a data base unit (denoted as batch) for model training. Then, based on each data training unit, the acoustic model is subjected to mixed supervised training in sequence. Specifically, for each data training unit, the acoustic model is subjected to unsupervised training corresponding to the first stage based on the first optimization parameter configuration and the second data in the data training unit, and then the acoustic model is subjected to supervised training corresponding to the second stage based on the second optimization parameter configuration and the second data in the data training unit. For example, the unsupervised data D self The first parameter update is generated by the Loss feedback model generated by the CE (Cross-Entropy) criterion and cluster label calculation, and then D two The second model parameter update is also triggered based on the CE criteria and the respective labels. The two model updates use separate optimizers and optimization parameter configurations.

[0063] Specifically, based on the training process of the unsupervised audio data (or the second data subset), a clustering method is used to obtain corresponding cluster labels. Figure 6 , extract the MFCC features of the audio data in the unsupervised audio data, and determine the corresponding first cluster label. For example, use the K-mean clustering algorithm to cluster the extracted MFCC features, generate a corresponding codebook, and map all the data in the data set to the corresponding class number (recorded as cluster label 1) according to the codebook. Based on the audio data and the first cluster label in the unsupervised audio data, perform the first stage model training operation on the acoustic model. Obtain the parameters of each model hidden layer of the acoustic model corresponding to the first stage of training convergence, and determine the corresponding second cluster label. For example, the model can be trained with cluster label 1, and after the model converges, the MFCC features in stage one are replaced with the model hidden layer, and the clustering operation is performed again to obtain cluster label 2. Based on the audio data and the second cluster label in the unsupervised audio data, perform the second stage model training operation on the acoustic model. For example, for cluster label 2, a subword segmentation algorithm (for example, sentencepiece) can be used to obtain the corresponding subword label for unsupervised training of the acoustic model.

[0064] Through the hybrid supervised acoustic model training module, supervised data, semi-supervised data and unsupervised data are used simultaneously in model training. Compared with the traditional training method that only uses supervised data, the optimization effect of the acoustic model can be effectively improved. Since semi-supervised data and unsupervised data do not require manual labeling and no cost is incurred, better model optimization effects can be achieved at the same cost.

[0065] In process node 7, the language model is trained and optimized using the supervised annotated text generated in step 5 and the language model iteration module to generate an optimized language model.

[0066] Based on the language model, the distribution of word sequences in the natural language can be constructed to assess the probability of a particular word sequence. In the embodiment of the present invention, a flexible and robust statistical language model is selected, and training optimization is achieved through self-training. Specifically, the application subdomain corresponding to the speech recognition system is obtained, and the language model corresponding to the application subdomain is determined. Then, the obtained language model is trained based on the annotated text corresponding to the annotated audio data.

[0067] Figure 7 FIG. 1 shows an operational flow chart of an example of a self-training module for a language model. Figure 7 As shown, the language model training module is based on a data management unit at the bottom layer. Based on the differences in the application fields of the entire recognition system, this data management unit divides the data into different granularities when classifying data, including online supervised annotated text data returned by annotation experts; the general text data module collects a large amount of written, spoken, interactive and other related field texts required by the speech recognition system; the T+1 data module uses a hot update method to update music, film, encyclopedia, character and other related corpora on a daily basis; the bug repair module mainly contains a large amount of statement data prepared to quickly repair online recognition bugs. The data of each sub-field needs to be divided into training sets and test sets. The training sets of each sub-field can generate the language model of that sub-field. The language model of each sub-field and the corresponding sub-field test set can use the confusion ppl (perplexity) to obtain the optimal interpolation coefficient of each sub-field in the optimal interpolation coefficient calculation module. In this embodiment of the present invention, the final language model of the speech recognition system is obtained by interpolating the language model of each sub-field with the optimal interpolation coefficient, and the corresponding decoding network is subsequently constructed for use in the next link.

[0068] In process node 8, the optimized acoustic model and language model are automatically processed together to form a test resource that can be used for engineering testing, and then enter the regression test, stress test and other testing stages to ensure the availability of the resource to be launched.

[0069] In process node 9, the resources to be put online after testing are pushed online for external services through the automatic online function.

[0070] In process node 10, the newly launched external service will accumulate new "online audio data to be screened" within a certain time period (such as one day).

[0071] At process node 11, some model resources (such as acoustic models) used in the newly launched external service will simultaneously update some model resources in the active learning data screening module; or some intermediate results (such as confidence levels) generated during the external service process will also become one of the input bases for data screening in the active learning data screening module. Therefore, through the closed loop of the system optimization process (screened data is used to generate the newly launched model, and the newly launched model is used to screen new data), on the one hand, it can continuously and effectively optimize the speech recognition effect at the lowest possible cost.

[0072] It should be noted that, unlike the traditional method of using only supervised data or combining supervised and unsupervised data, the embodiment scheme proposed in the present invention can make full use of the acquired data resources (including a large amount of data accumulated by the service and existing speech recognition results, etc.), give full play to the learning value of each data, and improve the optimization effect of iterative training. In the embodiment of the present invention, the automation of the entire process can also greatly shorten the cycle of each optimization and increase the frequency of optimization, so that the speech recognition system can be optimized and iterated more frequently, which can effectively solve the problem of inaccurate recognition of hot content within the cycle.

[0073] Figure 8 A flowchart of an example of a speech recognition method according to an embodiment of the present invention is shown.

[0074] like Figure 8 As shown, in step 810, voice data corresponding to the voice recognition request is obtained.

[0075] In step 820, the speech recognition system is called to determine the speech recognition result corresponding to the speech data. Here, the speech recognition system is optimized by the speech recognition system optimization method described in other embodiments above.

[0076] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of combined actions, but those skilled in the art should be aware that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention. In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0077] In some embodiments, an embodiment of the present invention provides a non-volatile computer-readable storage medium, which stores one or more programs including execution instructions, and the execution instructions can be read and executed by an electronic device (including but not limited to a computer, a server, or a network device, etc.) to execute any of the above-mentioned optimization methods of the speech recognition system of the present invention.

[0078] In some embodiments, an embodiment of the present invention further provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes any of the above-mentioned optimization methods for the speech recognition system.

[0079] In some embodiments, an embodiment of the present invention also provides an electronic device comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute an optimization method for a speech recognition system.

[0080] Figure 9 FIG. 1 is a schematic diagram of the hardware structure of an electronic device for executing a method for optimizing a speech recognition system according to another embodiment of the present invention. Figure 9 As shown, the device includes:

[0081] One or more processors 910 and memory 920, Figure 9 A processor 910 is taken as an example.

[0082] The device for executing the method for optimizing a speech recognition system may further include an input device 930 and an output device 940 .

[0083] The processor 910, the memory 920, the input device 930 and the output device 940 may be connected via a bus or other means. Figure 9 The bus connection is taken as an example.

[0084] Memory 920, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs, and modules, such as the program instructions / modules corresponding to the method for optimizing a speech recognition system in the embodiments of the present invention. Processor 910 executes the non-volatile software programs, instructions, and modules stored in memory 920 to execute various server functional applications and data processing, thereby implementing the method for optimizing a speech recognition system in the aforementioned method embodiment.

[0085] The memory 920 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created based on the use of the voice interaction device, etc. In addition, the memory 920 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 920 may optionally include a memory remotely located relative to the processor 910, and these remote memories may be connected to the voice interaction device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0086] The input device 930 can receive input digital or character information and generate signals related to user settings and function control of the voice interaction device. The output device 940 can include a display device such as a display screen.

[0087] The one or more modules are stored in the memory 920 and, when executed by the one or more processors 910 , perform the optimization method of the speech recognition system in any of the above method embodiments.

[0088] The above-mentioned product can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided by the embodiment of the present invention.

[0089] The electronic devices of the embodiments of the present invention exist in various forms, including but not limited to:

[0090] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and their primary purpose is to provide voice and data communications. These terminals include smartphones, multimedia phones, feature phones, and low-end phones.

[0091] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers and have computing and processing capabilities, and generally also have mobile Internet access. These terminals include PDAs, MIDs, and UMPCs.

[0092] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players, handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0093] (4) Other onboard electronic devices with data interaction functions, such as onboard computer devices installed in vehicles.

[0094] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0095] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, or of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiment.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for optimizing a speech recognition system, comprising: Acquire an initial data set, the initial data set comprising multiple audio data and corresponding matching recognition texts; Determining a first data subset and a second data subset based on an initial data set includes: identifying and filtering out invalid data in the initial data set; obtaining an identification confidence level corresponding to each data in the filtered initial data set, and determining the first data subset and the second data subset based on the identification confidence level; wherein the identification confidence level of the first data subset is greater than the identification confidence level of the second data subset; For each audio data in the filtered initial data set, obtain the recognition confidence of the audio data relative to the matching recognition text, and compare the recognition confidence with the recognition confidence intervals [a1], [a2] and [a3] corresponding to different recognition confidence levels, wherein [a1] < [a2] < [a3]; determine a second data subset based on the audio data corresponding to the recognition confidence interval [a2]; configure corresponding annotation text for the audio data corresponding to the recognition confidence interval [a1] to obtain corresponding annotated audio data; determine a first data subset based on the annotated audio data and the audio data corresponding to the recognition confidence interval [a3]; Based on the first data subset and the second data subset, the speech recognition system is subjected to mixed supervised training, wherein the first data subset is used for supervised training of the speech recognition system, and the second data subset is used for unsupervised training of the speech recognition system.

2. The method according to claim 1, wherein the speech recognition system comprises an acoustic model and a language model, in, Optimizing the speech recognition system based on the first data subset and the second data subset includes: Performing hybrid supervised training on the acoustic model based on the first data subset and the second data subset; The language model is trained based on the annotated text corresponding to the annotated audio data.

3. The method according to claim 2, wherein the performing hybrid supervised training on the acoustic model based on the first data subset and the second data subset comprises: Determining a plurality of data training units based on the first data subset and the second data subset; wherein a data volume ratio of the first data to the second data in each of the data training units is 1:N, where N>1; Based on each of the data training units, the acoustic model is sequentially subjected to mixed supervision training.

4. The method according to claim 3, wherein the hybrid supervised training of the acoustic model is performed sequentially based on each of the data training units, comprising: For each of the data training units, the acoustic model is subjected to unsupervised training corresponding to the first stage based on the first optimization parameter configuration and the second data in the data training unit, and the acoustic model is subjected to supervised training corresponding to the second stage based on the second optimization parameter configuration and the second data in the data training unit.

5. The method according to claim 2, wherein: The training of the language model based on the annotated text corresponding to the first data subset includes: Obtaining an application sub-field corresponding to the speech recognition system, and determining a language model corresponding to the application sub-field; The acquired language model is trained based on the annotated text corresponding to the annotated audio data.

6. A speech recognition method comprising: Get the voice data corresponding to the voice recognition request; Invoke a speech recognition system to determine a speech recognition result corresponding to the speech data; wherein the speech recognition system is optimized by the method according to any one of claims 1 to 5.

7. An electronic device comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1 to 6.

8. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images