Acoustic model optimization method and device applied to business scenarios

Abstract

The application discloses an acoustic model optimization method and device applied to a business scenario, applied to the field of artificial intelligence technology, and comprises the following steps: a general voice data is used to train a basic acoustic model to serve as a current acoustic model; a ratio of the general voice data and actual business scenario voice data is set as a current ratio; the following steps are repeatedly executed until a performance index of the current acoustic model reaches a preset requirement: according to the current ratio, mixed data of the general voice data and the actual business scenario voice data is generated; the current acoustic model is subjected to one round of optimization training based on the mixed data; after the one round of optimization training is completed, the performance index of the current acoustic model is calculated; and when the performance index does not meet the preset requirement, the current ratio is adjusted. The application can optimize the acoustic model applied to the business scenario and improve the performance of the acoustic model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an acoustic model optimization method and apparatus for application in business scenarios. Background Technology

[0002] This section is intended to provide background or context for the embodiments of the invention set forth in the claims. The description herein is not an admission that it is prior art simply because it is included in this section.

[0003] Currently, neural network-based speech recognition technology mainly relies on speech data as its driver, and the quantity and diversity of speech data directly determine the performance of speech recognition technology. Real-world business scenarios often contain uncommon characters and technical terms, leading to poor recognition results from acoustic models and ultimately degrading the performance of the speech recognition system. Therefore, it is necessary to optimize acoustic models for specific business scenarios.

[0004] Existing technologies only use real-world business scenario voice data to optimize acoustic models, resulting in poor final performance of the acoustic models. Summary of the Invention

[0005] This invention provides an acoustic model optimization method for business scenarios, used to optimize acoustic models applied in business scenarios and improve the performance of acoustic models. The method includes:

[0006] A basic acoustic model is trained using general speech data, which serves as the current acoustic model.

[0007] Set the ratio between general voice data and actual business scenario voice data as the current ratio;

[0008] Repeat the following steps until the performance metrics of the current acoustic model meet the preset requirements:

[0009] Based on the current ratio, generate mixed data of general voice data and actual business scenario voice data;

[0010] Based on the mixed data, the current acoustic model is subjected to one round of optimization training. After the first round of optimization training is completed, the performance index of the current acoustic model is calculated.

[0011] When the performance indicators do not meet the preset requirements, adjust the current ratio.

[0012] This invention also provides an acoustic model optimization device for business scenarios, used to optimize acoustic models applied in business scenarios and improve the performance of acoustic models. The device includes:

[0013] The basic acoustic model acquisition module is used to train a basic acoustic model using general speech data, which serves as the current acoustic model.

[0014] The ratio setting module is used to set the ratio between general voice data and actual business scenario voice data, which is then used as the current ratio.

[0015] The iteration module is used to repeatedly execute the following steps until the performance metrics of the current acoustic model meet the preset requirements:

[0016] Based on the current ratio, generate mixed data of general voice data and actual business scenario voice data;

[0017] Based on the mixed data, the current acoustic model is subjected to one round of optimization training. After the first round of optimization training is completed, the performance index of the current acoustic model is calculated.

[0018] When the performance indicators do not meet the preset requirements, adjust the current ratio.

[0019] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described acoustic model optimization method applied to a business scenario.

[0020] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned acoustic model optimization method applied to business scenarios.

[0021] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the aforementioned acoustic model optimization method applied to business scenarios.

[0022] In this embodiment of the invention, a basic acoustic model is trained using general-purpose speech data, which serves as the current acoustic model. A ratio is set between general-purpose speech data and actual business scenario speech data, which is then used as the current ratio. The following steps are repeated until the performance indicators of the current acoustic model meet preset requirements: Based on the current ratio, mixed data of general-purpose speech data and actual business scenario speech data is generated; based on the mixed data, the current acoustic model undergoes one round of optimization training; after one round of optimization training, the performance indicators of the current acoustic model are calculated; if the performance indicators do not meet the preset requirements, the current ratio is adjusted. Compared with existing technologies that only use actual business scenario speech data to optimize the acoustic model, this embodiment of the invention can dynamically mix general-purpose speech data and actual business scenario speech data. In each round of training, the ratio of general-purpose speech data to actual business scenario speech data is dynamically adjusted according to the performance indicators of the acoustic model until an optimal acoustic model is trained, thereby improving the performance of the acoustic model. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0024] Figure 1 This is a flowchart of an acoustic model optimization method applied to a business scenario in an embodiment of the present invention;

[0025] Figure 2 This is a flowchart illustrating the generation of mixed data in an embodiment of the present invention;

[0026] Figure 3 This is another flowchart illustrating the generation of mixed data in an embodiment of the present invention;

[0027] Figure 4 This is a flowchart illustrating the adjustment of the current ratio in an embodiment of the present invention;

[0028] Figure 5 This is a schematic diagram of an acoustic model optimization device applied to a business scenario in an embodiment of the present invention;

[0029] Figure 6 This is a schematic diagram of a computer device in an embodiment of the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0031] First, the terminology involved in this invention will be explained.

[0032] Acoustic model: The acoustic model is used to calculate the probability that a speech segment (such as a 10-millisecond speech segment) is a certain phoneme. It is one of the most important parts of the speech recognition system and directly determines the performance of speech recognition.

[0033] Business scenarios: refers to the business scenarios in which speech recognition technology is applied, such as intelligent customer service, intelligent outbound calling, and intelligent speakers.

[0034] Deep learning is a new research direction in the field of machine learning, introduced to bring it closer to its original goal—artificial intelligence. Deep learning learns the inherent patterns and hierarchical representations of sample data; the information gained during this learning process greatly aids in interpreting data such as text, images, and sound. Its ultimate goal is to enable machines to possess analytical and learning capabilities like humans, capable of recognizing data such as text, images, and sound. Deep learning is a complex machine learning algorithm that has achieved results in speech and image recognition far exceeding previous related technologies.

[0035] Speech recognition: generally refers to automatic speech recognition, which is a technology that converts human speech into computer text.

[0036] Acoustic model: Used to calculate the probability that a speech segment (e.g., a 10-millisecond speech segment) is a certain phoneme. It is one of the most important parts of a speech recognition system, directly determining the performance of speech recognition. Currently, the mainstream acoustic model is implemented using artificial neural networks.

[0037] Residual: Supervised learning neural networks need a function to measure the difference between their output value and the true value on a set of samples; this difference is called the residual.

[0038] Training set: A set of samples used for training, primarily for training the parameters in a neural network.

[0039] Validation set: A set of samples used to validate the model's performance during training. During neural network training, the model's performance is judged by calculating the residuals on the validation set; this set is sometimes also called the development set.

[0040] Figure 1 This is a flowchart of an acoustic model optimization method applied to a business scenario in an embodiment of the present invention, including:

[0041] Step 101: Use general speech data to train and obtain a basic acoustic model, which will serve as the current acoustic model;

[0042] Step 102: Set the ratio of general voice data to actual business scenario voice data as the current ratio;

[0043] Step 103: Repeat the following steps until the performance indicators of the current acoustic model meet the preset requirements:

[0044] Step 1031: Based on the current ratio, generate mixed data of general voice data and actual business scenario voice data;

[0045] Step 1032: Based on the mixed data, perform one round of optimization training on the current acoustic model. After the first round of optimization training is completed, calculate the performance index of the current acoustic model.

[0046] Step 1033: When the performance index does not meet the preset requirements, adjust the current ratio.

[0047] In this embodiment of the invention, compared with the prior art which only uses actual business scenario voice data to optimize the acoustic model, this embodiment of the invention can dynamically mix general voice data and actual business scenario voice data. In each round of training, the ratio of general voice data and actual business scenario voice data is dynamically adjusted according to the performance index of the acoustic model until the optimal acoustic model is trained, thereby improving the performance of the acoustic model.

[0048] Each step is described in detail below.

[0049] In step 101, a basic acoustic model is trained using general speech data and used as the current acoustic model.

[0050] Specifically, the larger the amount of general speech data used, the more accurate the basic acoustic model will be, and this basic acoustic model will not be optimized for any specific scenario.

[0051] In step 102, the ratio of general voice data to actual business scenario voice data is set as the current ratio;

[0052] This step sets the initial ratio, which needs to be dynamically adjusted later.

[0053] In step 103, repeat the following steps until the performance indicators of the current acoustic model meet the preset requirements:

[0054] In step 1031, based on the current ratio, a mixture of general voice data and actual business scenario voice data is generated;

[0055] Figure 2 This is a flowchart illustrating the generation of mixed data in an embodiment of the present invention. In one embodiment, mixed data of general voice data and actual business scenario voice data is generated according to the current ratio, including:

[0056] Step 201: Determine the current required duration of general voice data based on the duration and current proportion of voice data in the actual business scenario;

[0057] Step 202: Select the general speech data of the desired duration from the general speech data;

[0058] Step 203: Mix the actual business scenario voice data with the general voice data of the required duration to obtain mixed data.

[0059] Figure 3 As another flowchart for generating mixed data in this embodiment of the invention, after mixing the actual business scenario voice data with the general voice data of the currently required duration (step 203), the method further includes:

[0060] Step 301: Delete the general speech data that is already in use in the general speech data.

[0061] Because general-purpose speech data (tens of thousands of hours) far exceeds actual business scenario speech data (tens to hundreds of hours), each training round must use all actual business scenario speech data, but only a small portion of general-purpose speech data. Furthermore, the general-purpose speech data differs between training rounds. This allows for full utilization of the general-purpose speech data when optimizing the acoustic model. For example, if there are 10,000 hours of general-purpose speech data and 100 hours of actual business scenario speech data, and the ratio of general-purpose speech data to actual business scenario speech data calculated in B is 2:1, then each training round will use all 100 hours of actual business scenario speech data, while only 200 hours of general-purpose speech data will be used. After these 200 hours of general-purpose speech data are used, they will not be used in subsequent training rounds until all 10,000 hours of speech data have been used, at which point they will be reused.

[0062] In one embodiment, the actual business scenario voice data is either single business scenario voice data or mixed data of multiple business scenario voice data.

[0063] In step 1032, based on the mixed data, the current acoustic model is subjected to one round of optimization training. After the one round of optimization training is completed, the performance index of the current acoustic model is calculated.

[0064] In one embodiment, the performance metrics include a first residual of the current acoustic model on a validation set of general speech data and a second residual of the current acoustic model on a validation set of speech data from a real-world business scenario.

[0065] The preset requirement is that the difference between the first residual after the current round of optimization training and the first residual after the previous round of optimization training is within a first preset range, and the difference between the second residual after the current round of optimization training and the second residual after the previous round of optimization training is within a second preset range.

[0066] In one embodiment, the basic acoustic model is constructed using a neural network, and the residual is the neural network residual. Using all the mixed data at once constitutes one round of optimization training.

[0067] In step 1033, when the performance index does not meet the preset requirements, the current ratio is adjusted.

[0068] Figure 4 This is a flowchart illustrating the adjustment of the current ratio in an embodiment of the present invention. In one embodiment, adjusting the current ratio when the performance indicator does not meet the preset requirements includes:

[0069] Step 401: When the difference between the first residual after this round of optimization training and the first residual after the previous round of optimization training exceeds the first preset range, increase the proportion of general speech data.

[0070] Specifically, the difference between the first residual after this round of optimization training and the first residual after the previous round of optimization training becomes larger, that is, when the performance of the acoustic model on general speech data deteriorates, the proportion of general speech data is increased.

[0071] Step 402: When the difference between the second residual after this round of optimization training and the second residual after the previous round of optimization training exceeds the second preset range, increase the proportion of actual business scenario voice data.

[0072] Specifically, the difference between the second residual after this round of optimization training and the second residual after the previous round of optimization training becomes larger. That is, when the performance of the acoustic model on actual business scenario data deteriorates, the proportion of actual business scenario voice data is increased.

[0073] Finally, training ends when the residuals of the optimized model on the general speech data validation set and the residuals on the real-world business data validation set no longer decrease, and the acoustic model reaches its optimal performance.

[0074] In summary, the method proposed in the embodiments of the present invention has the following beneficial effects:

[0075] First, dynamic data mixing. General speech data is dynamically mixed with actual business scenario data. This is because the amount of general speech data (tens of thousands of hours) is far greater than that of actual business scenario data (tens to hundreds of hours). Each training round uses all actual business scenario data, but only a small portion of general speech data. At the same time, the general speech data is different between each training round. This way, when optimizing the acoustic model, the general speech data can be fully utilized.

[0076] Second, during each round of training, the ratio of general speech data to actual business scenario data is dynamically adjusted based on the model's training performance. When the acoustic model's performance on general speech data declines, the proportion of general speech data is increased; when the acoustic model's performance on actual business scenario data declines, the proportion of general speech data is decreased.

[0077] Third, the acoustic model is optimized by mixing voice data from multiple business scenarios, thereby training an acoustic model that is applicable to multiple real-world business scenarios.

[0078] This invention also proposes an acoustic model optimization device for business scenarios, the principle of which is similar to the acoustic model optimization method for business scenarios, and will not be described in detail here.

[0079] Figure 5 This is a schematic diagram of an acoustic model optimization device applied to a business scenario in an embodiment of the present invention, including:

[0080] The basic acoustic model acquisition module 501 is used to train a basic acoustic model using general speech data, which serves as the current acoustic model.

[0081] The ratio setting module 502 is used to set the ratio between general voice data and actual business scenario voice data as the current ratio.

[0082] Iteration module 503 is used to repeatedly execute the following steps until the performance indicators of the current acoustic model meet the preset requirements:

[0083] Based on the current ratio, generate mixed data of general voice data and actual business scenario voice data;

[0084] Based on the mixed data, the current acoustic model is subjected to one round of optimization training. After the first round of optimization training is completed, the performance index of the current acoustic model is calculated.

[0085] When the performance indicators do not meet the preset requirements, adjust the current ratio.

[0086] In one embodiment, the iteration module is specifically used for:

[0087] Determine the required duration of general voice data based on the duration and current proportion of voice data in the actual business scenario;

[0088] Select the desired duration of general speech data from the general speech data;

[0089] Mix the actual business scenario voice data with the general voice data required for the current duration to obtain mixed data.

[0090] In one embodiment, the iteration module is specifically used for:

[0091] After mixing the actual business scenario voice data with the general voice data required for the current duration, delete the general voice data that has already been used in the general voice data.

[0092] In one embodiment, the performance metrics include a first residual of the current acoustic model on a validation set of general speech data and a second residual of the current acoustic model on a validation set of speech data from a real-world business scenario.

[0093] The preset requirement is that the difference between the first residual after the current round of optimization training and the first residual after the previous round of optimization training is within a first preset range, and the difference between the second residual after the current round of optimization training and the second residual after the previous round of optimization training is within a second preset range.

[0094] In one embodiment, the basic acoustic model is constructed using a neural network, and the residual is a neural network residual.

[0095] In one embodiment, the iteration module is specifically used for:

[0096] When the difference between the first residual after this round of optimization training and the first residual after the previous round of optimization training exceeds the first preset range, the proportion of general speech data is increased.

[0097] When the difference between the second residual after this round of optimization training and the second residual after the previous round of optimization training exceeds the second preset range, the proportion of actual business scenario voice data is increased.

[0098] In one embodiment, the actual business scenario voice data is either single business scenario voice data or mixed data of multiple business scenario voice data.

[0099] In summary, the device proposed in the embodiments of the present invention has the following beneficial effects:

[0100] First, dynamic data mixing. General speech data is dynamically mixed with actual business scenario data. This is because the amount of general speech data (tens of thousands of hours) is far greater than that of actual business scenario data (tens to hundreds of hours). Each training round uses all actual business scenario data, but only a small portion of general speech data. At the same time, the general speech data is different between each training round. This way, when optimizing the acoustic model, the general speech data can be fully utilized.

[0101] Second, during each round of training, the ratio of general speech data to actual business scenario data is dynamically adjusted based on the model's training performance. When the acoustic model's performance on general speech data declines, the proportion of general speech data is increased; when the acoustic model's performance on actual business scenario data declines, the proportion of general speech data is decreased.

[0102] Third, the acoustic model is optimized by mixing voice data from multiple business scenarios, thereby training an acoustic model that is applicable to multiple real-world business scenarios.

[0103] This invention also provides a computer device. Figure 6This is a schematic diagram of a computer device in an embodiment of the present invention. The computer device 600 includes a memory 610, a processor 620, and a computer program 630 stored in the memory 610 and executable on the processor 620. When the processor 620 executes the computer program 630, it implements the above-mentioned acoustic model optimization method applied to business scenarios.

[0104] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described acoustic model optimization method applied to a business scenario.

[0105] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned acoustic model optimization method applied to business scenarios.

[0106] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the aforementioned acoustic model optimization method applied to business scenarios.

[0107] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0108] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0109] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0110] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0111] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An acoustic model optimization method applied to business scenarios, characterized in that, include: A basic acoustic model is trained using general speech data, which serves as the current acoustic model. Set the ratio between general voice data and actual business scenario voice data as the current ratio; Repeat the following steps until the performance metrics of the current acoustic model meet the preset requirements: Based on the current ratio, generate mixed data of general voice data and actual business scenario voice data; Based on the mixed data, the current acoustic model is subjected to one round of optimization training. After the first round of optimization training is completed, the performance index of the current acoustic model is calculated. When the performance indicators do not meet the preset requirements, adjust the current ratio; The performance metrics include the first residual of the current acoustic model on a validation set of general speech data and the second residual of the current acoustic model on a validation set of speech data in actual business scenarios; The preset requirement is that the difference between the first residual after the current round of optimization training and the first residual after the previous round of optimization training is within a first preset range, and the difference between the second residual after the current round of optimization training and the second residual after the previous round of optimization training is within a second preset range.

2. The method as described in claim 1, characterized in that, Based on the current ratio, generate mixed data of general voice data and actual business scenario voice data, including: Determine the required duration of general voice data based on the duration and current proportion of voice data in the actual business scenario; Select the desired duration of general speech data from the general speech data; Mix the actual business scenario voice data with the general voice data required for the current duration to obtain mixed data.

3. The method as described in claim 2, characterized in that, After mixing the actual business scenario voice data with the general voice data required for the current duration, it also includes: Remove the general speech data that is already in use from the general speech data.

4. The method as described in claim 1, characterized in that, The basic acoustic model is constructed using a neural network, and the residual is the neural network residual.

5. The method as described in claim 1, characterized in that, When the performance indicators do not meet the preset requirements, adjust the current ratio, including: When the difference between the first residual after this round of optimization training and the first residual after the previous round of optimization training exceeds the first preset range, the proportion of general speech data is increased. When the difference between the second residual after this round of optimization training and the second residual after the previous round of optimization training exceeds the second preset range, the proportion of actual business scenario voice data is increased.

6. The method as described in claim 1, characterized in that, In actual business scenarios, voice data may be either voice data from a single business scenario or a mixture of voice data from multiple business scenarios.

7. An acoustic model optimization device for business scenarios, characterized in that, include: The basic acoustic model acquisition module is used to train a basic acoustic model using general speech data, which serves as the current acoustic model. The ratio setting module is used to set the ratio between general voice data and actual business scenario voice data, which is then used as the current ratio. The iteration module is used to repeatedly execute the following steps until the performance metrics of the current acoustic model meet the preset requirements: Based on the current ratio, generate mixed data of general voice data and actual business scenario voice data; Based on the mixed data, the current acoustic model is subjected to one round of optimization training. After the first round of optimization training is completed, the performance index of the current acoustic model is calculated. When the performance indicators do not meet the preset requirements, adjust the current ratio; The performance metrics include the first residual of the current acoustic model on a validation set of general speech data and the second residual of the current acoustic model on a validation set of speech data in actual business scenarios; The preset requirement is that the difference between the first residual after the current round of optimization training and the first residual after the previous round of optimization training is within a first preset range, and the difference between the second residual after the current round of optimization training and the second residual after the previous round of optimization training is within a second preset range.

8. The apparatus as claimed in claim 7, characterized in that, The iteration module is specifically used for: Determine the required duration of general voice data based on the duration and current proportion of voice data in the actual business scenario; Select the desired duration of general speech data from the general speech data; Mix the actual business scenario voice data with the general voice data required for the current duration to obtain mixed data.

9. The apparatus as claimed in claim 8, characterized in that, The iteration module is specifically used for: After mixing the actual business scenario voice data with the general voice data required for the current duration, delete the general voice data that has already been used in the general voice data.

10. The apparatus as claimed in claim 7, characterized in that, The basic acoustic model is constructed using a neural network, and the residual is the neural network residual.

11. The apparatus as claimed in claim 7, characterized in that, The iteration module is specifically used for: When the difference between the first residual after this round of optimization training and the first residual after the previous round of optimization training exceeds the first preset range, the proportion of general speech data is increased. When the difference between the second residual after this round of optimization training and the second residual after the previous round of optimization training exceeds the second preset range, the proportion of actual business scenario voice data is increased.

12. The apparatus as claimed in claim 7, characterized in that, In actual business scenarios, voice data may be either voice data from a single business scenario or a mixture of voice data from multiple business scenarios.

13. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1 to 6.

15. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 6.

Images