A method, device, equipment and storage medium for optimizing speech recognition model

Abstract

The present application provides a method, device, equipment and storage medium for optimizing a speech recognition model, the method comprising: obtaining a first speech set and a second speech set; based on the result of natural language processing of a first text set corresponding to the first speech set and a second text set corresponding to the second speech set, intercepting corresponding speech segments from the first speech set and intercepting corresponding speech segments from the second speech set to obtain a first speech segment set and a second speech segment set; selecting at least one first speech, at least one first speech segment and at least one second speech segment from the first speech set, the first speech segment set and the second speech segment set to form a combined speech set, and annotating the combined speech set to obtain a sample set; optimizing the speech recognition model using the sample set. The method of the present application can improve the accuracy of speech recognition and the accuracy of the results obtained by natural language processing of speech recognition results.

Description

Technical Field

[0001] The present application relates to the field of speech recognition, and in particular, to a method, apparatus, device and storage medium for optimizing a speech recognition model. Background Art

[0002] With the rapid development of the information age, companies that provide sales, consultation, complaints and other services to customers over the phone basically require voice communication. Therefore, using artificial intelligence technology to identify and analyze call voice can obtain a lot of valuable information and help companies operate and develop better.

[0003] At present, the training of speech recognition models only involves finding speech samples with low speech recognition accuracy and manually labeling them to optimize the existing models.

[0004] However, in the above optimization process, only the overall accuracy of the model in speech recognition can be improved, and the accuracy of the natural language task in recognizing more important speech segments in the speech after speech recognition cannot be effectively improved, resulting in the situation where the accuracy of the recognition results of the natural language task remains unchanged or even decreases.

[0005] Therefore, how to improve the accuracy of speech recognition and the accuracy of the results obtained by natural language processing of speech recognition results is a technical problem that needs to be solved. Summary of the invention

[0006] The purpose of the embodiments of the present application is to provide a method for optimizing a speech recognition model. Through the technical solution of the embodiments of the present application, the accuracy of speech recognition and the accuracy of the results obtained by performing natural language processing on the speech recognition results can be improved.

[0007] In a first aspect, an embodiment of the present application provides a method for optimizing a speech recognition model, comprising: obtaining a first speech set and a second speech set, wherein the first speech set includes speech whose recognition accuracy of the speech recognition model is less than an accuracy threshold, and the second speech set includes speech whose recognition accuracy of the speech recognition model is greater than or equal to the accuracy threshold; according to the results of natural language processing of a first text set corresponding to the first speech set and a second text set corresponding to the second speech set, extracting corresponding speech segments from the first speech set and extracting corresponding speech segments from the second speech set to obtain a first speech segment set and a second speech segment set; selecting at least one first speech, at least one first speech segment and at least one second speech segment from the first speech set, the first speech segment set and the second speech segment set to form a combined speech set, and annotating the combined speech set to obtain a sample set, wherein the sample set includes a first speech, a text corresponding to the first speech with accurate recognition and annotated text corresponding to the first speech, a first speech segment and a text obtained by natural language processing of the first speech segment with accurate recognition, and a second speech segment and a text obtained by natural language processing of the second speech segment; and optimizing the speech recognition model using the sample set.

[0008] In the above process, the speech recognition model is optimized by means of a set consisting of the speech and speech segments in the first speech set, the first speech segment set and the second speech segment set, and the text annotations corresponding to the set, wherein the accuracy of speech recognition can be improved by optimizing the model by speech, and the accuracy of the results of natural language processing after speech recognition can be improved by optimizing the model by speech segments, and the accuracy of speech recognition and the accuracy of the results of natural language processing can be improved as a whole by optimizing the model by the above-mentioned speech and speech segments.

[0009] In one embodiment, obtaining the first speech set and the second speech set includes:

[0010] Randomly extract a preset number of voices from the database, and use the voice recognition model to perform voice recognition on the preset number of voices to obtain recognition results, wherein the recognition results include the recognition accuracy rate of each voice;

[0011] Based on the recognition result, selecting speech with recognition accuracy less than an accuracy threshold from a preset number of speech to obtain a first speech set;

[0012] Based on the recognition result, voices with a recognition accuracy greater than or equal to an accuracy threshold are screened from a preset number of voices to obtain a second voice set.

[0013] In the above process, the speech is divided into two sets based on the recognition accuracy. Optimizing the speech recognition model through the speech in the first speech set can improve the recognition accuracy of the speech model. Optimizing the speech recognition model through the speech fragments obtained from the speech in the first speech set and the second speech set can improve the accuracy of the results of natural language processing of the speech recognition results.

[0014] In one embodiment, according to the result of natural language processing of a first text set corresponding to a first voice set and a second text set corresponding to a second voice set, corresponding voice segments are intercepted from the first voice set, and corresponding voice segments are intercepted from the second voice set to obtain the first voice segment set and the second voice segment set, including:

[0015] Performing natural language processing on a first text set corresponding to the first voice set and a second text set corresponding to the second voice set, respectively, to obtain a first text segment set and a second text segment set;

[0016] Based on the first text segment set, corresponding speech segments are intercepted from the first speech set, and based on the second text segment set, corresponding speech segments are intercepted from the second speech set to obtain the first speech segment set and the second speech segment set.

[0017] In the above process, natural language processing is performed on the texts corresponding to the two sets respectively to obtain corresponding text fragments, and corresponding voice fragments can be obtained based on the text fragments. The speech recognition model is optimized through the voice fragments, which can improve the accuracy of the results obtained by natural language processing of the recognition results.

[0018] In one embodiment, at least one first speech, at least one first speech segment, and at least one second speech segment are selected from the first speech set, the first speech segment set, and the second speech segment set to form a combined speech set, and the combined speech set is annotated to obtain a sample set, including:

[0019] Determine the ratio of the first speech, the first speech segment, and the second speech segment in the sample set, wherein the ratio of the first speech, the first speech segment, and the second speech segment in the sample set is determined based on the total duration of all speech corresponding to each set of the first speech set, the first speech segment set, and the second speech segment set;

[0020] Based on the ratio of the first speech, the first speech fragment and the second speech fragment in the sample set, a combined speech set is determined, and the combined speech set is labeled to obtain a sample set, wherein the combined speech set is obtained by selecting a corresponding number of first speech, first speech fragment and second speech fragment from the first speech set, the first speech fragment set and the second speech fragment set based on the ratio of the first speech, the first speech fragment and the second speech fragment in the sample set.

[0021] In the above process, by using a certain proportion of speech and speech fragments and the annotation of each speech and each speech, the proportion and number of samples in the sample set can be accurately limited, and then the model can be optimized, which can maximize the recognition accuracy of the model and the accuracy of the results obtained by natural language processing of the recognition results.

[0022] In one embodiment, after optimizing the speech recognition model using the sample set, the method further includes:

[0023] Use the speech in the test set to test the optimized speech recognition model and obtain the test results;

[0024] If the test result shows that the recognition accuracy of the optimized speech recognition model for the speech in the test set is lower than the preset second accuracy threshold, increase the proportion of the first speech in the sample set and optimize the optimized speech recognition model;

[0025] If the test results show that the accuracy of the results obtained by the optimized speech recognition model for natural language processing of the speech in the test set is lower than the preset third accuracy threshold, increase the ratio of the first speech segment and the second speech segment in the sample set, and optimize the optimized speech recognition model.

[0026] In the above process, the proportion of corresponding samples can be increased in a targeted manner according to the accuracy of model recognition and the accuracy of the results obtained by natural language processing, thereby improving the recognition accuracy of the model and the accuracy of the results obtained by natural language processing of the recognition results.

[0027] In a second aspect, an embodiment of the present application provides a method for speech recognition, comprising:

[0028] Acquire speech to be recognized; input the speech to be recognized into an optimized speech recognition model to obtain a recognition result, wherein the optimized speech recognition model is obtained by optimizing the existing speech recognition model by selecting at least one first speech, at least one first speech segment and at least one second speech segment from a first speech set, a first speech segment set and a second speech segment set to form a combined speech set, and annotating the combined speech set to obtain a sample set, wherein the first speech set includes speech whose recognition accuracy of the speech recognition model is less than an accuracy threshold, the first speech segment is obtained by intercepting the corresponding speech segment from the first speech set based on the result of natural language processing of a first text set corresponding to the first speech set, the second speech set includes speech whose recognition accuracy of the speech recognition model is greater than or equal to the accuracy threshold, and the second speech segment is obtained by intercepting the corresponding speech segment from the second speech set based on the result of natural language processing corresponding to a second text set corresponding to the second speech set.

[0029] In the above process, the speech recognition model is optimized by using a set consisting of the speech and speech segments in the first speech set, the first speech segment set and the second speech segment set, and the text annotations corresponding to the set. This can make the model have a higher recognition accuracy for the speech to be recognized, and the accuracy of the natural language processing results of the recognition results is also high.

[0030] In a second aspect, an embodiment of the present application provides a device for optimizing a speech recognition model, comprising:

[0031] An acquisition module, configured to acquire a first speech set and a second speech set, wherein the first speech set includes speech whose recognition accuracy of the speech recognition model is less than an accuracy threshold, and the second speech set includes speech whose recognition accuracy of the speech recognition model is greater than or equal to the accuracy threshold;

[0032] An interception module, configured to intercept corresponding speech segments from the first speech set and intercept corresponding speech segments from the second speech set according to the result of natural language processing of the first text set corresponding to the first speech set and the second text set corresponding to the second speech set, so as to obtain a first speech segment set and a second speech segment set;

[0033] A combination module, used for selecting at least one first voice, at least one first voice segment and at least one second voice segment from the first voice set, the first voice segment set and the second voice segment set to form a combined voice set, and annotating the combined voice set to obtain a sample set, wherein the sample set includes the first voice, the text corresponding to the first voice accurately recognized and the annotated text corresponding to the first voice, the first voice segment and the text obtained by natural language processing of the first voice segment accurately corresponding to the text, and the second voice segment and the text obtained by natural language processing of the second voice segment;

[0034] The optimization module is used to optimize the speech recognition model using the sample set.

[0035] Optionally, the acquisition module is specifically used to:

[0036] Randomly extract a preset number of voices from the database, and use the voice recognition model to perform voice recognition on the preset number of voices to obtain recognition results, wherein the recognition results include the recognition accuracy rate of each voice;

[0037] Based on the recognition result, selecting speech with recognition accuracy less than an accuracy threshold from a preset number of speech to obtain a first speech set;

[0038] Based on the recognition result, voices with a recognition accuracy greater than or equal to an accuracy threshold are screened from a preset number of voices to obtain a second voice set.

[0039] Optionally, the interception module is specifically used to:

[0040] Performing natural language processing on a first text set corresponding to the first voice set and a second text set corresponding to the second voice set, respectively, to obtain a first text segment set and a second text segment set;

[0041] Based on the first text segment set, corresponding speech segments are intercepted from the first speech set, and based on the second text segment set, corresponding speech segments are intercepted from the second speech set to obtain the first speech segment set and the second text segment set.

[0042] Optionally, the combined module is specifically used for:

[0043] Determine the ratio of the first speech, the first speech segment, and the second speech segment in the sample set, wherein the ratio of the first speech, the first speech segment, and the second speech segment in the sample set is determined based on the total duration of all speech corresponding to each set of the first speech set, the first speech segment set, and the second speech segment set;

[0044] Based on the ratio of the first speech, the first speech fragment and the second speech fragment in the sample set, a combined speech set is determined, and the combined speech set is labeled to obtain a sample set, wherein the combined speech set is obtained by selecting a corresponding number of first speech, first speech fragment and second speech fragment from the first speech set, the first speech fragment set and the second speech fragment set based on the ratio of the first speech, the first speech fragment and the second speech fragment in the sample set.

[0045] Optionally, the device further comprises:

[0046] A test adjustment module, which is used for testing the optimized speech recognition model using speech in a test set after the optimization module optimizes the speech recognition model using a sample set, to obtain a test result;

[0047] If the test result shows that the recognition accuracy of the optimized speech recognition model for the speech in the test set is lower than the preset second accuracy threshold, increase the proportion of the first speech in the sample set and optimize the optimized speech recognition model;

[0048] If the test results show that the accuracy of the results obtained by the optimized speech recognition model for natural language processing of the speech in the test set is lower than the preset third accuracy threshold, increase the ratio of the first speech segment and the second speech segment in the sample set, and optimize the optimized speech recognition model.

[0049] In a fourth aspect, an embodiment of the present application provides a speech recognition device, including:

[0050] An acquisition module, used to acquire the speech to be recognized;

[0051] A recognition module is used to input the speech to be recognized into an optimized speech recognition model to obtain a recognition result, wherein the optimized speech recognition model is obtained by optimizing the existing speech recognition model by selecting at least one first speech, at least one first speech segment and at least one second speech segment from a first speech set, a first speech segment set and a second speech segment set to form a combined speech set, and annotating the combined speech set to obtain a sample set. The first speech set includes speech whose recognition accuracy of the speech recognition model is less than an accuracy threshold, the first speech segment is obtained by intercepting the corresponding speech segment from the first speech set based on the result of natural language processing of a first text set corresponding to the first speech set, the second speech set includes speech whose recognition accuracy of the speech recognition model is greater than or equal to the accuracy threshold, and the second speech segment is obtained by intercepting the corresponding speech segment from the second speech set based on the result of natural language processing corresponding to a second text set corresponding to the second speech set.

[0052] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method provided in the first or second aspect above are executed.

[0053] In a sixth aspect, an embodiment of the present application provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the method provided in the first or second aspect are performed.

[0054] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the embodiments of the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 A schematic diagram of a scenario of a speech recognition task and a natural language processing task provided in an embodiment of the present application;

[0057] Figure 2 A schematic diagram of the structure of a method for optimizing a speech recognition model provided in an embodiment of the present application;

[0058] Figure 3 A flowchart of a method for speech recognition provided in an embodiment of the present application;

[0059] Figure 4 A schematic block diagram of a device for optimizing a speech recognition model provided in an embodiment of the present application;

[0060] Figure 5 A schematic block diagram of the structure of a device for optimizing a speech recognition model provided in an embodiment of the present application;

[0061] Figure 6 A schematic block diagram of a speech recognition device provided in an embodiment of the present application;

[0062] Figure 7 A schematic block diagram of the structure of a speech recognition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0064] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0065] First, some terms involved in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0066] Speech recognition technology: also known as Automatic Speech Recognition (ASR), its goal is to convert the vocabulary content in human speech into computer-readable input, such as keystrokes, binary codes, or character sequences.

[0067] Natural Language Processing (NLP) is a science that integrates linguistics, computer science, and mathematics. It is a variety of theories and methods that can enable effective communication between people and computers using natural language.

[0068] This application is applied to the scenario of speech recognition, specifically in the call system, the speech recognition model in the upstream task is used to recognize the speech, and the natural language processing task in the downstream task is used to extract information from the speech recognition results to jointly meet the needs of users. For example, an enterprise provides sales, consulting, complaint and other services to customers over the phone. A large amount of recognition text can be obtained by using the speech recognition model to recognize and analyze call recordings, and the feature information of user needs in the recognition text can be obtained through the NLP (natural language processing) task.

[0069] However, the current training of speech recognition models is only to find speech samples with low speech recognition accuracy and manually annotate them to optimize the existing model. In the above optimization process, the overall accuracy of the model in recognizing speech can only be improved, and the accuracy of natural language processing technology in extracting more important feature information in speech during the recognition process cannot be improved. In addition, when optimizing the model, a large amount of speech data is used to optimize the model, but the accuracy of the results obtained by natural language processing of the recognized text after the model recognizes the speech cannot be improved. For example, in some speech, the duration of the feature information is very short, and the corresponding recognized text is also very small. Although the overall recognition accuracy of the model has been improved, the recognition accuracy of the speech segment corresponding to the corresponding part of the feature information may not be improved, or even reduced.

[0070] To this end, the present application optimizes the model by adding some speech segments corresponding to the feature information extracted by the natural language processing task on the basis of the original training samples. Specifically, the present application obtains a first speech set and a second speech set; based on the results of natural language processing of a first text set corresponding to the first speech set and a second text set corresponding to the second speech set, the corresponding speech segments are intercepted from the first speech set and the corresponding speech segments are intercepted from the second speech set to obtain a first speech segment set and a second speech segment set; at least one first speech, at least one first speech segment and at least one second speech segment are selected from the first speech set, the first speech segment set and the second speech segment set to form a combined speech set, and the combined speech set is labeled to obtain a sample set; the speech recognition model is optimized using the sample set. The method of the present application can achieve the effect of improving the accuracy of speech recognition and the accuracy of the results obtained by natural language processing of the speech recognition results.

[0071] In the embodiment of the present application, the execution entity can be a speech recognition model optimization device in a speech recognition model optimization system. In actual applications, the speech recognition model optimization device can be electronic devices such as terminal devices and servers, which is not limited here.

[0072] Combine the following Figure 1 The scenarios of speech recognition tasks and natural language processing tasks in the embodiments of the present application are described in detail.

[0073] Please see Figure 1 , Figure 1 A scenario diagram of a speech recognition task and a natural language processing task provided in an embodiment of the present application, the specific scenario is as follows:

[0074] The entire process of speech recognition and feature information extraction includes upstream tasks and downstream tasks. The upstream task performs speech recognition on the call recordings in the dialogue system through the speech recognition task to obtain the corresponding recognition text, and the downstream task performs natural language processing on the recognition text through the natural language processing task to finally obtain the feature information corresponding to the recording.

[0075] Among them, there is a large amount of call recording data stored in the call system. The speech recognition task is used to recognize the call recordings into text through the speech recognition model. The natural language processing task is used to recognize the text and extract feature information. The feature information includes region, role (user object), scene, customer occupation, purchased products, service introduction, seat quotation and recommended products and other data information.

[0076] Combine the following Figure 2 The method for optimizing the speech recognition model in the embodiment of the present application is described in detail.

[0077] Please see Figure 2 , Figure 2 A flowchart of a method for optimizing a speech recognition model provided in an embodiment of the present application is shown in FIG. Figure 2 The method for optimizing the speech recognition model shown includes:

[0078] Step 210: Acquire a first speech set and a second speech set.

[0079] The first speech set includes speech whose recognition accuracy of the speech recognition model is less than the accuracy threshold, and the second speech set includes speech whose recognition accuracy of the speech recognition model is greater than or equal to the accuracy threshold. The accuracy threshold can be set according to actual needs.

[0080] Specifically, when executing step 210, the following method may also be used:

[0081] A preset number of speech sounds are randomly extracted from a database, and speech recognition is performed on the preset number of speech sounds using a speech recognition model to obtain a recognition result; based on the recognition result, speech sounds with a recognition accuracy rate less than an accuracy threshold are screened from the preset number of speech sounds to obtain a first speech set; based on the recognition result, speech sounds with a recognition accuracy rate greater than or equal to the accuracy threshold are screened from the preset number of speech sounds to obtain a second speech set.

[0082] In the above process, the speech is divided into two sets based on the recognition accuracy. Optimizing the speech recognition model through the speech in the first speech set can improve the recognition accuracy of the speech model. Optimizing the speech recognition model through the speech fragments obtained from the speech in the first speech set and the second speech set can improve the accuracy of the results of natural language processing of the speech recognition results.

[0083] The recognition result includes the recognition accuracy of each voice. The preset number of voices can be set according to user needs or obtained after multiple optimization models. In addition, recordings with better and worse accuracy can be selected according to quantiles (such as third and fourth quartiles), and all voices can be divided into three or four parts according to the recognition accuracy. The part with high accuracy is taken as the first voice set, and the part with low accuracy is taken as the second voice set.

[0084] Step 220: Based on the results of natural language processing of the first text set corresponding to the first voice set and the second text set corresponding to the second voice set, corresponding voice segments are extracted from the first voice set, and corresponding voice segments are extracted from the second voice set to obtain a first voice segment set and a second voice segment set.

[0085] In the above process, by extracting corresponding text segments from the first speech set and the second speech set, the speech recognition model can be optimized to improve the accuracy of natural language processing of the recognition results of the speech recognition model.

[0086] Among them, natural language processing can be used to extract feature information of user needs in speech recognition text.

[0087] Specifically, when executing step 220, the following method may also be used:

[0088] Natural language processing is performed on a first text set corresponding to the first voice set and a second text set corresponding to the second voice set, respectively, to obtain a first text segment set and a second text segment set; corresponding voice segments are intercepted from the first voice set based on the first text segment set, and corresponding voice segments are intercepted from the second voice set based on the second text segment set, to obtain a first voice segment set and a second voice segment set.

[0089] In the above process, natural language processing is performed on the texts corresponding to the two sets respectively to obtain corresponding text fragments, and corresponding voice fragments can be obtained based on the text fragments. The speech recognition model is optimized through the voice fragments, which can improve the accuracy of the results obtained by natural language processing of the recognition results.

[0090] Among them, after natural language processing and before intercepting the corresponding voice segment, the text obtained by natural language processing needs to be preprocessed, including: expanding some incomplete text segments into a whole sentence or adding multiple characters before and after the text segment, filling the text segment into a whole sentence of text, and obtaining the final target text segment. It is also necessary to calculate the time interval of the speech corresponding to the target text segment, and expand the corresponding speech into a whole sentence of speech according to the time interval or delete some useless characters and shrink the speech segment corresponding to the sentence into a whole sentence, etc. For example: if the silence duration exceeds 500 milliseconds, discard the silence interval, discard a very short time interval (such as less than 1 second), expand the time interval, for example: add 2 seconds of silent speech at the beginning and end, in addition, if the time interval is too long, the interval can be divided into multiple segments by segmentation to obtain the final target time interval. According to the time interval, the corresponding voice segment can be obtained from the corresponding speech.

[0091] In addition, when natural language is used to correctly annotate the speech corresponding to the speech without manual speech annotation, the speech segments annotated with natural language can be directly used as the speech segments in the first speech set and the second speech set.

[0092] Step 230: Select at least one first speech, at least one first speech segment and at least one second speech segment from the first speech set, the first speech segment set and the second speech segment set to form a combined speech set, and annotate the combined speech set to obtain a sample set.

[0093] In the above process, the model is optimized through a sample set consisting of speech, speech fragments and corresponding annotations, which can improve the accuracy of the speech recognition model while improving the accuracy of the results obtained by natural language processing of the recognized text of the speech recognition model.

[0094] The sample set includes the first voice, the text corresponding to the first voice that is accurately recognized, and the annotated text corresponding to the first voice, such as silent voice, noisy voice, name, gender, object and other information annotations, and also includes the first voice segment and the text obtained by natural language processing of the first voice segment. The text also includes the second voice segment and the text obtained by natural language processing corresponding to the second voice segment. The annotation can be in the form of the correct text of the voice recognition and the correct result obtained by natural language processing, or it can be some characteristic information of user needs, or some other conditional factors, etc., and the present application is not limited to this.

[0095] Specifically, when executing step 230, the following method may also be used:

[0096] Determine the ratio of the first speech, the first speech segment, and the second speech segment in the sample set; determine a combined speech set based on the ratio of the first speech, the first speech segment, and the second speech segment in the sample set, and annotate the combined speech set to obtain a sample set.

[0097] In the above process, by using a certain proportion of speech and speech fragments and the annotation of each speech and each speech, the proportion and number of samples in the sample set can be accurately limited, and then the model can be optimized, which can maximize the recognition accuracy of the model and the accuracy of the results obtained by natural language processing of the recognition results.

[0098] The ratio of the first speech, the first speech segment and the second speech segment in the sample set is determined based on the total duration of all speech corresponding to each set in the first speech set, the first speech segment set and the second speech segment set, respectively. The combined speech set is obtained by selecting the corresponding number of first speech, first speech segment and second speech segment from the first speech set, the first speech segment set and the second speech segment set based on the ratio of the first speech, the first speech segment set and the second speech segment set.

[0099] Step 240: Optimize the speech recognition model using the sample set.

[0100] Furthermore, after executing step 240, the following method may also be used:

[0101] The optimized speech recognition model is tested using the speech in the test set to obtain test results; if the test results show that the recognition accuracy of the optimized speech recognition model for the speech in the test set is lower than a preset second accuracy threshold, the proportion of the first speech in the sample set is increased, and the optimized speech recognition model is optimized; if the test results show that the accuracy of the result obtained by the optimized speech recognition model performing natural language processing on the speech in the test set is lower than a preset third accuracy threshold, the proportion of the first speech segment and the second speech segment in the sample set is increased, and the optimized speech recognition model is optimized.

[0102] In the above process, the proportion of corresponding samples can be increased in a targeted manner according to the accuracy of model recognition and the accuracy of the results obtained by natural language processing, thereby improving the recognition accuracy of the model and the accuracy of the results obtained by natural language processing of the recognition results.

[0103] The second accurate threshold and the third accurate threshold can be set according to user needs, or can be obtained after multiple model optimizations.

[0104] For example: the total duration of the speech of the three sets is 30.12 seconds, 20.03 seconds and 9.99 seconds respectively. It can be determined that the ratio of the first speech, the first speech segment and the second speech segment in the sample set is approximately 3:2:1. Therefore, if the ratio interval is (0,1), the initial ratio can be determined as 1 / 2:1 / 3:1 / 6. If the accuracy of the speech recognition model in recognizing the speech is less than the preset second accuracy threshold, the ratio of the first speech in the sample set can be increased. For example, after the ratio is increased, the ratio of the speech in the three sets in the sample set is: 2 / 3, 1 / 4, 1 / 12. If the accuracy of the result after natural language processing of the recognized text is less than the preset third accuracy threshold, the ratio of the first speech segment and the second speech segment in the sample set can be increased. For example, after the ratio is increased, the ratio of the speech in the three sets in the sample set is: 1 / 6, 1 / 2, 1 / 3.

[0105] Previous article Figure 2 Describes the method of optimizing the speech recognition model. Figure 3 The method of speech recognition in the embodiment of the present application is described in detail.

[0106] Please see Figure 3 , Figure 3 A flowchart of a method for speech recognition provided in an embodiment of the present application is shown in FIG. Figure 3 The voice recognition method shown includes:

[0107] Step 310: Acquire speech to be recognized.

[0108] Step 320: Input the speech to be recognized into the optimized speech recognition model to obtain a recognition result.

[0109] The recognition of the speech to be recognized by the optimized speech recognition model can improve the recognition accuracy of the speech recognition model and the accuracy of the subsequent natural language processing results of the recognized text.

[0110] The optimized speech recognition model is obtained by optimizing the existing speech recognition model by selecting at least one first speech, at least one first speech segment and at least one second speech segment from the first speech set, the first speech segment set and the second speech segment set to form a combined speech set, and annotating the combined speech set to obtain a sample set, wherein the first speech set includes speech whose recognition accuracy of the speech recognition model is less than an accuracy threshold, and the first speech segment is obtained by intercepting the corresponding speech segment from the first speech set based on the result of natural language processing of a first text set corresponding to the first speech set, and the second speech set includes speech whose recognition accuracy of the speech recognition model is greater than or equal to the accuracy threshold, and the second speech segment is obtained by intercepting the corresponding speech segment from the second speech set based on the result of natural language processing corresponding to the second text set corresponding to the second speech set.

[0111] Previous article Figure 2 describes methods for optimizing speech recognition models, Figure 3 Describes the method of using the optimized speech recognition model for speech recognition. Figure 4-Figure 5 Describes an apparatus for optimizing a speech recognition model and Figure 6-Figure 7 Describe the apparatus for speech recognition.

[0112] Please refer to Figure 4 , is a schematic block diagram of a speech recognition model optimization device 400 provided in an embodiment of the present application, and the device 400 may be a module, program segment or code on an electronic device. Figure 2 The method embodiment corresponds to and can be executed Figure 2 The various steps involved in the method embodiment and the specific functions of the device 400 can be found in the description below. To avoid repetition, the detailed description is appropriately omitted here.

[0113] Optionally, the device 400 includes:

[0114] An acquisition module 410 is used to acquire a first speech set and a second speech set, wherein the first speech set includes speech whose recognition accuracy of the speech recognition model is less than an accuracy threshold, and the second speech set includes speech whose recognition accuracy of the speech recognition model is greater than or equal to the accuracy threshold;

[0115] The interception module 420 is used to intercept corresponding speech segments from the first speech set and intercept corresponding speech segments from the second speech set according to the result of natural language processing of the first text set corresponding to the first speech set and the second text set corresponding to the second speech set, so as to obtain the first speech segment set and the second speech segment set;

[0116] The combining module 430 is used to select at least one first voice, at least one first voice segment and at least one second voice segment from the first voice set, the first voice segment set and the second voice segment set to form a combined voice set, and annotate the combined voice set to obtain a sample set, wherein the sample set includes the first voice, the text corresponding to the first voice accurately recognized and the annotated text corresponding to the first voice, the first voice segment and the text obtained by natural language processing of the first voice segment accurately corresponding to the text, and the second voice segment and the text obtained by natural language processing of the second voice segment;

[0117] The optimization module 440 is used to optimize the speech recognition model using the sample set.

[0118] Optionally, the acquisition module is specifically used to:

[0119] Randomly extract a preset number of voices from the database, and use the voice recognition model to perform voice recognition on the preset number of voices to obtain recognition results, wherein the recognition results include the recognition accuracy rate of each voice;

[0120] Based on the recognition result, selecting speech with recognition accuracy less than an accuracy threshold from a preset number of speech to obtain a first speech set;

[0121] Based on the recognition result, voices with a recognition accuracy greater than or equal to an accuracy threshold are screened from a preset number of voices to obtain a second voice set.

[0122] Optionally, the interception module is specifically used to:

[0123] Performing natural language processing on a first text set corresponding to the first voice set and a second text set corresponding to the second voice set, respectively, to obtain a first text segment set and a second text segment set;

[0124] Based on the first text segment set, corresponding speech segments are intercepted from the first speech set, and based on the second text segment set, corresponding speech segments are intercepted from the second speech set to obtain the first speech segment set and the second text segment set.

[0125] Optionally, the combined module is specifically used for:

[0126] Determine the ratio of the first speech, the first speech segment, and the second speech segment in the sample set, wherein the ratio of the first speech, the first speech segment, and the second speech segment in the sample set is determined based on the total duration of all speech corresponding to each set of the first speech set, the first speech segment set, and the second speech segment set;

[0127] Based on the ratio of the first speech, the first speech fragment and the second speech fragment in the sample set, a combined speech set is determined, and the combined speech set is labeled to obtain a sample set, wherein the combined speech set is obtained by selecting a corresponding number of first speech, first speech fragment and second speech fragment from the first speech set, the first speech fragment set and the second speech fragment set based on the ratio of the first speech, the first speech fragment and the second speech fragment in the sample set.

[0128] Optionally, the device further comprises:

[0129] A test adjustment module, which is used for testing the optimized speech recognition model using speech in a test set after the optimization module optimizes the speech recognition model using a sample set, to obtain a test result;

[0130] If the test result shows that the recognition accuracy of the optimized speech recognition model for the speech in the test set is lower than the preset second accuracy threshold, increase the proportion of the first speech in the sample set and optimize the optimized speech recognition model;

[0131] If the test results show that the accuracy of the results obtained by the optimized speech recognition model for natural language processing of the speech in the test set is lower than the preset third accuracy threshold, increase the ratio of the first speech segment and the second speech segment in the sample set, and optimize the optimized speech recognition model.

[0132] Please refer to Figure 5 The structure of a device 500 for optimizing a speech recognition model provided in an embodiment of the present application is schematically shown. The device may include a memory 510 and a processor 520. Optionally, the device may also include: a communication interface 530 and a communication bus 540. The device is similar to the above Figure 2 The method embodiment corresponds to and can be executed Figure 2 The various steps involved in the method embodiment and the specific functions of the device can be found in the description below.

[0133] Specifically, the memory 510 is used to store computer-readable instructions.

[0134] Processor 520, configured to process readable instructions stored in the memory, capable of executing Figure 2 The various steps of method embodiments 210 to 240.

[0135] The communication interface 530 is used for signaling or data communication with other node devices, for example, for communication with a server or a terminal, or for communication with other device nodes, but the embodiments of the present application are not limited thereto.

[0136] The communication bus 540 is used to realize direct connection and communication among the above components.

[0137] The communication interface 530 of the device in the embodiment of the present application is used to communicate signals or data with other node devices. The memory 510 can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The memory 510 can also be at least one storage device located away from the aforementioned processor. The memory 510 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 520, the electronic device executes the aforementioned Figure 2 The method process shown. The processor 520 can be used on the device 400 and used to perform the functions in the present application. Exemplarily, the above-mentioned processor 520 can be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and the embodiments of the present application are not limited thereto.

[0138] Please refer to Figure 6 , is a schematic block diagram of a speech recognition device 600 provided in an embodiment of the present application, and the device 600 may be a module, program segment or code on an electronic device. Figure 3 The method embodiment corresponds to and can be executed Figure 3 The various steps involved in the method embodiment and the specific functions of the device 600 can be found in the description below. To avoid repetition, the detailed description is appropriately omitted here.

[0139] Optionally, the device 600 includes:

[0140] An acquisition module 610 is used to acquire speech to be recognized;

[0141] The recognition module 620 is used to input the speech to be recognized into the optimized speech recognition model to obtain a recognition result, wherein the optimized speech recognition model is obtained by selecting at least one first speech, at least one first speech segment and at least one second speech segment from a first speech set, a first speech segment set and a second speech segment set to form a combined speech set, and annotating the combined speech set to obtain a sample set optimized by the existing speech recognition model, the first speech set includes speech whose recognition accuracy of the speech recognition model is less than an accuracy threshold, the first speech segment is obtained by intercepting the corresponding speech segment from the first speech set based on the result of natural language processing of a first text set corresponding to the first speech set, the second speech set includes speech whose recognition accuracy of the speech recognition model is greater than or equal to the accuracy threshold, and the second speech segment is obtained by intercepting the corresponding speech segment from the second speech set based on the result of natural language processing corresponding to the second text set corresponding to the second speech set.

[0142] Please refer to Figure 7 700 is a schematic block diagram of a speech recognition device 700 provided in an embodiment of the present application. The device may include a memory 710 and a processor 720. Optionally, the device may also include: a communication interface 730 and a communication bus 740. The device is similar to the above Figure 3 The method embodiment corresponds to and can be executed Figure 3 The various steps involved in the method embodiment and the specific functions of the device can be found in the description below.

[0143] Specifically, the memory 710 is used to store computer-readable instructions.

[0144] Processor 720 is used to process the readable instructions stored in the memory and execute Figure 3 The various steps of method embodiments 310 to 320.

[0145] The communication interface 730 is used for signaling or data communication with other node devices, for example, for communication with a server or a terminal, or for communication with other device nodes, but the embodiments of the present application are not limited thereto.

[0146] The communication bus 740 is used to realize direct connection and communication among the above components.

[0147] The communication interface 730 of the device in the embodiment of the present application is used to communicate signals or data with other node devices. The memory 710 can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The memory 710 can also be at least one storage device located away from the aforementioned processor. The memory 710 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 720, the electronic device executes the aforementioned Figure 3 The method process shown. The processor 720 can be used on the device 600 and used to perform the functions in the present application. Exemplarily, the above-mentioned processor 720 can be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and the embodiments of the present application are not limited thereto.

[0148] The embodiment of the present application also provides a readable storage medium, when the computer program is executed by a processor, Figure 2 or Figure 3 The method process in the method embodiment shown is executed by the electronic device.

[0149] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method, and will not be described in detail here.

[0150] In summary, the embodiments of the present application provide a method, device, electronic device and storage medium for optimizing a speech recognition model, the method comprising: obtaining a first speech set and a second speech set; based on the results of natural language processing of a first text set corresponding to the first speech set and a second text set corresponding to the second speech set, intercepting corresponding speech segments from the first speech set and intercepting corresponding speech segments from the second speech set to obtain a first speech segment set and a second speech segment set; selecting at least one first speech, at least one first speech segment and at least one second speech segment from the first speech set, the first speech segment set and the second speech segment set to form a combined speech set, and annotating the combined speech set to obtain a sample set; optimizing the speech recognition model using the sample set, and the method of the present application can achieve the effect of improving the accuracy of speech recognition and the accuracy of the results obtained by natural language processing of the speech recognition results.

[0151] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0152] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0153] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0154] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0155] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0156] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A method for optimizing a speech recognition model, characterized in that: include: Acquire a first speech set and a second speech set, wherein the first speech set includes speech whose recognition accuracy of the speech recognition model is less than an accuracy threshold, and the second speech set includes speech whose recognition accuracy of the speech recognition model is greater than or equal to the accuracy threshold; According to the result of natural language processing of a first text set corresponding to the first speech set and a second text set corresponding to the second speech set, corresponding speech segments are intercepted from the first speech set, and corresponding speech segments are intercepted from the second speech set to obtain a first speech segment set and a second speech segment set; Selecting at least one first voice, at least one first voice segment and at least one second voice segment from the first voice set, the first voice segment set and the second voice segment set to form a combined voice set, and annotating the combined voice set to obtain a sample set, wherein the sample set includes the first voice, the text corresponding to the first voice accurately recognized and the annotated text corresponding to the first voice, the first voice segment and the text obtained by natural language processing of the first voice segment accurately corresponding to the text, and the second voice segment and the text obtained by natural language processing corresponding to the second voice segment; The speech recognition model is optimized using the sample set.

2. The method according to claim 1, characterized in that The obtaining of the first speech set and the second speech set comprises: Randomly extract a preset number of voices from the database, and use the voice recognition model to perform voice recognition on the preset number of voices to obtain recognition results, wherein the recognition results include the recognition accuracy rate of each voice; Based on the recognition result, filtering speech whose recognition accuracy is less than an accuracy threshold from the preset number of speech to obtain the first speech set; Based on the recognition result, voices having a recognition accuracy greater than or equal to the accuracy threshold are screened from the preset number of voices to obtain the second voice set.

3. The method according to claim 1 or 2, characterized in that: The method of extracting corresponding speech segments from the first speech set and extracting corresponding speech segments from the second speech set according to the result of natural language processing of the first text set corresponding to the first speech set and the second text set corresponding to the second speech set, to obtain the first speech segment set and the second speech segment set, comprises: Performing natural language processing on a first text set corresponding to the first speech set and a second text set corresponding to the second speech set, respectively, to obtain a first text segment set and a second text segment set; Based on the first text segment set, corresponding speech segments are intercepted from the first speech set, and based on the second text segment set, corresponding speech segments are intercepted from the second speech set to obtain the first speech segment set and the second speech segment set.

4. The method according to claim 1 or 2, characterized in that: The method further comprises: selecting at least one first speech, at least one first speech segment, and at least one second speech segment from the first speech set, the first speech segment set, and the second speech segment set to form a combined speech set, and annotating the combined speech set to obtain a sample set, including: Determine the ratio of the first speech, the first speech segment, and the second speech segment in the sample set, wherein the ratio of the first speech, the first speech segment, and the second speech segment in the sample set is determined based on the total duration of all speech corresponding to each set of the first speech set, the first speech segment set, and the second speech segment set; Based on the ratio of the first speech, the first speech segment and the second speech segment in the sample set, the combined speech set is determined, and the combined speech set is labeled to obtain a sample set, wherein the combined speech set is obtained by selecting a corresponding number of the first speech, the first speech segment and the second speech segment from the first speech set, the first speech segment set and the second speech segment set based on the ratio of the first speech, the first speech segment and the second speech segment in the sample set.

5. The method according to claim 1 or 2, characterized in that: After optimizing the speech recognition model by using the sample set, the method further includes: Use the speech in the test set to test the optimized speech recognition model and obtain the test results; If the test result shows that the recognition accuracy of the optimized speech recognition model for the speech in the test set is lower than a preset second accuracy threshold, increasing the proportion of the first speech in the sample set, and optimizing the optimized speech recognition model; If the test results show that the accuracy of the results obtained by the optimized speech recognition model through natural language processing of the speech in the test set is lower than a preset third accuracy threshold, increase the proportion of the first speech segment and the second speech segment in the sample set, and optimize the optimized speech recognition model.

6. A method for speech recognition, characterized in that: include: Obtain the speech to be recognized; The speech to be recognized is input into an optimized speech recognition model to obtain a recognition result, wherein the optimized speech recognition model is obtained by optimizing the existing speech recognition model by selecting at least one first speech, at least one first speech segment and at least one second speech segment from a first speech set, a first speech segment set and a second speech segment set to form a combined speech set, and annotating the combined speech set to obtain a sample set, wherein the first speech set includes speech whose recognition accuracy rate of the speech recognition model is less than an accuracy threshold, the first speech segment is obtained by intercepting the corresponding speech segment from the first speech set based on the result of natural language processing of a first text set corresponding to the first speech set, the second speech set includes speech whose recognition accuracy rate of the speech recognition model is greater than or equal to the accuracy threshold, and the second speech segment is obtained by intercepting the corresponding speech segment from the second speech set based on the result of natural language processing corresponding to a second text set corresponding to the second speech set.

7. A device for optimizing a speech recognition model, characterized in that: include: An acquisition module, configured to acquire a first speech set and a second speech set, wherein the first speech set includes speech whose recognition accuracy of a speech recognition model is less than an accuracy threshold, and the second speech set includes speech whose recognition accuracy of the speech recognition model is greater than or equal to the accuracy threshold; an interception module, configured to intercept corresponding speech segments from the first speech set and intercept corresponding speech segments from the second speech set according to a result of natural language processing of a first text set corresponding to the first speech set and a second text set corresponding to the second speech set, so as to obtain a first speech segment set and a second speech segment set; a combining module, configured to select at least one first voice, at least one first voice segment and at least one second voice segment from the first voice set, the first voice segment set and the second voice segment set to form a combined voice set, and annotate the combined voice set to obtain a sample set, wherein the sample set includes the first voice, the text corresponding to the first voice accurately recognized and the annotated text corresponding to the first voice, the first voice segment and the text obtained by natural language processing of the first voice segment accurately corresponding to the text, and the second voice segment and the text obtained by natural language processing of the second voice segment corresponding to the text; An optimization module is used to optimize the speech recognition model using the sample set.

8. A speech recognition device, characterized in that: include: An acquisition module, used to acquire the speech to be recognized; A recognition module is used to input the speech to be recognized into an optimized speech recognition model to obtain a recognition result, wherein the optimized speech recognition model is obtained by optimizing the existing speech recognition model by selecting at least one first speech, at least one first speech segment and at least one second speech segment from a first speech set, a first speech segment set and a second speech segment set to form a combined speech set, and annotating the combined speech set to obtain a sample set. The first speech set includes speech whose recognition accuracy of the speech recognition model is less than an accuracy threshold, the first speech segment is obtained by intercepting the corresponding speech segment from the first speech set based on the result of natural language processing of a first text set corresponding to the first speech set, the second speech set includes speech whose recognition accuracy of the speech recognition model is greater than or equal to the accuracy threshold, and the second speech segment is obtained by intercepting the corresponding speech segment from the second speech set based on the result of natural language processing corresponding to a second text set corresponding to the second speech set.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method according to any one of claims 1 to 5 or 6 are executed.

10. A computer-readable storage medium, characterized in that: include: A computer program, when the computer program is run on a computer, causes the computer to execute the method according to any one of claims 1 to 5 or 6.

Images