Domain-adaptive speech recognition method and device, computer device and storage medium

Abstract

The application discloses a domain self-adaptive speech recognition method and device, computer equipment and a storage medium, and the method comprises the steps of obtaining to-be-recognized speech data; performing primary decoding on the to-be-recognized speech data, obtaining a plurality of candidate recognition results, and obtaining an optimal recognition result from the plurality of candidate recognition results; performing domain judgment according to the optimal recognition result, and determining a target domain; and performing secondary decoding on the plurality of candidate recognition results by using a target language model corresponding to the target domain, and obtaining a target recognition result. The method can adapt to the changeable speaking domain of a user, improve the accuracy during multi-domain and cross-domain recognition, improve the robustness of speech recognition, and provide better user experience.

Description

Technical Field

[0001] This invention relates to the field of speech recognition technology, and in particular to a domain-adaptive speech recognition method, apparatus, computer device, and storage medium. Background Technology

[0002] Speech recognition technology has been widely applied in many scenarios and has achieved good results. However, due to the complexity of human language and speech signals themselves, speech recognition systems cannot guarantee high accuracy in all scenarios, depending on the performance of current models. Existing speech recognition models are generally more accurate for speech in specific domains. When considering multiple domains, or even when not limiting the speech domain, their accuracy drops significantly. For example, a speech recognition system in the financial field has high accuracy for speech related to finance, but its accuracy drops drastically for speech related to music, games, or other domains. In real-world applications, the content spoken by users often falls into a very broad range of domains, and users may speak about different domains at different times. Ordinary speech recognition systems struggle to handle such complex user needs, impacting the user experience. Summary of the Invention

[0003] This invention provides a domain-adaptive speech recognition method, apparatus, computer device, and storage medium to address the issue of accuracy in multi-domain or cross-domain speech recognition.

[0004] A domain-adaptive speech recognition method includes:

[0005] Acquire the speech data to be recognized;

[0006] The speech data to be recognized is decoded once to obtain multiple candidate recognition results, and the optimal recognition result is obtained from the multiple candidate recognition results.

[0007] Based on the optimal recognition result, a domain determination is performed to identify the target domain;

[0008] Using the target language model corresponding to the target domain, the multiple candidate recognition results are decoded a second time to obtain the target recognition result.

[0009] A domain-adaptive speech recognition device, comprising:

[0010] The speech data acquisition module is used to acquire the speech data to be recognized;

[0011] A single decoding result acquisition module is used to perform a single decoding on the speech data to be recognized, obtain multiple candidate recognition results, and obtain the optimal recognition result from the multiple candidate recognition results;

[0012] The target domain determination module is used to determine the target domain by making a domain judgment based on the optimal recognition result.

[0013] The target recognition result acquisition module uses the target language model corresponding to the target domain to perform secondary decoding on multiple candidate recognition results to obtain the target recognition result.

[0014] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the aforementioned domain-adaptive speech recognition method.

[0015] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned domain-adaptive speech recognition method.

[0016] The aforementioned adaptive speech recognition methods, devices, computer equipment, and storage media employ a target language model corresponding to the target domain to perform secondary decoding on multiple candidate recognition results, thereby making the obtained target recognition results more accurate and more in line with reality. In particular, based on the optimal recognition result of the speech data to be recognized, the target domain of the speech data to be recognized is automatically determined to adapt to the user's changing speaking domain, improve the accuracy of multi-domain and cross-domain speech recognition, enhance the robustness of speech recognition, and provide a better user experience. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of an application environment for a domain-adaptive speech recognition method according to an embodiment of the present invention;

[0019] Figure 2 This is a flowchart of a domain-adaptive speech recognition method according to an embodiment of the present invention;

[0020] Figure 3 This is another flowchart of a domain-adaptive speech recognition method in one embodiment of the present invention;

[0021] Figure 4 This is another flowchart of a domain-adaptive speech recognition method in one embodiment of the present invention;

[0022] Figure 5This is another flowchart of a domain-adaptive speech recognition method in one embodiment of the present invention;

[0023] Figure 6 This is another flowchart of a domain-adaptive speech recognition method in one embodiment of the present invention;

[0024] Figure 7 This is another flowchart of a domain-adaptive speech recognition method in one embodiment of the present invention;

[0025] Figure 8 This is another flowchart of a domain-adaptive speech recognition method in one embodiment of the present invention;

[0026] Figure 9 This is a schematic diagram of a domain-adaptive speech recognition device according to an embodiment of the present invention;

[0027] Figure 10 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] The domain-adaptive speech recognition method provided in this invention can be applied to, for example... Figure 1 The application environment is shown. Specifically, this domain-adaptive speech recognition method is applied in a domain-adaptive speech recognition system, which includes, for example, [examples of such systems]. Figure 1 The diagram illustrates a client and server that communicate over a network to implement domain-adaptive speech recognition. The client, also known as the user terminal, is the program that provides local services to the client, corresponding to the server. The client can be installed on, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0030] In one embodiment, such as Figure 2 As shown, a domain-adaptive speech recognition method is provided, which can be applied to... Figure 1 Taking the server in the example, the following steps are included:

[0031] S201: Acquire the speech data to be recognized;

[0032] S202: Decode the speech data to be recognized once to obtain multiple candidate recognition results, and obtain the optimal recognition result from the multiple candidate recognition results;

[0033] S203: Based on the optimal recognition result, determine the target domain;

[0034] S204: Using the target language model corresponding to the target domain, perform secondary decoding on multiple candidate recognition results to obtain the target recognition result.

[0035] The speech data to be recognized refers to the speech data that needs to be recognized. This speech data can be speech data from multiple fields or cross-field speech data, and its content can cover various fields.

[0036] As an example, in step S201, the server can acquire the speech data to be recognized. This speech data can be multi-domain speech data or cross-domain speech data. In this example, the server acquires the speech data to be recognized from various domains as input data for a subsequent decoding, ensuring the feasibility of the subsequent decoding.

[0037] In this context, "one-time decoding" refers to the process of processing the speech data to be recognized to obtain multiple speech recognition results. Candidate recognition results refer to the multiple recognition results with higher probabilities obtained after one-time decoding of the speech data; specifically, they can be understood as multiple recognition paths or multiple original recognition sentences with higher recognition probabilities. The optimal recognition result is the one with the best recognition performance among the multiple candidate recognition results.

[0038] As an example, in step S202, the server performs a decoding process on the acquired speech data to be recognized, obtaining multiple candidate recognition results, and then selects the optimal recognition result from the multiple candidate recognition results. In this example, when decoding the speech data to be recognized, the acoustic model is first used to process the speech data to be recognized, and the acoustic model processing result is obtained; then, a general language model is used to process the speech data to be recognized, and the general language model processing result is obtained; multiple recognition results with better recognition performance from the acoustic model processing result and the general language model processing result are selected as multiple candidate recognition results; and the recognition result with the best recognition performance from the multiple candidate recognition results is selected as the optimal recognition result.

[0039] In this example, both an acoustic model and a general language model are used to process the speech data to be recognized, resulting in multiple recognition results. Several of the better-performing results are selected as candidate results, ensuring that the input data for the subsequent secondary decoding process is more realistic and the secondary decoding results are more accurate. From these candidate results, the best one is selected as the optimal result, which is then used for subsequent domain identification. This ensures the accuracy of the domain identification, guarantees a more precise target domain determination, and ensures that the target language model used in the secondary decoding process is targeted, thus contributing to the accuracy of the final target recognition result.

[0040] As an example, in step S203, the server performs domain determination on the speech data to be recognized based on the optimal recognition result to obtain the target domain of the speech data to be recognized. In this example, the server uses a domain determination model to determine the domain of the input optimal recognition result as follows: the optimal recognition result is input into the domain determination model, the probability value of each domain corresponding to the optimal recognition result is output, and the domain with the highest probability value is selected as the target domain.

[0041] In this example, a domain judgment model is used to determine the target domain based on the optimal recognition result, which not only provides feasibility for secondary decoding but also ensures the accuracy of secondary decoding.

[0042] The target language model refers to the language model corresponding to the target domain. It can be understood as a language model obtained by training with training data corresponding to the target domain. This language model has high recognition accuracy for input data in the target domain. Secondary decoding refers to the process of recognizing multiple candidate recognition results to obtain the target recognition result.

[0043] As an example, in step S204, the server uses the target language model corresponding to the target domain to perform secondary decoding on multiple candidate recognition results to obtain the target recognition result. The specific process is as follows: based on obtaining the target domain, the server obtains the target language model corresponding to the target domain, uses the target language model to recognize multiple candidate recognition results, obtains multiple recognition results and a score corresponding to each recognition result, and selects the recognition result with the highest score as the target recognition result.

[0044] In this example, the target language model corresponding to the target domain is used to identify and score multiple candidate recognition results to obtain the target recognition result. Since the target language model is more accurate in recognizing input data in the target domain, using the target language model to identify multiple candidate recognition results can make the target recognition result more accurate and more in line with reality.

[0045] The domain-adaptive speech recognition method provided in this embodiment first decodes the speech data to be recognized once, obtaining multiple candidate recognition results and the optimal recognition result. This makes subsequent domain determination of the speech data feasible and ensures the accuracy of the secondary decoding process, making the target recognition result more realistic. Based on the optimal recognition result, domain determination is performed to identify the target domain. Then, the target language model corresponding to the target domain is used to perform secondary decoding on the multiple candidate recognition results, making the obtained target recognition result more accurate and realistic. In other words, based on the optimal recognition result of the speech data to be recognized, the target domain of the speech is automatically determined to adapt to the user's changing speaking domain, improving the accuracy of multi-domain and cross-domain speech recognition, enhancing the robustness, accuracy, and stability of speech recognition, and providing a better user experience.

[0046] In one embodiment, such as Figure 3 As shown, step S202, which involves decoding the speech data to be recognized once to obtain multiple candidate recognition results, and then selecting the optimal recognition result from the multiple candidate recognition results, includes:

[0047] S301: An acoustic model is used to decode the speech data to be recognized, and multiple original recognition sentences and the target acoustic score corresponding to each original recognition sentence are obtained.

[0048] S302: Use a general language model to decode the speech data to be recognized, and obtain multiple original recognition sentences and the first language score corresponding to each original recognition sentence;

[0049] S303: Process the target acoustic score and first language score corresponding to multiple original recognized sentences to obtain the first recognition score corresponding to multiple original recognized sentences;

[0050] S304: Sort the first recognition scores corresponding to multiple original recognition sentences, and determine the top N original recognition sentences with the largest first recognition scores as multiple candidate recognition results, where N≥2;

[0051] S305: Select the original recognition sentence with the highest first recognition score from multiple candidate recognition results and determine it as the optimal recognition result.

[0052] As an example, in step S301, the server uses an acoustic model to decode the speech data to be recognized, obtaining multiple original recognized sentences. Simultaneously, it obtains the score corresponding to each original recognized sentence, which serves as the target acoustic score for each original recognized sentence. Here, the target acoustic score can be understood as the score of each original recognized sentence output by the acoustic model.

[0053] In this example, the server uses an acoustic model to decode the speech data to be recognized, obtaining multiple original recognition sentences and a target acoustic score for each original recognition sentence. For example, the acoustic model recognizes a certain speech data to be recognized, obtaining X original recognition sentences, where X≥2, and each original recognition sentence outputs a corresponding target acoustic score.

[0054] As an example, in step S302, the server uses a general language model to decode the speech data to be recognized, obtaining multiple original recognition sentences. At the same time, it obtains the score corresponding to each original recognition sentence, which is used as the first language score corresponding to each original recognition sentence.

[0055] In one embodiment, the server uses a general language model to decode the speech data to be recognized, obtaining multiple original recognition sentences and a first language score corresponding to each original recognition sentence. For example, the general language model recognizes Y original recognition sentences from a certain speech data to be recognized, where Y ≥ 2, and each original recognition sentence outputs a corresponding first language score. Here, the first language score can be understood as the score of each original recognition sentence output by the general language model.

[0056] As an example, in step S303, the server performs fusion processing on the target acoustic scores and first language scores corresponding to multiple original recognition sentences to obtain the scores corresponding to multiple original recognition sentences, which are used as the first recognition scores corresponding to the original recognition sentences.

[0057] In one embodiment, the server performs fusion processing on the target acoustic scores and first language scores corresponding to multiple original recognition sentences to obtain the corresponding first recognition score. For example, an acoustic model recognizes a certain speech data to be recognized, obtaining X original recognition sentences and corresponding target acoustic scores, where X≥2; a general language model recognizes the same speech data to be recognized, obtaining Y original recognition sentences and corresponding first language scores, where Y≥2; among the X original recognition sentences, an acoustic weight w1 is used to weight the target acoustic score corresponding to each original recognition sentence, and among the Y original recognition sentences, a language weight w2 is used to weight the first language score corresponding to each original recognition sentence. The above two weighted sums are then obtained to obtain the first recognition score corresponding to each original recognition sentence, and finally, multiple original recognition sentences and their corresponding first recognition scores are obtained.

[0058] As an example, in step S304, the server sorts the first recognition scores corresponding to multiple original recognition sentences and determines the top N original recognition sentences with larger first recognition scores as multiple candidate recognition results, where N≥2.

[0059] In one embodiment, the server selects the top N original recognized sentences with the largest first recognition scores as multiple candidate recognition results, where N ≥ 2. For example, based on obtaining the first recognition scores corresponding to multiple original recognized sentences, the server selects the top N original recognized sentences with the largest first recognition scores as multiple candidate recognition results.

[0060] As an example, in step S305, the server selects the original recognition sentence with the largest first recognition score from multiple candidate recognition results and determines it as the optimal recognition result. Specifically, the original recognition sentence with the largest first recognition score from multiple candidate recognition results is determined as the optimal recognition result.

[0061] In one embodiment, the server selects the original recognized sentence with the highest first recognition score as the optimal recognition result. For example, if there are N candidate recognition results, each containing the first recognition score of an original recognized sentence, the server can select the original recognized sentence with the highest first recognition score from among the N candidate recognition results as the optimal recognition result.

[0062] In the domain-adaptive speech recognition method provided in this embodiment, an acoustic model and a general language model are used to decode the speech data to be recognized, respectively, to obtain multiple original recognition sentences. The acoustic model can identify the user's accent in the speech data to be recognized, reducing the impact of the user's accent on the first decoding result. The original recognition sentences of the acoustic model and each original recognition sentence of the general language model are scored separately, and the scores are fused to obtain a first recognition score for each original recognition sentence. The first recognition scores are then sorted, and the top N original recognition sentences with the largest first recognition scores are selected as multiple candidate recognition results. This ensures that the input data for the subsequent secondary decoding process is more realistic, making the secondary decoding result more accurate. From the multiple candidate recognition results, the original recognition sentence with the largest first recognition score is selected as the optimal recognition result, ensuring the accuracy of the subsequent domain judgment. This ensures that the target domain judgment is more accurate, and at the same time, it ensures that the target language model used in the secondary decoding process is more accurate.

[0063] In one embodiment, such as Figure 4 As shown, step S204, which involves using the target language model corresponding to the target domain to perform secondary decoding on multiple candidate recognition results to obtain the target recognition result, includes:

[0064] S401: Using the target language model corresponding to the target domain, multiple candidate recognition results are decoded twice to obtain multiple candidate recognition sentences and the second language score corresponding to each candidate recognition sentence;

[0065] S402: Process the second language score and target acoustic score corresponding to each candidate recognition sentence to obtain the second recognition score corresponding to each candidate recognition sentence;

[0066] S403: Sort multiple candidate recognition sentences according to the second recognition score, and determine the candidate recognition sentence with the largest second recognition score as the target recognition result.

[0067] The candidate recognition results include candidate recognition sentences and the target acoustic scores corresponding to the candidate recognition sentences.

[0068] As an example, in step S401, the server confirms the language model corresponding to the target domain as the target language model, and uses the target language model to perform secondary decoding and recognition on multiple candidate recognition results to obtain multiple candidate recognition sentences and the score corresponding to each candidate recognition sentence, which is used as the second language score of each candidate recognition sentence.

[0069] In one embodiment, the server uses a target language model to perform secondary decoding and recognition on multiple candidate recognition results, obtaining multiple candidate recognition sentences and a second language score corresponding to each candidate recognition sentence. For example, based on selecting multiple candidate recognition results of number N, the target language model is used to perform secondary decoding and recognition on the multiple candidate recognition results, obtaining N candidate recognition sentences and a second language score corresponding to each candidate recognition sentence.

[0070] As an example, in step S402, the server fuses the second language score and the target acoustic score corresponding to each candidate recognition sentence to obtain the second recognition score corresponding to each candidate recognition sentence.

[0071] In one embodiment, the server fuses the second language score and the target acoustic score corresponding to each candidate recognition sentence to obtain a second recognition score for each candidate recognition sentence. In this example, the second language scores and target acoustic scores corresponding to N candidate recognition sentences are fused to obtain the second recognition scores for the N candidate recognition sentences. For example, for the candidate recognition sentences corresponding to the N candidate recognition results, the target acoustic score corresponding to each candidate recognition sentence is weighted using acoustic weight w1, and the second language score corresponding to each candidate recognition sentence is weighted using language weight w2. The two weighted values ​​are then summed to obtain the second recognition score for each candidate recognition sentence, and finally, the second recognition scores for the N candidate recognition sentences are obtained.

[0072] As an example, in step S403, the server sorts multiple candidate recognition sentences according to the second recognition score, and determines the candidate recognition sentence with the highest second recognition score as the target recognition result. In one embodiment, the candidate recognition sentence with the highest second recognition score is selected as the target recognition result of the speech data to be recognized. For example, based on obtaining the second recognition scores corresponding to N candidate recognition sentences, the candidate recognition sentence with the highest second recognition score among the N candidate recognition sentences is selected as the target recognition result.

[0073] In the domain-adaptive speech recognition method provided in this embodiment, a target language model corresponding to the target domain is used to recognize multiple candidate recognition results, ensuring the recognition accuracy of the multiple candidate recognition sentences and the second language score. The second language score corresponding to each candidate recognition sentence is fused with the target acoustic score to obtain a second recognition score, and the candidate recognition sentence corresponding to the largest second recognition score is selected as the target recognition result corresponding to the speech data to be recognized, making the recognition result of the obtained speech data to be recognized more accurate and more in line with reality.

[0074] In one embodiment, such as Figure 5 As shown, step S402, which involves processing the second language score and target acoustic score corresponding to each candidate recognition sentence to obtain the second recognition score corresponding to each candidate recognition sentence, includes:

[0075] S501: Determine the target language score for each candidate recognition sentence based on the first language score and the second language score corresponding to each candidate recognition sentence;

[0076] S502: Process the target language score and the target acoustic score corresponding to each candidate recognition sentence to obtain the second recognition score corresponding to each candidate recognition sentence.

[0077] The candidate recognition results also include the first language score corresponding to the candidate recognized sentence.

[0078] As an example, in step S501, after the server performs secondary decoding on multiple candidate recognition results using the target language model corresponding to the target domain to obtain the second language scores corresponding to multiple candidate recognition sentences, it can fuse the second language scores corresponding to multiple candidate recognition sentences with the first language scores corresponding to multiple candidate recognition sentences obtained in step S302 using the general language model to obtain the target language score corresponding to each candidate recognition sentence. Understandably, since the target language score is a fusion of the first language score output by the general language model and the second language score output by the target language model, fusing the recognition results of the two language models makes the target language score more accurate compared to the recognition results of a single language model (i.e., either the general language model or the target language model).

[0079] As an example, in step S502, the server fuses the target language score and the target acoustic score corresponding to each candidate recognition sentence to obtain a second recognition score for each candidate recognition sentence. For instance, for the candidate recognition sentences corresponding to N candidate recognition results, the target acoustic score corresponding to each candidate recognition sentence is weighted using acoustic weight w1, and the target language score corresponding to each candidate recognition sentence is weighted using language weight w2. The two weighted values ​​are then summed to obtain the second recognition score for each candidate recognition sentence, thus obtaining the second recognition scores for the N candidate recognition sentences. Understandably, the acoustic model can identify the user's accent in the speech data to be recognized, obtain the target acoustic score based on the candidate recognition sentences of the acoustic model, and fuse the target acoustic score with the target language score to obtain the second recognition score. Since in step S403, the candidate recognition sentence with the largest second recognition score is selected as the target recognition result, the target acoustic score is taken into account when obtaining the second recognition score, which can reduce the impact of the user's accent in the speech data to be recognized on the accuracy of the target recognition result.

[0080] In the domain-adaptive speech recognition method provided in this embodiment, a second recognition score is obtained for each candidate recognition sentence, making it feasible to obtain the target recognition result.

[0081] In one embodiment, such as Figure 6 As shown, step S501, which involves determining the target language score for each candidate recognition sentence based on the first language score and the second language score, includes:

[0082] S601: Obtain the first fusion weight corresponding to the first language score and the second fusion weight corresponding to the second language score;

[0083] S602: Obtain the target language score corresponding to each candidate recognition sentence based on the first language score, the first fusion weight, the second language score, and the second fusion weight.

[0084] As an example, in step S601, the server, based on obtaining the first language score and the second language score corresponding to each candidate recognition sentence, respectively obtains the first fusion weight corresponding to the first language score and the second fusion weight corresponding to the second recognition score. Since the first language score is based on the score recognized by the general language model, while the second language score is based on the score recognized by the target language model corresponding to the target domain, generally, after determining the target domain based on the optimal recognition result, the recognition result of the target language model will be higher than the recognition result of the general language model. Therefore, the second fusion weight can be set to be greater than the first fusion weight so that the second language score output by the target language model has a higher weight. For example, the first fusion weight can be set to 0.3, and the second fusion weight can be set to 0.7.

[0085] As an example, in step S602, the server fuses the first language score and the second language score according to the first fusion weight and the second fusion weight respectively to obtain the target language score corresponding to each candidate recognition sentence.

[0086] In one embodiment, for each candidate recognition sentence, the weighted sum of the first fusion weight on the first language score and the weighted sum of the second fusion weight on the second language score are fused to obtain the target language score corresponding to each candidate recognition sentence. For example, after obtaining the first language score, first fusion weight, second language score, and second fusion weight corresponding to N candidate recognition sentences, the first fusion weight is used to weight the first language score corresponding to each candidate recognition sentence, and the second fusion weight is used to weight the second language score corresponding to each candidate recognition sentence. The sum of the above two weighted sums is then obtained to obtain the target language score corresponding to each candidate recognition sentence, ultimately yielding the target language scores corresponding to the N candidate recognition sentences.

[0087] In the domain-adaptive speech recognition method provided in this embodiment, the target language score corresponding to each candidate recognition sentence is obtained, making it feasible to obtain the second recognition score.

[0088] In another embodiment, such as Figure 7 As shown, step S204, which involves using the target language model corresponding to the target domain to perform secondary decoding on multiple candidate recognition results to obtain the target recognition result, includes:

[0089] S701: Using the target language model corresponding to the target domain, multiple candidate recognition results are decoded twice to obtain multiple candidate recognition sentences and the second language score corresponding to each candidate recognition sentence;

[0090] S702: Process the second language score and the first recognition score corresponding to each candidate recognition sentence to obtain the third recognition score corresponding to each candidate recognition sentence;

[0091] S703: Sort multiple candidate recognition sentences according to the third recognition score, and determine the candidate recognition sentence with the highest third recognition score as the target recognition result.

[0092] The candidate recognition results include the candidate recognition sentence and the first recognition score corresponding to the candidate recognition sentence.

[0093] As an example, in step S701, the server uses the target language model corresponding to the target domain to perform secondary decoding on multiple candidate recognition results, and obtains multiple candidate recognition sentences and the second language score corresponding to each candidate recognition sentence.

[0094] In one embodiment, the server selects N candidate recognition results and uses a target language model to perform secondary decoding and recognition on the multiple candidate recognition results to obtain N candidate recognition sentences and a second language score corresponding to each candidate recognition sentence.

[0095] As an example, in step S702, the server fuses the second language score and the first recognition score corresponding to each candidate recognition sentence to obtain the third recognition score corresponding to each candidate recognition sentence. Understandably, when obtaining the third recognition score, considering fusing the target acoustic score and the first language score after one decoding step to obtain the first recognition score, and then combining the obtained first recognition score with the second language score obtained from a second decoding step, a more accurate third recognition score can be obtained. This makes the subsequent target recognition results obtained based on the third recognition score more consistent with the actual situation of the speech data to be recognized.

[0096] In one embodiment, the server fuses the second language scores and the first recognition scores corresponding to N candidate recognition sentences to obtain a third recognition score corresponding to the N candidate recognition sentences. For example, for the candidate recognition sentences corresponding to the N candidate recognition results, the first recognition score corresponding to each candidate recognition sentence is weighted using a preset weight w3, and the second language score corresponding to each candidate recognition sentence is weighted using a preset weight w4. The above two weighted sums are then used to obtain the third recognition score corresponding to each candidate recognition sentence, and finally, the third recognition scores corresponding to the N candidate recognition sentences are obtained. Understandably, fusing the first recognition score and the second language score to obtain the third recognition score simultaneously evaluates the recognition performance of the acoustic model, the general language model, and the target language model, making the obtained third recognition score more accurate.

[0097] As an example, in step S703, the server sorts multiple candidate recognition sentences according to the third recognition score, and determines the candidate recognition sentence with the highest third recognition score as the target recognition result.

[0098] In one embodiment, the candidate recognition sentence with the highest third recognition score is selected as the target recognition result of the speech data to be recognized. Based on obtaining the third recognition scores corresponding to N candidate recognition sentences, the candidate recognition sentence with the highest third recognition score among the N candidate recognition sentences is selected as the target recognition result.

[0099] In the domain-adaptive speech recognition method provided in this embodiment, a target language model corresponding to the target domain is used to recognize multiple candidate recognition results, ensuring the recognition accuracy of the multiple candidate recognition sentences and the second language score. The second language score corresponding to each candidate recognition sentence is fused with the first recognition score to obtain a third recognition score, and the candidate recognition sentence corresponding to the largest third recognition score is selected as the target recognition result corresponding to the speech data to be recognized, making the recognition result of the obtained speech data to be recognized more accurate and more in line with reality.

[0100] In one embodiment, such as Figure 8 As shown, step S203, which involves determining the target domain based on the optimal recognition result, includes:

[0101] S801: The optimal recognition result is spatially mapped using a word vector mapping model to obtain the target vector;

[0102] S802: The target vector is processed using a neural network model to determine the recognition probability corresponding to multiple configuration domains;

[0103] S803: Identify the configuration area with the highest probability of recognition as the target area.

[0104] The domain determination model includes a word vector mapping model and a neural network model. The word vector mapping model converts the original recognized sentence corresponding to the optimal recognition result into a target vector. The neural network model is used to obtain the recognition probability of the optimal recognition result in different domain configurations based on the target vector.

[0105] As an example, in step S801, the server uses a word vector mapping model to spatially map the optimal recognition result, converting the original recognition sentence corresponding to the optimal recognition result into a vector of a specific format, and using the vector of the specific format as the target vector.

[0106] In one embodiment, the server employs a word-embedding layer, a word vector mapping model, to convert the original recognized sentence corresponding to the optimal recognition result into a target vector. This method ensures the feasibility of subsequently obtaining the recognition probability through a neural network.

[0107] As an example, in step S802, the server uses a neural network model to process the target vector, that is, the target vector is input into the neural network model, and the target vector is identified by language models of multiple configuration domains inside the neural network model, and the optimal recognition result is output as the recognition probability corresponding to different configuration domains.

[0108] In one embodiment, the server employs an LSTM layer neural network model to process the target vector. Specifically, it uses multiple domain-specific language models within the LSTM layer to identify the target vector. After passing through two fully connected layers, it outputs the recognition probability corresponding to each domain. Understandably, the neural network model includes, but is not limited to, LSTM.

[0109] Understandably, before performing the above steps, the server needs to train multiple language models for different configuration domains based on training data from multiple domains. The training process includes:

[0110] The server inputs the domain-labeled text data into the neural network model and outputs the corresponding domain labels. The domain-labeled text data can come from multiple domains. By training with a large amount of domain-labeled text data from different domains, a language model for the corresponding domain is obtained, ultimately training multiple language models for different domains. For example, using text data labeled with the game domain for training results in a game domain language model. In one embodiment, the domain-labeled text data can be either language model training corpus or labeled text from a speech training set.

[0111] As an example, in step S803, the server identifies the configuration domain with the highest probability and determines it as the target domain. For instance, if the optimal identification result is used for domain identification to obtain a certain category of the configuration domain with the highest identification probability, then the domain corresponding to that category is taken as the target domain.

[0112] In the domain-adaptive speech recognition method provided in this embodiment, the recognition probability of the optimal recognition result in multiple configuration domains is obtained through a domain judgment model. The configuration domain with the highest recognition probability is selected as the target domain, which can ensure the accuracy of target domain recognition.

[0113] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0114] In one embodiment, a domain-adaptive speech recognition device is provided, which corresponds one-to-one with the domain-adaptive speech recognition method described in the above embodiments. For example... Figure 9 As shown, the adaptive speech recognition device for this domain includes a speech data acquisition module 901, a first decoding result acquisition module 902, a target domain determination module 903, and a target recognition result acquisition module 904. Detailed descriptions of each functional module are as follows:

[0115] The speech data acquisition module 901 is used to acquire the speech data to be recognized.

[0116] The decoding result acquisition module 902 is used to perform a single decoding of the speech data to be recognized, acquire multiple candidate recognition results, and obtain the optimal recognition result from the multiple candidate recognition results.

[0117] The target domain determination module 903 is used to determine the target domain based on the optimal recognition result.

[0118] The target recognition result acquisition module 904 uses a target language model corresponding to the target domain to perform secondary decoding on multiple candidate recognition results to obtain the target recognition result.

[0119] In one embodiment, the decoding result acquisition module 902 includes:

[0120] The target acoustic score acquisition submodule uses an acoustic model to decode the speech data to be recognized, and obtains multiple original recognition sentences and the target acoustic score corresponding to each original recognition sentence;

[0121] The first language score acquisition submodule uses a general language model to decode the speech data to be recognized, and obtains multiple original recognition sentences and the first language score corresponding to each original recognition sentence.

[0122] The first recognition score acquisition submodule is used to process the target acoustic score and first language score corresponding to multiple original recognition sentences to obtain the first recognition score corresponding to multiple original recognition sentences.

[0123] The candidate recognition result determination submodule is used to determine the top N original recognition sentences with the largest first recognition scores as multiple candidate recognition results, where N≥2;

[0124] The optimal recognition result determination submodule is used to determine the original recognition sentence with the highest first recognition score as the optimal recognition result.

[0125] In one embodiment, the target area determination module 903 includes:

[0126] The target vector acquisition submodule uses a word vector mapping model to spatially map the optimal recognition result and obtain the target vector.

[0127] The probability determination submodule uses a neural network model to process the target vector and determine the recognition probability corresponding to multiple configuration domains.

[0128] The target domain determination submodule is used to determine the configuration domain with the highest recognition probability as the target domain.

[0129] In one embodiment, the target recognition result acquisition module 904 includes:

[0130] The second language score acquisition submodule uses the target language model corresponding to the target domain to perform secondary decoding on multiple candidate recognition results, and obtains multiple candidate recognition sentences and the second language score corresponding to each candidate recognition sentence; wherein, the candidate recognition results include candidate recognition sentences and the target acoustic scores corresponding to the candidate recognition sentences;

[0131] The second recognition score acquisition submodule is used to process the second language score and target acoustic score corresponding to each candidate recognition sentence to obtain the second recognition score corresponding to each candidate recognition sentence;

[0132] The target recognition result determination submodule is used to determine the candidate recognition sentence with the highest second recognition score as the target recognition result.

[0133] In one embodiment, the second identification score acquisition submodule includes:

[0134] The target language score determination unit is used to determine the target language score corresponding to each candidate recognition sentence based on the first language score and the second language score corresponding to each candidate recognition sentence; wherein, the candidate recognition result also includes the first language score corresponding to the candidate recognition sentence;

[0135] The second recognition score acquisition unit is used to process the target language score and the target acoustic score corresponding to each candidate recognition sentence to obtain the second recognition score corresponding to each candidate recognition sentence.

[0136] In one embodiment, the target language score determination unit includes:

[0137] The fusion weight acquisition sub-unit is used to acquire the first fusion weight corresponding to the first language score and the second fusion weight corresponding to the second language score;

[0138] The target language score acquisition subunit is used to obtain the target language score corresponding to each candidate recognition sentence based on the first language score, the first fusion weight, the second language score, and the second fusion weight.

[0139] In another embodiment, the target recognition result acquisition module 904 includes:

[0140] The second language score acquisition submodule uses the target language model corresponding to the target domain to perform secondary decoding on multiple candidate recognition results, and obtains multiple candidate recognition sentences and the second language score corresponding to each candidate recognition sentence.

[0141] The third recognition score acquisition submodule is used to process the second language score and the first recognition score corresponding to each candidate recognition sentence to obtain the third recognition score corresponding to each candidate recognition sentence;

[0142] The target recognition result determination submodule is used to sort multiple candidate recognition sentences based on the third recognition score, and determine the candidate recognition sentence with the highest third recognition score as the target recognition result.

[0143] Specific limitations regarding the domain-adaptive speech recognition device can be found in the limitations of the domain-adaptive speech recognition method above, and will not be repeated here. Each module in the aforementioned domain-adaptive speech recognition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0144] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data used or generated during the execution of a domain-adaptive speech recognition method. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a domain-adaptive speech recognition method.

[0145] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the domain-adaptive speech recognition method described in the above embodiments, for example... Figure 2 As shown in S201-S204, or Figures 3 to 8 As shown, to avoid repetition, it will not be described again here. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in this embodiment of the domain-adaptive speech recognition device, for example... Figure 9 The functions of the speech data acquisition module 901, the first decoding result acquisition module 902, the target region determination module 903, and the target recognition result acquisition module 904 shown are not described again here to avoid repetition.

[0146] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program implements the domain-adaptive speech recognition method described in the above embodiment, for example... Figure 2 As shown in S201-S204, or Figures 3 to 8 As shown, to avoid repetition, it will not be described again here. Alternatively, when the computer program is executed by the processor, it implements the functions of each module / unit in this embodiment of the domain-adaptive speech recognition device, for example... Figure 9 The functions of the speech data acquisition module 901, the first decoding result acquisition module 902, the target region determination module 903, and the target recognition result acquisition module 904 shown are not described again here to avoid repetition. The computer-readable storage medium can be non-volatile or volatile.

[0147] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0148] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0149] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method of domain-adaptive speech recognition, characterized by, The method comprises the following steps: acquiring to-be-recognized voice data; the to-be-recognized voice data is multi-field voice data or cross-field voice data; decoding and processing the to-be-recognized voice data by using an acoustic model to acquire a plurality of original recognition sentences and a target acoustic score corresponding to each original recognition sentence; decoding and processing the to-be-recognized voice data by using a general language model to acquire a plurality of original recognition sentences and a first language score corresponding to each original recognition sentence; processing the target acoustic score and the first language score corresponding to each of the plurality of original recognition sentences to acquire a first recognition score corresponding to each of the plurality of original recognition sentences; sorting the first recognition scores corresponding to the plurality of original recognition sentences, determining the first N original recognition sentences with the largest first recognition scores as a plurality of candidate recognition results, and N is greater than or equal to 2; selecting an original recognition sentence with the largest first recognition score from the plurality of candidate recognition results to determine an optimal recognition result; the candidate recognition result comprises a candidate recognition sentence and a first recognition score corresponding to the candidate recognition sentence; judging a target field according to the optimal recognition result; performing secondary decoding on the plurality of candidate recognition results by using a target language model corresponding to the target field to acquire a target recognition result; the step of performing secondary decoding on the plurality of candidate recognition results by using the target language model corresponding to the target field to acquire the target recognition result comprises the following steps: performing secondary decoding on the plurality of candidate recognition results by using the target language model corresponding to the target field to acquire a plurality of candidate recognition sentences and a second language score corresponding to each candidate recognition sentence; 2. The method of claim 1, wherein the field adaptation is performed by a plurality of field adaptation modules, each field adaptation module being associated with a respective field adaptation type. processing the second language score corresponding to each candidate recognition sentence and the first recognition score to acquire a third recognition score corresponding to each candidate recognition sentence; sorting the plurality of candidate recognition sentences according to the third recognition score, and determining a candidate recognition sentence with the largest third recognition score as the target recognition result. the candidate recognition result comprises a candidate recognition sentence and a target acoustic score corresponding to the candidate recognition sentence; the step of performing secondary decoding on the plurality of candidate recognition results by using the target language model corresponding to the target field to acquire the target recognition result comprises the following steps: performing secondary decoding on the plurality of candidate recognition results by using the target language model corresponding to the target field to acquire a plurality of candidate recognition sentences and a second language score corresponding to each candidate recognition sentence; processing the second language score corresponding to each candidate recognition sentence and the target acoustic score to acquire a second recognition score corresponding to each candidate recognition sentence; 3. The method of claim 2, wherein the field adaptation is performed by: sorting the plurality of candidate recognition sentences according to the second recognition score, and determining a candidate recognition sentence with the largest second recognition score as the target recognition result. the candidate recognition result further comprises a first language score corresponding to the candidate recognition sentence; the step of processing the second language score corresponding to each candidate recognition sentence and the target acoustic score to acquire a second recognition score corresponding to each candidate recognition sentence comprises the following steps: determine a target language score corresponding to each of the candidate recognition sentences according to the first language score and the second language score corresponding to each of the candidate recognition sentences; process the target language score corresponding to each of the candidate recognition sentences and the target acoustic score corresponding to each of the candidate recognition sentences to obtain a second recognition score corresponding to each of the candidate recognition sentences.

4. The method of claim 3, wherein the field adaptation is performed by using a plurality of acoustic models, each of which is associated with a different field. The method further includes: obtaining a first fusion weight corresponding to the first language score and a second fusion weight corresponding to the second language score; obtaining a target language score corresponding to each of the candidate recognition sentences according to the first language score, the first fusion weight, the second language score and the second fusion weight.

5. The method of claim 1, wherein the field adaptation is performed by a plurality of field adaptation modules, each field adaptation module being associated with a respective field adaptation type. The method further includes: performing spatial mapping on the optimal recognition result by using a word vector mapping model to obtain a target vector; processing the target vector by using a neural network model to determine recognition probabilities corresponding to a plurality of configuration domains; determining a target domain as the configuration domain with the largest recognition probability.

6. A domain-adaptive speech recognition apparatus, characterized by, The method further includes: a to-be-recognized voice data obtaining module configured to obtain to-be-recognized voice data, the to-be-recognized voice data being multi-domain voice data or cross-domain voice data; a first decoding result obtaining module configured to perform decoding processing on the to-be-recognized voice data by using an acoustic model to obtain a plurality of original recognition sentences and a target acoustic score corresponding to each of the original recognition sentences, and perform decoding processing on the to-be-recognized voice data by using a general language model to obtain the plurality of original recognition sentences and a first language score corresponding to each of the original recognition sentences, and process the target acoustic scores and the first language scores corresponding to the plurality of original recognition sentences to obtain first recognition scores corresponding to the plurality of original recognition sentences; sort the first recognition scores corresponding to the plurality of original recognition sentences, determine the first N original recognition sentences with the largest first recognition scores as a plurality of candidate recognition results, N≥2, and select one original recognition sentence with the largest first recognition score from the plurality of candidate recognition results as an optimal recognition result; the candidate recognition result includes a candidate recognition sentence and a first recognition score corresponding to the candidate recognition sentence; a target domain determining module configured to determine a target domain according to the optimal recognition result; a target recognition result obtaining module configured to perform secondary decoding on the plurality of candidate recognition results by using a target language model corresponding to the target domain to obtain a target recognition result; the target recognition result obtaining module includes: performing secondary decoding on the plurality of candidate recognition results by using a target language model corresponding to the target domain to obtain a plurality of candidate recognition sentences and a second language score corresponding to each of the candidate recognition sentences. The second language score corresponding to each of the candidate recognition sentences and the first recognition score are processed to obtain a third recognition score corresponding to each of the candidate recognition sentences; and the candidate recognition sentences are sorted according to the third recognition scores, and a candidate recognition sentence with the largest third recognition score is determined as a target recognition result.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The computer program, when executed by the processor, implements the domain-adaptive speech recognition method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 7. The computer program, when executed by the processor, implements the domain-adaptive speech recognition method according to any one of claims 1 to 5.

Images