Speech recognition method, device, electronic device and storage medium

Abstract

The present invention provides a speech recognition method, apparatus, electronic device, and storage medium. The method comprises: determining a speech to be recognized; performing acoustic state prediction on the spectral features of the speech to be recognized based on a first speech recognition model to obtain a first acoustic state posterior probability of the speech to be recognized; performing acoustic state prediction on the semantic features of the speech to be recognized based on a second speech recognition model to obtain a second acoustic state posterior probability of the speech to be recognized; fusing the first acoustic state posterior probability with the second acoustic state posterior probability, and performing speech recognition decoding based on the fused posterior probability to obtain recognized text of the speech to be recognized. The speech recognition method, apparatus, electronic device, and storage medium provided by the present invention can accurately perform speech recognition in domain scenarios.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to a speech recognition method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of artificial intelligence technology, speech recognition technology has been widely used in various fields such as education, entertainment, medical care, and transportation.

[0003] Currently, speech recognition is often performed by collecting large amounts of corpus data from general scenarios to train a speech recognition model. However, when this speech recognition model is applied to specific domain scenarios, the speech recognition accuracy is low. Summary of the Invention

[0004] The present invention provides a speech recognition method, device, electronic device and storage medium to solve the defect of low speech recognition accuracy in specific field scenarios in the prior art.

[0005] The present invention provides a speech recognition method, comprising:

[0006] Determine the speech to be recognized;

[0007] Based on the first speech recognition model, performing acoustic state prediction on the spectral features of the speech to be recognized to obtain a first acoustic state posterior probability of the speech to be recognized;

[0008] Based on the second speech recognition model, performing acoustic state prediction on the semantic features of the speech to be recognized to obtain a second acoustic state posterior probability of the speech to be recognized;

[0009] fusing the first acoustic state posterior probability and the second acoustic state posterior probability, performing speech recognition decoding based on the fused posterior probability obtained by fusion, and obtaining a recognition text of the speech to be recognized;

[0010] The first speech recognition model is trained based on the spectral features of domain sample speech and its label recognition text, and the second speech recognition model is trained based on the semantic features of the domain sample speech and its label recognition text.

[0011] According to a speech recognition method provided by the present invention, the speech recognition decoding is performed based on the fusion posterior probability obtained by fusion to obtain the recognition text of the speech to be recognized, including:

[0012] Perform speech recognition decoding based on the fused posterior probability to obtain at least two first candidate texts;

[0013] Determining a recognition score for each first candidate text based on the acoustic fit of each first candidate text with the speech to be recognized and / or the language fluency of each first candidate text;

[0014] Based on the recognition scores of the first candidate texts, the recognition text of the speech to be recognized is determined.

[0015] According to a speech recognition method provided by the present invention, determining the recognition text of the speech to be recognized based on the recognition scores of the first candidate texts includes:

[0016] Performing speech recognition on the spectral features of the speech to be recognized based on a third speech recognition model to obtain at least two second candidate texts;

[0017] Determining a reference recognition text based on the recognition score of each second candidate text;

[0018] Determining the number of characters per unit duration of the reference recognition text based on the number of characters in the reference recognition text and the voice duration of the voice to be recognized;

[0019] Determining the recognized text based on the number of characters per unit time length of the reference recognized text and the recognition score of each first candidate text;

[0020] The second candidate texts are obtained by performing speech recognition on the spectral features of the speech to be recognized based on a third speech recognition model, and the third speech recognition model is trained based on the spectral features of domain sample speech and its label recognition text.

[0021] According to a speech recognition method provided by the present invention, determining the recognized text based on the number of characters per unit time length of the reference recognized text and the recognition score of each first candidate text includes:

[0022] When the number of characters per unit time length of the reference recognition text is less than or equal to a character threshold, the recognition text is determined based on the recognition scores of the first candidate texts and the recognition scores of the second candidate texts.

[0023] According to a speech recognition method provided by the present invention, determining the recognized text based on the number of characters per unit time length of the reference recognized text and the recognition score of each first candidate text includes:

[0024] When the number of characters per unit time length of the reference recognition text is greater than a character threshold, the recognition text is determined based on the recognition scores of the first candidate texts.

[0025] According to a speech recognition method provided by the present invention, the step of determining the acoustic fit between each first candidate text and the speech to be recognized includes:

[0026] performing speech recognition on the to-be-recognized speech based on a fourth speech recognition model, obtaining a recognition probability of each first candidate text being a recognition result of the fourth speech recognition model, and determining the recognition probability as the acoustic fit;

[0027] The fourth speech recognition model is trained based on the spectral features of the domain sample speech and its label recognition text, and the structure of the fourth speech recognition model is different from that of the second speech recognition model;

[0028] The step of determining the language fluency of each of the first candidate texts and the speech to be recognized comprises:

[0029] performing speech recognition on the speech to be recognized based on the fifth speech recognition model, obtaining a recognition probability of each first candidate text being a recognition result of the fifth speech recognition model, and using the recognition probability as the language fluency;

[0030] The fifth speech recognition model is trained based on the spectrum features of common sample speech and its label recognition text. The structure of the fifth speech recognition model is different from that of the fourth speech recognition model.

[0031] According to a speech recognition method provided by the present invention, the fusing of the first acoustic state posterior probability and the second acoustic state posterior probability, performing speech recognition decoding based on the fused posterior probability obtained by fusion, and obtaining the recognition text of the speech to be recognized includes:

[0032] Based on a preset weight, performing weighted fusion on the first acoustic state posterior probability and the second acoustic state posterior probability to obtain the fused posterior probability;

[0033] Speech recognition decoding is performed based on the fused posterior probability to obtain a recognition text of the speech to be recognized.

[0034] The present invention also provides a speech recognition device, comprising:

[0035] A speech determination unit, configured to determine a speech to be recognized;

[0036] a first prediction unit, configured to perform acoustic state prediction on the spectral features of the to-be-recognized speech based on a first speech recognition model, and obtain a first acoustic state posterior probability of the to-be-recognized speech;

[0037] a second prediction unit, configured to perform acoustic state prediction on the semantic features of the to-be-recognized speech based on a second speech recognition model, to obtain a second acoustic state posterior probability of the to-be-recognized speech;

[0038] a speech recognition unit, configured to fuse the first acoustic state posterior probability and the second acoustic state posterior probability, perform speech recognition decoding based on the fused posterior probability, and obtain a recognition text of the speech to be recognized;

[0039] The first speech recognition model is trained based on the spectral features of domain sample speech and its label recognition text, and the second speech recognition model is trained based on the semantic features of the domain sample speech and its label recognition text.

[0040] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-described speech recognition methods when executing the program.

[0041] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which implements any of the above-mentioned speech recognition methods when executed by a processor.

[0042] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned speech recognition methods.

[0043] The speech recognition method, device, electronic device and storage medium provided by the present invention fuse the first acoustic state posterior probability and the second acoustic state posterior probability, so that the fused posterior probability obtained by fusion contains the posterior probability of the spectral feature and the posterior probability of the semantic feature, thereby realizing the complementarity of the spectral feature and the semantic feature during speech recognition, thereby enabling accurate speech recognition decoding based on the fused posterior probability obtained after fusion, and further accurately recognizing the speech in the domain scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 1 is a flow chart of the speech recognition method provided by the present invention;

[0046] Figure 2 This is one of the flowcharts of the implementation of step 140 in the speech recognition method provided by the present invention;

[0047] Figure 3 is a flowchart of an implementation of step 143a in the speech recognition method provided by the present invention;

[0048] Figure 4 This is a second flowchart of an implementation of step 140 in the speech recognition method provided by the present invention;

[0049] Figure 5 It is a structural diagram of the speech recognition device provided by the present invention;

[0050] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0051] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0052] With the rapid development of artificial intelligence (AI), speech recognition technology has been widely used in various fields, including education, entertainment, healthcare, and transportation. Currently, speech recognition is performed based on a speech recognition model obtained by collecting a large amount of corpus data from common scenarios. This model has good recognition performance when applied to common scenarios.

[0053] However, since the speech to be recognized corresponding to specific domain scenarios and general scenarios has different degrees of differences in channels, topics, speakers, environmental noise, etc., the recognition effect is poor when the speech recognition model trained by traditional methods is applied to specific scenarios.

[0054] In addition, if a speech recognition model in a specific domain scenario is trained based on corpus data in a specific domain scenario, in order to enable the trained model to have a better recognition effect, a large amount of corpus data in the specific domain scenario is required. However, corpus data in specific domain scenarios are usually not easy to obtain, that is, it is difficult to obtain sufficient corpus data in specific domain scenarios.

[0055] To this end, the present invention provides a speech recognition method. Figure 1 It is a flow chart of the speech recognition method provided by the present invention, such as Figure 1 As shown, the method includes the following steps:

[0056] Step 110: Determine the speech to be recognized.

[0057] Specifically, the speech to be recognized is the speech data that needs to be recognized. The speech to be recognized can be speech data recorded in real time by a user through an electronic device. The electronic device here can be a smartphone, a tablet computer, or a smart appliance such as a stereo, a television, or an air conditioner. After obtaining the speech to be recognized, the electronic device can also amplify and reduce noise of the speech to be recognized. In addition, the speech to be recognized can also be stored or received speech data, which is not specifically limited in the embodiments of the present invention.

[0058] Step 120: Based on the first speech recognition model, perform acoustic state prediction on the spectral features of the speech to be recognized to obtain a first acoustic state posterior probability of the speech to be recognized; the first speech recognition model is trained based on the spectral features of the domain sample speech and its label recognition text.

[0059] Specifically, the spectral features of the speech to be recognized are used to represent the syllable information of each word in the speech to be recognized, that is, it can be understood that the spectral features of the speech to be recognized are used to represent the detailed feature information of the speech to be recognized.

[0060] After determining the speech to be recognized, the spectral features of the speech to be recognized may be extracted. The spectral features of the speech to be recognized may be extracted using a feature extraction algorithm, such as a Fourier transform-based spectral feature extraction algorithm.

[0061] After obtaining the spectral features of the speech to be recognized, the spectral features are input into the first speech recognition model. The first speech recognition model then performs acoustic state prediction based on the syllable information of each word segment contained in the spectral features to obtain a first acoustic state posterior probability of the speech to be recognized. The first acoustic state posterior probability is used to represent the posterior probability corresponding to the spectral features. This means that the first acoustic state posterior probability represents the importance of the spectral features for speech recognition. A greater first acoustic state posterior probability indicates a more important spectral feature for speech recognition, and therefore a more accurate recognition result is obtained from speech recognition based on the spectral features.

[0062] The first speech recognition model is trained based on domain sample speech and its label recognition text. The domain sample speech is speech collected in a domain scenario, and its label recognition text is the label corresponding to the domain sample speech. Speech in a domain scenario differs from speech in a general scenario in terms of channel, topic, speaker, and ambient noise. For example, a general scenario can be a general life scenario, while a domain scenario can be a specific industry scenario, such as a medical scenario.

[0063] Step 130: Based on the second speech recognition model, the semantic features of the speech to be recognized are predicted to obtain a second acoustic state posterior probability of the speech to be recognized; the second speech recognition model is trained based on the semantic features of the domain sample speech and its label recognition text.

[0064] Specifically, the semantic features of the speech to be recognized are used to represent the contextual semantic information between each word in the speech to be recognized, that is, it can be understood that the spectral features of the speech to be recognized are used to represent the global feature information of the speech to be recognized.

[0065] After determining the speech to be recognized, semantic features of the speech to be recognized are extracted. The semantic features of the speech to be recognized can be extracted based on a domain speech recognition model. The domain speech recognition model can be trained based on domain sample speech and its labeled text recognition, using a second initial model. The second initial model can be trained based on general sample speech and its labeled text recognition. The general sample speech is speech collected in a general scenario. The structure of the first initial model can be an end-to-end model structure.

[0066] After obtaining the semantic features of the speech to be recognized, the semantic features are input into the second speech recognition model. The second speech recognition model then performs acoustic state prediction based on the contextual information between the word segments contained in the semantic features to obtain a second acoustic state posterior probability of the speech to be recognized. The second acoustic state posterior probability is used to represent the posterior probability corresponding to the semantic features. This can be understood as representing the importance of the semantic features for speech recognition. A greater second acoustic state posterior probability indicates a more important semantic feature for speech recognition, and therefore a more accurate recognition result is obtained from speech recognition based on the semantic features.

[0067] The second speech recognition model is trained based on domain sample speech and its label recognition text. The domain sample speech is speech collected in a domain scenario, and its label recognition text is the label corresponding to the domain sample speech. The domain scenario can be a specific industry domain scenario, such as a medical domain scenario. The label recognition text of the domain sample speech can be determined based on the first initial model of the first speech recognition model or the second speech recognition model. The first initial model can be trained based on general sample speech and its label recognition text.

[0068] It should be noted that the structures of the first speech recognition model and the second speech recognition model can be the same or different. For example, the structures of the first speech recognition model and the second speech recognition model can both be end-to-end model structures, or the structure of the first speech recognition model can be an acoustic model + language model structure, and the structure of the second speech recognition model can be an end-to-end model structure. The embodiment of the present invention does not make specific limitations on this.

[0069] It is understandable that the first speech recognition model and the second speech recognition model can be trained based on the domain sample speech and its labeled recognition text to obtain the general speech recognition model. Among them, the general speech recognition model is trained based on the general sample speech and its labeled recognition text. Since the general speech recognition model has a good recognition effect in general scenarios, that is, the general speech recognition model can accurately recognize general words, general sentences, etc. in the speech, the first speech recognition model and the second speech recognition model do not need to incrementally obtain the corresponding domain sample speech for training based on general words, general sentences, etc. in the domain scenario, but can be trained based on specific words, specific sentences, etc. in the domain scenario, which not only reduces the amount of domain sample speech collected, but also speeds up the convergence of the model.

[0070] Step 140: Fuse the first acoustic state posterior probability and the second acoustic state posterior probability, perform speech recognition decoding based on the fused posterior probability, and obtain a recognition text of the speech to be recognized.

[0071] Specifically, spectral features are used to represent the syllable information of each word segment, and semantic features are used to represent the contextual information between each word segment. That is, spectral features represent the feature information of the speech to be recognized from a detailed level, and speech features represent the feature information of the speech to be recognized from a global level.

[0072] For different speech, the importance of spectral features and semantic features in speech recognition may be different, that is, for some speech, the importance of spectral features may be greater than that of semantic features; for some speech, the importance of semantic features may be greater than that of spectral features.

[0073] To this end, an embodiment of the present invention fuses the first acoustic state posterior probability and the second acoustic state posterior probability, so that the fused posterior probability obtained by fusion includes the posterior probability of spectral features and the posterior probability of semantic features, and the spectral features and the semantic features are feature information that characterize the speech to be recognized from different levels, that is, the spectral features and the semantic features can complement each other, so that speech recognition decoding can be accurately performed based on the fused posterior probability obtained after fusion, and then the recognition text of the speech to be recognized can be accurately obtained.

[0074] It should be noted that the embodiments of the present invention achieve complementarity between spectral features and semantic features, that is, it can simultaneously perform speech recognition with the help of information in spectral features and semantic features, so there is no need to incrementally collect domain sample speech and its label recognition text for training to learn the information in spectral features or semantic features, thereby reducing the amount of domain sample speech collected.

[0075] The speech recognition method provided by the embodiment of the present invention fuses the posterior probability of the first acoustic state and the posterior probability of the second acoustic state, so that the fused posterior probability obtained by fusion contains the posterior probability of the spectral feature and the posterior probability of the semantic feature, thereby realizing the complementarity of the spectral feature and the semantic feature during speech recognition, thereby enabling accurate speech recognition decoding based on the fused posterior probability obtained after fusion, and further accurately recognizing the speech in the domain scenario.

[0076] Based on the above embodiments, Figure 2 This is one of the flow charts of the implementation of step 140 in the speech recognition method provided by the present invention, such as Figure 2 As shown, in step 140, speech recognition decoding is performed based on the fused posterior probability obtained by fusion to obtain the recognition text of the speech to be recognized, including:

[0077] Step 141a: Perform speech recognition decoding based on the fused posterior probability to obtain at least two first candidate texts.

[0078] Specifically, after performing speech recognition decoding based on the fused posterior probability, at least two candidate recognition texts can be obtained, and the confidence levels of each candidate recognition text may be different. All candidate recognition texts can be used as the first candidate text, or candidate recognition texts with a confidence level greater than a threshold can be selected as the first candidate text. This is not specifically limited in the embodiments of the present invention. The confidence level of each candidate recognition text can be represented by the probability that each first candidate recognition text is the recognition result of the speech to be recognized.

[0079] Among them, a weighted finite state transducer (WFST) can be used to jointly fuse the posterior probability for speech recognition decoding to obtain multiple first candidate texts.

[0080] Step 142a: A recognition score for each first candidate text based on the acoustic fit of each first candidate text with the speech to be recognized and / or the language fluency of each first candidate text;

[0081] Step 143a: Determine the recognition text of the speech to be recognized based on the recognition scores of the first candidate texts.

[0082] Specifically, the acoustic fit is used to characterize the degree of match between each word in each first candidate text and the corresponding word in the speech to be recognized from an acoustic level. The higher the acoustic fit, the higher the degree of match between each word in the corresponding first candidate text and the corresponding word in the speech to be recognized, that is, the greater the probability that the corresponding first candidate text is the recognized text. Language fluency is used to characterize the degree of fluency of the language expression of each first candidate text from a linguistic level. The higher the language fluency, the smoother the language expression of the corresponding first candidate text, that is, the greater the probability that the corresponding first candidate text is the recognized text. The recognition score of each first candidate text is used to characterize the confidence that each first candidate text is the recognized text of the speech to be recognized. The higher the recognition score, the higher the confidence that the corresponding first candidate text is the recognized text.

[0083] Among them, the acoustic fit between each first candidate text and the speech to be recognized can be used as the recognition score of the corresponding first candidate text, the language fluency of each first candidate text can be used as the recognition score of the corresponding first candidate text, and the acoustic fit and language fluency of each first candidate text can also be weightedly fused, and the fused result can be used as the recognition score of the corresponding first candidate text.

[0084] After determining the recognition scores of the first candidate texts, the first candidate text corresponding to the maximum recognition score may be used as the recognized text, or the first candidate text with a recognition score greater than a threshold may be used as the recognized text, which is not specifically limited in the embodiment of the present invention.

[0085] It can be seen from this that the embodiment of the present invention determines the recognition score of each first candidate text from the acoustic level and / or language level based on the acoustic fit of each first candidate text with the speech to be recognized, and / or the language fluency of each first candidate text, and then can accurately determine the recognition text of the speech to be recognized based on the recognition score of each first candidate text.

[0086] Based on any of the above embodiments, Figure 3 143a in the speech recognition method provided by the present invention, as shown in FIG. Figure 3 As shown, step 143a specifically includes:

[0087] Step 1431a: Based on the third speech recognition model, perform speech recognition on the spectral features of the speech to be recognized to obtain at least two second candidate texts.

[0088] Specifically, the second candidate recognition text is obtained by performing speech recognition on the spectral features of the speech to be recognized based on the third speech recognition model. Since the spectral features are used to represent the detailed feature information of the speech to be recognized, the second candidate recognition text can be understood as the recognition result obtained from the local detail level.

[0089] Step 1432a: Determine a reference recognition text based on the recognition score of each second candidate text.

[0090] Specifically, the recognition score of each second candidate text is used to characterize the confidence that each second candidate text is the recognized text of the speech to be recognized. The higher the recognition score, the higher the confidence that the corresponding second candidate text is the recognized text. Among them, the recognition score of each second candidate text can be determined based on the acoustic fit of each second candidate text and the speech to be recognized, and / or the language fluency of each second candidate text. For example, the acoustic fit of each second candidate text and the speech to be recognized can be used as the recognition score of the corresponding second candidate text, or the language fluency of each second candidate text can be used as the recognition score of the corresponding second candidate text. The acoustic fit and language fluency of each second candidate text can also be weightedly fused, and the fused result can be used as the recognition score of the corresponding second candidate text.

[0091] Optionally, the embodiment of the present invention may use the second candidate text corresponding to the highest recognition score as the reference recognition text, or may use the second candidate text with a recognition score greater than a threshold as the reference recognition text, which is not specifically limited in the embodiment of the present invention.

[0092] Step 1433a: Determine the number of characters per unit duration of the reference recognition text based on the number of characters of the reference recognition text and the speech duration of the speech to be recognized.

[0093] Specifically, the number of characters per unit time length of the reference recognition text refers to the number of characters per unit time length corresponding to the reference recognition text obtained when the third speech recognition model recognizes the speech to be recognized. It can be determined by the speech duration of the speech to be recognized and the number of characters in the reference recognition text, such as the number of characters per unit time length of the reference recognition text = the number of characters in the reference recognition text / the speech duration of the speech to be recognized.

[0094] Step 1434a: Determine the recognized text based on the number of characters per unit time length of the reference recognized text and the recognition score of each first candidate text.

[0095] Specifically, the third speech recognition model may have insertion errors when performing speech recognition on the speech to be recognized. For example, the speech to be recognized is "I love work", but the third speech recognition model may have insertion errors during the recognition process, resulting in the reference recognition text obtained being "I love love love love love love work", which obviously contains too many inserted words "love".

[0096] Given that there is usually an upper limit to the number of characters a user can speak per unit time, if the reference recognition text has a larger number of characters per unit time, then the probability of insertion errors in the reference recognition text output by the third speech recognition model is higher, and each second candidate recognition text is determined based on the third speech recognition model, which means that the probability of insertion errors in each second candidate recognition text is also higher. In this case, the recognition text can be determined from each first candidate text based on the recognition score of each first candidate text. If the reference recognition text has a smaller number of characters per unit time, then the probability of insertion errors in the reference recognition text output by the third speech recognition model is lower, and each second candidate recognition text is determined based on the third speech recognition model, which means that the probability of insertion errors in each second candidate recognition text is also lower, which means that the confidence of the second candidate recognition text is higher. At the same time, since each first candidate text is a recognition result obtained by combining spectral features and semantic features, that is, the confidence of each first candidate text is higher, the recognition text can be determined from each first candidate text and each second candidate text with higher confidence.

[0097] It can be seen that the embodiment of the present invention can determine whether there is an insertion error in the reference recognition text based on the number of characters per unit time length of the reference recognition text, and thus can accurately obtain the recognition text.

[0098] Based on any of the above embodiments, step 1434a specifically includes:

[0099] When the number of characters per unit time length of the reference recognition text is less than or equal to the character threshold, the recognition text is determined based on the recognition scores of the first candidate texts and the recognition scores of the second candidate texts.

[0100] It should be noted that the reference recognition text is the text determined from each second candidate text, that is, the reference recognition text is the recognition result obtained from the local detail level, and the first candidate text is obtained by speech recognition based on semantic features, and the semantic features are used to represent the global feature information in the speech to be recognized, so the first candidate recognition text can be understood as the recognition result obtained from the global level. The number of characters per unit time length of the reference recognition text can be understood as the global feature information used to represent the speech to be recognized, while the reference recognition text is the recognition result obtained by focusing on the detail feature information, which may ignore the global feature information during the recognition process. Therefore, the embodiment of the present invention determines whether there is an insertion error in the reference recognition text based on the number of characters per unit time length of the reference recognition text.

[0101] When the number of characters per unit time length of the reference recognition text is less than or equal to the character threshold, it indicates that the probability of insertion errors in the reference recognition text output in the third speech recognition model is low, and each second candidate recognition text is determined based on the third speech recognition model, which indicates that the probability of insertion errors in each second candidate recognition text is also low, that is, the confidence of the second candidate recognition text is high; at the same time, since each first candidate text is a recognition result obtained by combining spectral features and semantic features, that is, the confidence of each first candidate text is high, at this time, the recognition text can be determined from the first candidate texts and the second candidate texts with high confidence levels, so as to accurately obtain the recognition text of the speech to be recognized.

[0102] Based on any of the above embodiments, step 1434a specifically includes:

[0103] When the number of characters per unit time length of the reference recognition text is greater than the character threshold, the recognition text is determined based on the recognition scores of the first candidate texts.

[0104] Specifically, when the number of characters per unit time length of the reference recognition text is greater than the character threshold, it indicates that the probability of insertion errors in the reference recognition text output in the third speech recognition model is higher, and each second candidate recognition text is determined based on the third speech recognition model, which also indicates that the probability of insertion errors in each second candidate recognition text is also high, that is, the confidence of each second candidate recognition text is low. In this case, the recognition text is determined from each first candidate text to accurately obtain the recognition text of the speech to be recognized.

[0105] Based on any of the above embodiments, the step of determining the acoustic fit between each first candidate text and the speech to be recognized includes:

[0106] Performing speech recognition on the speech to be recognized based on the fourth speech recognition model, obtaining a recognition probability of each first candidate text as a recognition result of the fourth speech recognition model, and determining the recognition probability as the acoustic fit;

[0107] The fourth speech recognition model is trained based on the spectral features of the domain sample speech and its label recognition text. The structure of the fourth speech recognition model is different from that of the second speech recognition model.

[0108] Specifically, the acoustic fit is used to characterize the matching degree between each word in each first candidate text and the corresponding word in the speech to be recognized from the acoustic level. The higher the acoustic fit, the higher the matching degree between each word in the corresponding first candidate text and the corresponding word in the speech to be recognized, that is, the greater the probability that the corresponding first candidate text is the recognized text.

[0109] Among them, the acoustic fit of each first candidate text is obtained by inputting the spectral features of the speech to be recognized into the fourth speech recognition model, and the fourth speech recognition model uses each first candidate text as the recognition result to determine the recognition probability of each first candidate text as the recognition result, and uses the recognition probability as the acoustic fit of each first candidate text.

[0110] Similarly, the acoustic fit of each second candidate text is obtained by inputting the spectral features of the speech to be recognized into the fourth speech recognition model, and the fourth speech recognition model uses each second candidate text as the recognition result to determine the recognition probability of each second candidate text as the recognition result, and uses the recognition probability as the acoustic fit of each second candidate text.

[0111] It should be noted that the structure of the fourth speech recognition model and the third speech recognition model may be the same or different. If the structure of the fourth speech recognition model is the same as that of the third speech recognition model, the acoustic fit of the second candidate text, the acoustic fit of the first candidate text, and the acoustic fit of the second candidate text are all determined based on the same model, that is, the acoustic fit of the first candidate text and the acoustic fit of the second candidate text are determined under the same conditions, and the accuracy is higher.

[0112] Based on any of the above embodiments, the step of determining the language fluency of each first candidate text and the speech to be recognized includes:

[0113] Based on the fifth speech recognition model, performing speech recognition on the speech to be recognized, obtaining a recognition probability of each first candidate text as a recognition result of the fifth speech recognition model, and using the recognition probability as the language fluency;

[0114] The fifth speech recognition model is trained based on the spectral features of common sample speech and its label recognition text. The structure of the fifth speech recognition model is different from that of the fourth speech recognition model.

[0115] Specifically, language fluency is used to characterize the fluency of the language expression of each first candidate text from a language level. The higher the language fluency, the smoother the language expression of the corresponding first candidate text, that is, the greater the probability that the corresponding first candidate text is the recognition text.

[0116] Among them, the language fluency of each first candidate text is obtained by inputting the spectral features of the speech to be recognized into the fifth speech recognition model, and the fifth speech recognition model uses each first candidate text as the recognition result to determine the recognition probability of each first candidate text as the recognition result, and uses the recognition probability as the language fluency of each first candidate text.

[0117] Similarly, the language fluency of each second candidate text is obtained by inputting the spectral features of the speech to be recognized into the fifth speech recognition model, and the fifth speech recognition model uses each second candidate text as the recognition result to determine the recognition probability of each second candidate text as the recognition result, and uses the recognition probability as the language fluency of each second candidate text.

[0118] Among them, the structure of the fifth speech recognition model is different from that of the fourth speech recognition model, so that the fifth speech recognition model and the fourth speech recognition model can determine the acoustic fit and language fluency of the candidate text from different levels. For example, the fourth speech recognition model can be an acoustic model (such as an end-to-end model), and the fifth speech recognition model can be a language model (such as a GPT-2 language model), so that the fourth speech recognition model can determine the acoustic fit from the acoustic level, and the fifth speech recognition model can determine the language fluency from the language level, thereby accurately obtaining the recognition score of the candidate text to accurately determine the recognition text of the speech to be recognized.

[0119] Based on any of the above embodiments, Figure 4 This is a second flow chart of the implementation of step 140 in the speech recognition method provided by the present invention, as shown in FIG. Figure 4 As shown, step 140 specifically includes:

[0120] Step 141b: Based on preset weights, perform weighted fusion on the first acoustic state posterior probability and the second acoustic state posterior probability to obtain a fused posterior probability;

[0121] Step 142b: Perform speech recognition decoding based on the fused posterior probability to obtain the recognition text of the speech to be recognized.

[0122] Specifically, the first acoustic state posterior probability is used to represent the posterior probability corresponding to the spectral feature, and the second acoustic state posterior probability is used to represent the posterior probability corresponding to the semantic feature. For different speech sounds, the importance of spectral features and semantic features in speech recognition may vary, that is, the weights corresponding to the first and second acoustic state posterior probabilities are different.

[0123] To this end, the embodiment of the present invention performs a weighted fusion of the first acoustic state posterior probability and the second acoustic state posterior probability based on preset weights, so that the fused posterior probability obtained by fusion incorporates the posterior probability of the spectral features and the posterior probability of the semantic features. The spectral features and the semantic features are feature information that characterize the speech to be recognized from different levels. That is, the spectral features and the semantic features can complement each other, thereby enabling accurate speech recognition decoding based on the fused posterior probability obtained after fusion, and further accurately obtaining the recognized text of the speech to be recognized. Among them, the preset weights can be specifically set based on actual conditions, and the embodiment of the present invention does not specifically limit this.

[0124] Optionally, the fused posterior probability can be determined based on the following formula:

[0125]

[0126] Where, represents the fusion posterior probability, represents the posterior probability of the first acoustic state, represents the posterior probability of the second acoustic state, α represents the preset weight, and its value range is (0,1).

[0127] Based on any of the above embodiments, the present invention further provides a speech recognition method, the method comprising:

[0128] First, the speech to be recognized is determined, and its spectral features are extracted. Semantic features of the speech to be recognized are then extracted based on a domain speech recognition model. The domain speech recognition model can be trained on domain sample speech and its labeled recognition text, using a general speech recognition model. The general speech recognition model is trained on general sample speech and its labeled recognition text, and its model structure can be an end-to-end model.

[0129] Next, based on the first speech recognition model, the spectral features of the speech to be recognized are acoustically predicted to obtain the first acoustic state posterior probability of the speech to be recognized. At the same time, based on the second speech recognition model, the semantic features of the speech to be recognized are acoustically predicted to obtain the second acoustic state posterior probability of the speech to be recognized. Among them, the first speech recognition model is based on the spectral features of the domain sample speech and its label recognition text, and the first initial model is trained. The second speech recognition model is based on the semantic features of the domain sample speech and its label recognition text, and the first initial model is trained. The first initial model is trained based on the general sample speech and its label recognition text. The structure of the first initial model can be an acoustic model + language model structure, that is, the structure of the first speech recognition model is the same as that of the second speech recognition model.

[0130] When training the first speech recognition model and the second speech recognition model based on the first initial model, transfer learning or other learning methods may be used for training, which is not specifically limited in the embodiment of the present invention. Furthermore, the label recognition text used to train the first speech recognition model and the second speech recognition model may be frame-level state label text, which can be used to perform speech recognition determination based on the spectral features of the domain sample speech based on the first initial model.

[0131] After determining the posterior probability of the first acoustic state and the posterior probability of the second acoustic state, the two are fused based on the preset weights to obtain the fused posterior probability, and WFST is used for joint speech recognition decoding to obtain multiple first candidate texts. At the same time, based on the third speech recognition model, speech recognition is performed on the spectral features of the speech to be recognized to obtain multiple second candidate texts. Among them, the third speech recognition model is trained based on the spectral features of the domain sample speech and its label recognition text. The structure of the third speech recognition model is different from that of the first speech recognition model. For example, the structure of the first speech recognition model is an acoustic model + language model structure, while the structure of the third speech recognition model is an end-to-end model structure.

[0132] Based on the third speech recognition model, using each first candidate text as the answer, the acoustic fit of each first candidate text with the speech to be recognized is determined, and based on the fifth speech recognition model, the language fluency of each first candidate text is determined. Then, a weighted fusion is performed on the acoustic fit of each first candidate text with the speech to be recognized and the language fluency of each first candidate text to determine the recognition score of each first candidate text. The fifth speech recognition model is trained based on the spectral features of general sample speech and its labeled recognition text, and the structure of the fifth speech recognition model can be a GPT-2 language model structure.

[0133] Similarly, the recognition score of each second candidate text is determined using the method for determining the recognition score of each first candidate text. The weights used to determine the recognition scores of each first candidate text and each second candidate text can be set according to actual conditions and are not specifically limited in this embodiment of the present invention.

[0134] Next, the text with the highest recognition score is selected from each second candidate text as the reference recognition text, and the number of characters per unit time length of the reference recognition text is determined based on the number of characters in the reference recognition text and the speech duration of the speech to be recognized.

[0135] If the number of characters per unit time of the reference recognition text is less than or equal to the character threshold, the text with the highest recognition score among the first candidate texts and the second candidate texts is selected as the recognized text for the speech to be recognized. If the number of characters per unit time of the target text is greater than the character threshold, the text with the highest recognition score among the first candidate texts is selected as the recognized text for the speech to be recognized.

[0136] Thus, it can be seen that the embodiment of the present invention achieves the complementarity of spectral features and semantic features during speech recognition based on the fused posterior probability obtained by fusion, thereby accurately performing speech recognition decoding based on the fused posterior probability obtained after fusion, and then accurately obtaining each first candidate text. At the same time, the recognition score of each first candidate text and the recognition score of each second candidate text are determined in combination with the third speech recognition model and the fifth speech recognition model, thereby accurately determining the recognition text of the speech to be recognized from each first candidate text and each second candidate text.

[0137] The speech recognition device provided by the present invention is described below. The speech recognition device described below and the speech recognition method described above can be referenced to each other.

[0138] Based on any of the above embodiments, the present invention further provides a speech recognition device, Figure 5 Schematic diagram of the structure of the speech recognition device provided by the present invention, such as Figure 5 As shown, the device includes:

[0139] A speech determination unit 510 is used to determine the speech to be recognized;

[0140] A first prediction unit 520 is configured to perform acoustic state prediction on the spectral features of the speech to be recognized based on a first speech recognition model to obtain a first acoustic state posterior probability of the speech to be recognized;

[0141] The second prediction unit 530 is configured to perform acoustic state prediction on the semantic features of the speech to be recognized based on the second speech recognition model to obtain a posterior probability of a second acoustic state of the speech to be recognized;

[0142] The speech recognition unit 540 is configured to fuse the first acoustic state posterior probability and the second acoustic state posterior probability, perform speech recognition decoding based on the fused posterior probability, and obtain a recognition text of the speech to be recognized;

[0143] The first speech recognition model is trained based on the spectral features of domain sample speech and its label recognition text, and the second speech recognition model is trained based on the semantic features of the domain sample speech and its label recognition text.

[0144] Based on any of the above embodiments, the speech recognition unit 540 includes:

[0145] a decoding unit, configured to perform speech recognition decoding based on the fused posterior probability to obtain at least two first candidate texts;

[0146] a score determination unit, configured to determine a recognition score for each first candidate text based on an acoustic fit between each first candidate text and the speech to be recognized, and / or a language fluency of each first candidate text;

[0147] The recognized text determining unit is configured to determine the recognized text of the speech to be recognized based on the recognition scores of the first candidate texts.

[0148] Based on any of the above embodiments, the text determination unit includes:

[0149] a candidate text determination unit, configured to perform speech recognition on the spectral features of the speech to be recognized based on a third speech recognition model to obtain at least two second candidate texts;

[0150] a reference text determining unit, configured to determine a reference recognition text based on the recognition score of each second candidate text;

[0151] a character number determining unit, configured to determine the number of characters per unit time length of the reference recognition text based on the number of characters in the reference recognition text and the voice time length of the voice to be recognized;

[0152] a text determination subunit, configured to determine the recognized text based on the number of characters per unit time length of the reference recognized text and the recognition score of each first candidate text;

[0153] The second candidate texts are obtained by performing speech recognition on the spectral features of the speech to be recognized based on a third speech recognition model, and the third speech recognition model is trained based on the spectral features of domain sample speech and its label recognition text.

[0154] Based on any of the above embodiments, the text determination subunit is used to:

[0155] When the number of characters per unit time length of the reference recognition text is less than or equal to a character threshold, the recognition text is determined based on the recognition scores of the first candidate texts and the recognition scores of the second candidate texts.

[0156] Based on any of the above embodiments, the text determination subunit is used to:

[0157] When the number of characters per unit time length of the reference recognition text is greater than a character threshold, the recognition text is determined based on the recognition scores of the first candidate texts.

[0158] Based on any of the above embodiments, the device further includes:

[0159] a fit determination unit, configured to perform speech recognition on the speech to be recognized based on a fourth speech recognition model, obtain a recognition probability of each first candidate text being a recognition result of the fourth speech recognition model, and determine the recognition probability as the acoustic fit;

[0160] The fourth speech recognition model is trained based on the spectral features of the domain sample speech and its label recognition text, and the structure of the fourth speech recognition model is different from that of the second speech recognition model;

[0161] a fluency determination unit, configured to perform speech recognition on the speech to be recognized based on a fifth speech recognition model, obtain a recognition probability of each first candidate text being a recognition result of the fifth speech recognition model, and use the recognition probability as the speech fluency;

[0162] The fifth speech recognition model is trained based on the spectrum features of common sample speech and its label recognition text. The structure of the fifth speech recognition model is different from that of the fourth speech recognition model.

[0163] Based on any of the above embodiments, the speech recognition unit 540 includes:

[0164] a fusion unit, configured to perform weighted fusion on the first acoustic state posterior probability and the second acoustic state posterior probability based on a preset weight to obtain the fused posterior probability;

[0165] A decoding unit is used to perform speech recognition decoding based on the fused posterior probability to obtain a recognition text of the speech to be recognized.

[0166] Figure 6 Schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 6As shown, the electronic device may include: a processor 610, a memory 620, a communication interface 630, and a communication bus 640, wherein the processor 610, the memory 620, and the communication interface 630 communicate with each other via the communication bus 640. The processor 610 may call the logic instructions in the memory 620 to execute a speech recognition method, which includes: determining a speech to be recognized; performing acoustic state prediction on the spectral features of the speech to be recognized based on a first speech recognition model to obtain a first acoustic state posterior probability of the speech to be recognized; performing acoustic state prediction on the semantic features of the speech to be recognized based on a second speech recognition model to obtain a second acoustic state posterior probability of the speech to be recognized; fusing the first acoustic state posterior probability and the second acoustic state posterior probability, and performing speech recognition decoding based on the fused posterior probability to obtain a recognized text of the speech to be recognized; the first speech recognition model is trained based on the spectral features of domain sample speech and its label recognition text, and the second speech recognition model is trained based on the semantic features of the domain sample speech and its label recognition text.

[0167] In addition, the logic instructions in the above-mentioned memory 620 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0168] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the speech recognition method provided by the above methods, which method includes: determining a speech to be recognized; based on a first speech recognition model, performing acoustic state prediction on the spectral features of the speech to be recognized to obtain a first acoustic state posterior probability of the speech to be recognized; based on a second speech recognition model, performing acoustic state prediction on the semantic features of the speech to be recognized to obtain a second acoustic state posterior probability of the speech to be recognized; fusing the first acoustic state posterior probability and the second acoustic state posterior probability, and performing speech recognition decoding based on the fused posterior probability to obtain the recognized text of the speech to be recognized; the first speech recognition model is trained based on the spectral features of domain sample speech and its label recognition text, and the second speech recognition model is trained based on the semantic features of the domain sample speech and its label recognition text.

[0169] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the above-mentioned speech recognition methods, the methods comprising: determining a speech to be recognized; based on a first speech recognition model, performing acoustic state prediction on the spectral features of the speech to be recognized to obtain a first acoustic state posterior probability of the speech to be recognized; based on a second speech recognition model, performing acoustic state prediction on the semantic features of the speech to be recognized to obtain a second acoustic state posterior probability of the speech to be recognized; fusing the first acoustic state posterior probability and the second acoustic state posterior probability, performing speech recognition decoding based on the fused posterior probability, and obtaining a recognized text of the speech to be recognized; the first speech recognition model is trained based on the spectral features of domain sample speech and its label recognition text, and the second speech recognition model is trained based on the semantic features of the domain sample speech and its label recognition text.

[0170] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0171] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0172] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A speech recognition method, characterized in that: include: Determine the speech to be recognized; Based on the first speech recognition model, performing acoustic state prediction on the spectral features of the speech to be recognized to obtain a first acoustic state posterior probability of the speech to be recognized; the spectral features of the speech to be recognized are used to represent syllable information of each word in the speech to be recognized, and the first acoustic state posterior probability is used to represent the importance of the spectral features for speech recognition; Based on the second speech recognition model, performing acoustic state prediction on the semantic features of the speech to be recognized to obtain a second acoustic state posterior probability of the speech to be recognized; the semantic features of the speech to be recognized are used to represent contextual semantic information between each word in the speech to be recognized, and the second acoustic state posterior probability is used to represent the importance of the semantic features for speech recognition; fusing the first acoustic state posterior probability and the second acoustic state posterior probability, performing speech recognition decoding based on the fused posterior probability obtained by fusion, and obtaining a recognition text of the speech to be recognized; The first speech recognition model is trained based on the spectral features of domain sample speech and its label recognition text, and the second speech recognition model is trained based on the semantic features of the domain sample speech and its label recognition text.

2. The speech recognition method according to claim 1, wherein: The performing speech recognition decoding based on the fusion posterior probability obtained by fusion to obtain the recognition text of the speech to be recognized includes: Perform speech recognition decoding based on the fused posterior probability to obtain at least two first candidate texts; Determining a recognition score for each first candidate text based on the acoustic fit of each first candidate text with the speech to be recognized and / or the language fluency of each first candidate text; Based on the recognition scores of the first candidate texts, the recognition text of the speech to be recognized is determined.

3. The speech recognition method according to claim 2, wherein: The step of determining the recognized text of the speech to be recognized based on the recognition scores of the first candidate texts includes: Performing speech recognition on the spectral features of the speech to be recognized based on a third speech recognition model to obtain at least two second candidate texts; Determining a reference recognition text based on the recognition score of each second candidate text; Determining the number of characters per unit duration of the reference recognition text based on the number of characters in the reference recognition text and the voice duration of the voice to be recognized; Determining the recognized text based on the number of characters per unit time length of the reference recognized text and the recognition score of each first candidate text; The second candidate texts are obtained by performing speech recognition on the spectral features of the speech to be recognized based on a third speech recognition model, and the third speech recognition model is trained based on the spectral features of domain sample speech and its label recognition text.

4. The speech recognition method according to claim 3, wherein: The determining the recognized text based on the number of characters per unit time length of the reference recognized text and the recognition score of each first candidate text includes: When the number of characters per unit time length of the reference recognition text is less than or equal to a character threshold, the recognition text is determined based on the recognition scores of the first candidate texts and the recognition scores of the second candidate texts.

5. The speech recognition method according to claim 3, wherein: The determining the recognized text based on the number of characters per unit time length of the reference recognized text and the recognition score of each first candidate text includes: When the number of characters per unit time length of the reference recognition text is greater than a character threshold, the recognition text is determined based on the recognition scores of the first candidate texts.

6. The speech recognition method according to claim 2, wherein: The step of determining the acoustic fit between each of the first candidate texts and the speech to be recognized includes: performing speech recognition on the to-be-recognized speech based on a fourth speech recognition model, obtaining a recognition probability of each first candidate text being a recognition result of the fourth speech recognition model, and determining the recognition probability as the acoustic fit; The fourth speech recognition model is trained based on the spectral features of the domain sample speech and its label recognition text, and the structure of the fourth speech recognition model is different from that of the second speech recognition model; The step of determining the language fluency of each first candidate text comprises: performing speech recognition on the speech to be recognized based on the fifth speech recognition model, obtaining a recognition probability of each first candidate text being a recognition result of the fifth speech recognition model, and using the recognition probability as the language fluency; The fifth speech recognition model is trained based on the spectrum features of common sample speech and its label recognition text. The structure of the fifth speech recognition model is different from that of the fourth speech recognition model.

7. The speech recognition method according to any one of claims 1 to 6, characterized in that: The fusing of the first acoustic state posterior probability and the second acoustic state posterior probability, and performing speech recognition decoding based on the fused posterior probability to obtain the recognition text of the speech to be recognized, includes: Based on a preset weight, performing weighted fusion on the first acoustic state posterior probability and the second acoustic state posterior probability to obtain the fused posterior probability; Speech recognition decoding is performed based on the fused posterior probability to obtain a recognition text of the speech to be recognized.

8. A speech recognition device, characterized in that: include: A speech determination unit, configured to determine a speech to be recognized; a first prediction unit, configured to perform acoustic state prediction on the spectral features of the speech to be recognized based on a first speech recognition model to obtain a first acoustic state posterior probability of the speech to be recognized; the spectral features of the speech to be recognized are used to represent syllable information of each word in the speech to be recognized, and the first acoustic state posterior probability is used to represent the importance of the spectral features for speech recognition; a second prediction unit, configured to perform acoustic state prediction on the semantic features of the speech to be recognized based on a second speech recognition model to obtain a second acoustic state posterior probability of the speech to be recognized; the semantic features of the speech to be recognized are used to represent contextual semantic information between each word in the speech to be recognized, and the second acoustic state posterior probability is used to represent the importance of the semantic features for speech recognition; a speech recognition unit, configured to fuse the first acoustic state posterior probability and the second acoustic state posterior probability, perform speech recognition decoding based on the fused posterior probability, and obtain a recognition text of the speech to be recognized; The first speech recognition model is trained based on the spectral features of domain sample speech and its label recognition text, and the second speech recognition model is trained based on the semantic features of the domain sample speech and its label recognition text.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the speech recognition method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the speech recognition method according to any one of claims 1 to 7 is implemented.

Images