Speech recognition text scoring method and apparatus, electronic device, and storage medium

Abstract

The application provides a speech recognition text scoring method and device, electronic equipment and storage medium. The speech recognition text scoring method obtains standard audio features of target speech based on a speech coding process in a speech recognition process of the target speech from the perspective of speech recognition, thereby providing a reference standard for scoring of the recognized text. From the perspective of scoring of the recognized text, audio distribution features corresponding to the first recognized text are obtained based on the first recognized text of the target speech. The first recognized text is scored in combination with the audio distribution features and the standard audio features. The method considers the correlation between the speech recognition text and the audio during scoring of the recognized text, thereby achieving scoring of the speech recognition text.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of speech recognition, and in particular to a speech recognition scoring method and device, an electronic device, and a storage medium. BACKGROUND

[0002] In recent years, with the development of deep learning technology and the maturity of tools, speech recognition technology has been provided with strong technical and development support, and has made significant progress. However, due to the limited semantic understanding ability of the speech recognition network, for example, the understanding ability of homophonic characters is poor. Therefore, after obtaining multiple candidate texts of audio through speech recognition technology, it is usually necessary to re-score the multiple candidate texts to re-evaluate the multiple candidate texts and obtain the optimal speech recognition text.

[0003] Therefore, how to score the candidate text of speech recognition has become a technical problem that needs to be solved by those skilled in the art. SUMMARY

[0004] The present application provides a speech recognition text scoring method, device, electronic device, and storage medium to score the candidate text of speech recognition.

[0005] According to a first aspect of an embodiment of the present application, a speech recognition text scoring method is provided, comprising:

[0006] obtaining a standard audio feature of a target speech and a first recognition text corresponding to the target speech; the standard audio feature of the target speech is obtained based on encoding of the target speech;

[0007] determining an audio distribution feature corresponding to the first recognition text according to the first recognition text of the target speech;

[0008] obtaining a score of the first recognition text according to at least the audio distribution feature corresponding to the first recognition text and the standard audio feature of the target speech.

[0009] In an optional embodiment of the present application, the obtaining of the standard audio feature of the target speech and the first recognition text corresponding to the target speech comprises:

[0010] obtaining an encoding result of the target speech in a speech recognition process of the target speech, taking the encoding result of the target speech as the standard audio feature of the target speech;

[0011] and obtaining a recognition result of the target speech, the recognition result of the target speech including the first recognition text.

[0012] In an optional implementation of the present application, in the process of obtaining the speech recognition result of the target speech, the encoding result of the target speech is taken as the standard audio feature of the target speech; and the first recognition text is included in the recognition result of the target speech, which includes:

[0013] obtaining the spectral feature of the target speech;

[0014] inputting the spectral feature of the target speech into a pre-trained speech recognition model, performing feature encoding processing on the spectral feature through an encoder of the speech recognition model to obtain the standard audio feature of the target speech;

[0015] and performing decoding processing on the standard audio feature through a decoder of the speech recognition model to obtain a plurality of recognition texts of the target speech;

[0016] taking any recognition text in the plurality of recognition texts as the first recognition text.

[0017] In an optional implementation of the present application, the audio distribution feature corresponding to the first recognition text of the target speech is determined according to the first recognition text, which includes:

[0018] extracting the text feature of the first recognition text;

[0019] performing mapping processing on the text feature of the first recognition text by using a pre-trained first text scoring model to obtain the audio distribution feature corresponding to the first recognition text.

[0020] In an optional implementation of the present application, the text feature of the first recognition text is extracted, which includes:

[0021] performing speech frame level text feature encoding processing on the first recognition text to obtain the speech frame level text feature of the first recognition text.

[0022] In an optional implementation of the present application, the pre-trained first text scoring model is trained by the following method:

[0023] obtaining a sample speech with a standard recognition text and a sample audio feature of the sample speech;

[0024] performing speech recognition processing on the sample speech by using a pre-trained speech recognition model to obtain a plurality of candidate recognition texts of the sample speech;

[0025] extracting a standard text feature of the standard recognized text and a candidate text feature of each candidate recognized text;

[0026] mapping the standard text feature and the candidate text features respectively by using a first text scoring model constructed in advance to obtain a standard audio distribution feature corresponding to the standard recognized text and a candidate audio distribution feature corresponding to each candidate recognized text;

[0027] determining a standard likelihood value of the standard recognized text audio distribution feature according to the standard audio distribution feature and the sample audio feature, and determining a candidate likelihood value of each candidate recognized text audio distribution feature according to the candidate audio distribution feature and the sample audio feature;

[0028] optimizing the first text scoring model constructed in advance based on the standard likelihood value and the candidate likelihood values.

[0029] In an optional embodiment of the present application, the obtaining of the score of the first recognized text according to at least the audio distribution feature corresponding to the first recognized text and the standard audio feature of the target voice includes:

[0030] determining a first likelihood value of the first recognized text audio distribution feature according to the audio feature distribution corresponding to the first recognized text and the standard audio feature of the target voice;

[0031] obtaining the score of the first recognized text according to at least the first likelihood value.

[0032] In an optional embodiment of the present application, the obtaining of the score of the first recognized text according to at least the first likelihood value includes:

[0033] obtaining a first basic score of the first recognized text of the target voice; wherein the first basic score is obtained based on a recognition result of the target voice, and the recognition result of the target voice includes the first recognized text and the first basic score of the first recognized text;

[0034] obtaining the score of the first recognized text based on the first basic score and the first likelihood value, and combining a weight coefficient of the first basic score and the first likelihood value.

[0035] In an optional embodiment of the present application, the obtaining of the score of the first recognized text according to at least the first likelihood value includes:

[0036] performing text difference-based scoring processing on the first recognized text by using a second text scoring model constructed in advance to obtain a second basic score of the first recognized text.

[0037] On the basis of the second basic score and the first likelihood value, a weight coefficient of the second basic score and the first likelihood value is combined to obtain a score of the first recognized text.

[0038] In an optional embodiment of the present application, the obtaining of the score of the first recognized text according to the first likelihood value comprises:

[0039] obtaining a first basic score of the first recognized text of the target speech; wherein the first basic score is obtained based on a recognition result of the target speech, and the recognition result of the target speech comprises the first recognized text and the first basic score of the first recognized text;

[0040] performing text difference-based score processing on the first recognized text by using a pre-constructed second text scoring model to obtain a second basic score of the first recognized text;

[0041] On the basis of the first basic score, the second basic score and the first likelihood value, a weight coefficient of the first basic score, the second basic score and the first likelihood value is combined to obtain the first basic score.

[0042] In an optional embodiment of the present application, the second text scoring model is obtained by training in the following manner:

[0043] obtaining a sample standard text of a sample speech and a sample candidate text obtained by performing speech recognition on the sample speech;

[0044] obtaining a sample basic score of the sample candidate text according to a text difference between the sample standard text and the sample candidate text;

[0045] training the second text scoring model by taking the sample basic score and the sample standard text as training samples.

[0046] According to a second aspect of the embodiments of the present application, a speech recognized text scoring device is provided, comprising:

[0047] a first unit configured to obtain a standard audio feature of a target speech and a first recognized text corresponding to the target speech; the standard audio feature of the target speech is obtained based on encoding of the target speech;

[0048] a second unit configured to determine an audio distribution feature corresponding to the first recognized text according to the first recognized text of the target speech;

[0049] The third unit is configured to obtain a score of the first recognized text according to at least the audio distribution feature corresponding to the first recognized text and the standard audio feature of the target speech.

[0050] According to a third aspect of the embodiments of the present application, an electronic device is provided, which comprises:

[0051] a processor;

[0052] a memory configured to store instructions executable by the processor;

[0053] the processor is configured to execute the above-mentioned method for scoring a recognized speech text by running the instructions in the memory.

[0054] According to a fourth aspect of the embodiments of the present application, a computer storage medium is provided, which stores a computer program, and the computer program is configured to execute the above-mentioned method for scoring a recognized speech text when executed by a processor.

[0055] Compared with the prior art, the present application has the following advantages:

[0056] The present application provides a method and device for scoring a recognized speech text, an electronic device and a storage medium. The method for scoring a recognized speech text comprises: obtaining a standard audio feature of a target speech and a first recognized text corresponding to the target speech; the standard audio feature of the target speech is obtained based on a speech encoding process of a speech recognition process of the target speech; determining an audio distribution feature corresponding to the first recognized text according to the first recognized text of the target speech; and obtaining a score of the first recognized text according to at least the audio distribution feature corresponding to the first recognized text and the standard audio feature of the target speech.

[0057] The method obtains the standard audio feature of the target speech based on the speech encoding process of the speech recognition process of the target speech from the perspective of speech recognition, thereby providing a reference standard for scoring the recognized text; the audio distribution feature corresponding to the first recognized text is obtained based on the first recognized text of the target speech from the perspective of scoring the recognized text; and the first recognized text is scored by combining the audio distribution feature and the standard audio feature, which considers the correlation between the recognized speech text and the audio in the process of scoring the recognized text, thereby realizing the scoring of the recognized speech text. BRIEF DESCRIPTION OF DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the accompanying drawings in the following description only only the embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0059] FIG. 1 The application scenario diagram of the scoring method of the speech recognition text provided by an embodiment of the present application is shown in the figure.

[0060] FIG. 2 The flowchart of the scoring method of the speech recognition text provided by another embodiment of the present application is shown in the figure.

[0061] FIG. 3 The structural diagram of the scoring device of the speech recognition text provided by another embodiment of the present application is shown in the figure.

[0062] FIG. 4 The structural diagram of the electronic device provided by another embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the present application will be described clearly and completely in the following description of the embodiments of the present application with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0064] In recent years, with the development of deep learning technology and the maturity of tools, speech recognition technology has been greatly supported by powerful technology and development, and has made significant progress. However, due to the limited semantic understanding ability of speech recognition network, for example, the understanding ability of homophonic characters is poor. Therefore, after obtaining multiple candidate texts of audio through speech recognition technology, it is usually necessary to re-score the multiple candidate texts to re-evaluate the multiple candidate texts and obtain the optimal speech recognition text.

[0065] Therefore, how to score the candidate text of speech recognition has become a technical problem that those skilled in the art need to solve.

[0066] To solve the above technical problems, the present application provides a scoring method and device for speech recognition text, electronic equipment and storage medium, which are described in detail one by one in the following embodiments.

[0067] Example implementation environment

[0068] First, in order to facilitate the understanding of the scoring method, device, electronic equipment and storage medium of the speech recognition text provided by the present application, the present application embodiment first introduces the method in combination with the specific application scenario of the scoring method of the speech recognition text.

[0069] In the scenario embodiment of the present application, the scoring method of the speech recognition text is applied to scoring the speech recognition text of the conference speaker in the conference scenario.

[0070] Please refer to FIG. 1 , FIG. 1 The application scenario diagram of the scoring method of the speech recognition text provided by an embodiment of the present application.

[0071] As FIG. 1 shown, FIG. 1 , the method comprises a speech recognition model 101 and a first text scoring model 102.

[0072] In the present embodiment, the speech recognition model 101 is used to input the target speech to be recognized, and the standard audio features h of the target speech are obtained by performing feature coding processing on the target speech through the encoder of the speech recognition model;

[0073] and the plurality of recognition texts (y1 to y t ) of the target speech are obtained by performing decoding processing on the standard audio features h through the decoder of the speech recognition model.

[0074] Among them, the target speech is the speech of the conference speaker in the conference scenario.

[0075] Then, each recognition text of the plurality of recognition texts is scored.

[0076] The process of scoring each recognition text is realized based on the first text scoring model 102. Specifically, in the actual application process, first, the text features of the recognition text to be scored are extracted; then, the text features are input into the first text scoring model 102 to map the text features through the forward full connection layer of the first text scoring model, and the audio distribution features (Gaussian weight, mean, variance) of the recognition text to be scored are obtained.

[0077] Finally, the audio distribution features and the standard audio features h are combined to obtain the score of the recognition text to be scored.

[0078] It can be understood that the above introduction of the scene embodiments of the present application is only for better understanding of the speech recognition text scoring method provided by the present application, and is not used to limit the application scene of the speech recognition text scoring method. The speech recognition text scoring method can also be applied to other scenes, such as being used for recognizing conversation voice, recognizing recording information, etc.

[0079] Example method

[0080] The embodiments of the present application also provide a speech recognition text scoring method, the core of which is that, from the perspective of speech recognition, based on the speech coding process of the speech recognition process of the target voice, the standard audio features of the target voice are obtained, which provides a reference standard for scoring the recognized text; from the perspective of scoring the recognized text, based on the first recognized text of the target voice, the audio distribution features corresponding to the first recognized text are obtained; and the first recognized text is scored in combination with the audio distribution features and the standard audio features. In the process of scoring the recognized text, the association between the speech recognition text and the audio is considered, and the scoring of the speech recognition text is realized.

[0081] In an optional embodiment of the present application, the implementation subject of the speech recognition text scoring method can be a notebook computer, a tablet computer, a desktop computer, a set-top box, a mobile device (for example, a mobile phone, a portable music player, a personal digital assistant, a dedicated message device, a game console, a recording pen), various types of user terminals or a combination of any two or more of these data processing devices, or a server.

[0082] Please refer to FIG. 2 , FIG. 2 The speech recognition text scoring method flowchart provided by another embodiment of the present application.

[0083] As FIG. 2 shown, the speech recognition text scoring method includes the following steps S201 to S203:

[0084] Step S201, obtaining the standard audio features of the target voice and the first recognized text corresponding to the target voice; the standard audio features of the target voice are obtained based on the coding of the target voice.

[0085] The target voice can be understood as voice data that needs to be recognized. In an optional embodiment of the present application, the target voice can be a piece of audio data obtained through the Internet, can also be a piece of voice of a conference speaker obtained in a conference scene, or can also be a voice of a game player delivered through the built-in voice system of the game in a game scene. The present application does not make any limitation in this regard.

[0086] In an optional embodiment of the present application, the encoding process of the target speech can be implemented based on a machine encoding manner, for example, the target speech is encoded by an encoder to obtain the standard audio features; for another example, the target speech can be encoded by an acoustic model to obtain the standard audio features; in another optional embodiment of the present application, the encoding process of the target speech can be implemented based on a manual encoding method, for example, the encoding of the target speech can be implemented based on a certain encoding rule by a manual encoding manner to obtain the standard audio features.

[0087] In another optional embodiment of the present application, the standard audio features of the target speech and the first recognized text corresponding to the target speech can be obtained based on a speech recognition process of the target speech.

[0088] Specifically, the obtaining of the standard audio features of the target speech and the first recognized text corresponding to the target speech comprises:

[0089] In the process of the speech recognition of the target speech, the encoding result of the target speech is taken as the standard audio features of the target speech.

[0090] In addition, the recognition result of the target speech is obtained, and the first recognized text is included in the recognition result of the target speech.

[0091] In an optional embodiment of the present application, the process of the speech recognition of the target speech is implemented based on a pre-trained speech recognition model.

[0092] The speech recognition model can be understood as a neural network, and in the process of specific application, the personalized transcription model can be trained by a machine learning (Machine Learning, ML) manner. Machine learning (a multi-field cross discipline, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines) is specially used for studying to obtain new knowledge or skills through training samples, reorganize existing knowledge structure and constantly improve the performance of itself. Machine learning usually includes artificial neural network, belief network, reinforcement learning, transfer learning, inductive learning and other technologies, and belongs to a branch of artificial intelligence (Artificial Intellingence, AI) technology.

[0093] In an optional embodiment of the present application, the speech recognition model comprises a speech encoder and a feature decoder.

[0094] In the process of specific application, the target speech can be encoded by the speech encoder to obtain standard audio features of the target speech. Then, the standard audio features are decoded by the feature decoder to obtain a plurality of candidate recognition texts of the target speech and basic scores of each candidate recognition text. The first recognition text mentioned in step S201 can be any one of the candidate recognition texts.

[0095] In another optional embodiment of the present application, the speech recognition model comprises an acoustic feature extraction model and a text extraction model.

[0096] In the process of specific application, the acoustic feature extraction model can be used to extract phoneme features from the target audio, and the extracted phoneme features are used as the standard audio features. Then, the text extraction model is used to decode and recognize the standard audio features to obtain a plurality of candidate recognition texts of the target speech and basic scores of each candidate recognition text. The first recognition text mentioned in step S201 can be any one of the candidate recognition texts.

[0097] In an optional embodiment of the present application, the standard audio features can be the output of the speech encoder or the acoustic feature extraction model or the output of a certain layer, which is not limited in the present application.

[0098] In an optional embodiment of the present application, in order to better realize speech recognition of the target speech, the spectral features of the target speech are extracted first. Then, the spectral features of the target speech are segmented by a voice activity detection (VAD) model. Finally, the segmented spectral features are input into a pre-trained speech recognition model to obtain the standard audio features and the first recognition text of the target speech.

[0099] Further, in order to make the speech recognition model more accurate in recognizing the target speech, in an optional embodiment of the present application, the speech recognition model can be trained in the following way:

[0100] First, a sample speech labeled with a correct recognition text and a basic score of the correct recognition text are obtained.

[0101] Second, the sample recognition text and the basic score are used as training samples to train a pre-constructed speech recognition model.

[0102] Further, the speech text scoring method provided by the application aims to score the candidate recognition texts of the target speech to obtain a text that is closest to or completely identical to the meaning expressed by the target speech from the candidate recognition texts.

[0103] In the prior art, the scoring of the candidate recognition texts depends on the basic score of the target speech by the aforementioned speech recognition model, or is directly based on a language model to score the candidate recognition texts, i.e., scoring the candidate recognition texts from the text perspective.

[0104] Considering that the prior art does not take into account the relationship between the candidate recognition texts and the target speech, the scoring task of the candidate texts is not stable enough. Therefore, the following steps S202 and S203 are further performed.

[0105] In step S202, audio feature distribution corresponding to the first recognition text of the target speech is determined according to the first recognition text of the target speech.

[0106] The purpose of determining the audio feature distribution corresponding to the first recognition text according to the first recognition text of the target speech is to obtain the result from the process of rescorring the first recognition text, which is different from the process of obtaining the basic score of the first recognition text based on the speech recognition model mentioned in step S201, and is based on a text scoring model, which is referred to as a first text scoring model in the embodiment of the application.

[0107] Specifically, determining the audio feature distribution corresponding to the first recognition text according to the first recognition text of the target speech includes:

[0108] extracting the text features of the first recognition text;

[0109] mapping the text features of the first recognition text by using a pre-trained first text scoring model to obtain the audio feature distribution corresponding to the first recognition text.

[0110] In an optional embodiment of the application, the first text scoring model can be a mature structure such as a Transformer structure or a flow model (Flow). The process of obtaining the audio feature distribution is described in detail below by taking a Gaussian mixture model composed of a Transformer structure as the first text scoring model as an example.

[0111] After obtaining the first recognition text of the target speech, the first recognition text is first text encoded to extract the text features of the first recognition text.

[0112] In an optional embodiment of the present application, when the text features of the first recognized text are extracted, in order to obtain the text features of the first recognized text in a fine-grained manner and facilitate the recognition process of the target speech, the text features of the first recognized text are encoded at the speech frame level corresponding to each speech frame of the target speech, and the text features of the first recognized text at the speech frame level are obtained.

[0113] Then, the text features of each first recognized text are input into the first text scoring model, the text features are mapped to each Gaussian model of the Gaussian mixture model through the forward full connection layer of the text scoring model, the feature distribution parameters of each Gaussian model are obtained, and the feature distribution parameters are taken as the audio feature distribution.

[0114] In the embodiments of the present application, the feature distribution parameters of each Gaussian model are respectively the weight ω, the mean μ and the variance σ of the Gaussian distribution 2 .

[0115] Further, in an optional embodiment of the present application, the first text scoring model is obtained by the following steps S1 to S6:

[0116] Step S1, obtaining a sample speech with a standard recognized text and a sample audio feature of the sample speech.

[0117] In the embodiments of the present application, the standard recognized text of the sample speech can be understood as the correct text of the sample speech, for example, assuming that the sample speech is the audio data of “today the weather is really good”, then the standard recognized text should be the text data of “today the weather is really good”. In order to facilitate the description of the following text, the standard recognized text of the sample speech is set to The sample audio feature of the sample speech is set to h.

[0118] Step S2, performing speech recognition processing on the sample speech by using a pre-trained speech recognition model to obtain a plurality of candidate recognized texts of the sample speech.

[0119] The plurality of candidate recognized texts of the sample speech can be understood as a plurality of candidate recognized texts obtained by recognizing the sample speech based on the pre-trained speech recognition model.

[0120] In the embodiments of the present application, in order to facilitate the description of the following text, the plurality of candidate recognized texts are set to y i .

[0121] Step S3, extracting standard text features of the standard recognized text and candidate text features of each candidate recognized text.

[0122] Step S4: Using the pre-built first text scoring model, the standard text features and each candidate text feature are mapped to obtain the standard audio distribution features corresponding to the standard recognized text and the candidate audio distribution features corresponding to the candidate recognized text.

[0123] In this embodiment of the application, the standard audio distribution features corresponding to the standard recognized text and the candidate audio distribution features corresponding to the candidate recognized text mentioned in step S4 above are used.

[0124] Step S5: Determine the standard likelihood value of the standard recognized text audio distribution feature based on the standard audio distribution feature and the sample audio feature; and determine the candidate likelihood value of each candidate recognized text audio distribution feature based on each candidate audio distribution feature and the sample audio feature.

[0125] Specifically, the candidate likelihood values ​​of each candidate recognized text audio feature can be obtained using the following formula (1):

[0126]

[0127] Where L(h|y) is a candidate likelihood value for a candidate recognition text audio feature distribution; w i μ i , These represent the Gaussian weights, mean, and variance of the i-th Gaussian model obtained based on the candidate recognized text, respectively; K represents the number of mixtures in the Gaussian mixture model; h t This represents the sample audio features of the t-th frame of the sample speech.

[0128] Furthermore, the standard likelihood value of the standard audio feature distribution corresponding to the standard recognized text can be obtained by the following formula (2):

[0129]

[0130] in, The standard likelihood value for identifying the audio distribution features of the text; w i μ i , These represent the Gaussian weights, mean, and variance corresponding to the i-th Gaussian model obtained based on the standard recognized text, respectively; K represents the number of mixtures in the Gaussian mixture model; h t This represents the sample audio features of the t-th frame of the sample speech.

[0131] Step S6: Optimize the pre-built first text scoring model based on the standard likelihood value and each candidate likelihood value.

[0132] After the standard likelihood value and each candidate likelihood value are obtained, a loss function of the first text scoring model can be established according to the standard likelihood value and each candidate likelihood value. Specifically, the loss function can be represented by the following formula (3):

[0133]

[0134] wherein L represents the loss value of the first text scoring model, and a represents the adjustable hyperparameter of the first text scoring model. is the standard likelihood value of the standard recognized text audio distribution feature; L(h|y i represents the candidate likelihood value of the i th candidate recognized text audio distribution feature; and M represents the total number of the candidate recognized texts.

[0135] In the loss function represented by the above formula (3) to represent the discriminability of the first text scoring model to the error part in the candidate recognized text. Based on the above loss function, only by continuously increasing the likelihood value of the correct sample and reducing the likelihood value of the error sample, the loss value of the above loss function can be continuously reduced. In the process of optimizing the first scoring model, the first text scoring model can be optimized by a gradient descent algorithm until the loss value of the loss function is reduced to a preset loss threshold, and an optimized first text scoring model is obtained.

[0136] In an optional embodiment of the present application, to reduce the calculation amount of the loss function, the same candidate recognized text as the standard recognized text can be eliminated from the candidate recognized text. Then, the candidate text after eliminating the same candidate recognized text is used to calculate

[0137] In step S203, a score of the first recognized text is obtained at least according to the audio distribution feature corresponding to the first recognized text and the standard audio feature of the target voice.

[0138] In an optional embodiment of the present application, the score of the first recognized text is obtained at least according to the audio distribution feature corresponding to the first recognized text and the standard audio feature of the target voice, including:

[0139] A first likelihood value of the first recognized text audio distribution feature is determined according to the audio feature distribution corresponding to the first recognized text and the standard audio feature of the target voice.

[0140] A score of the first recognized text is obtained at least according to the first likelihood value.

[0141] In the embodiments of the present application, the first audio feature distribution corresponding to the first recognized text is the Gaussian weight ω, the mean μ and the variance σ of each Gaussian model mentioned in step S202 2 .

[0142] Further, if the standard audio feature of the target voice is h t , the first likelihood value of the first recognized text audio feature distribution, then the first likelihood value can be obtained by the above formula (1).

[0143] In an optional embodiment of the present application, the first likelihood value can be used as the score of the first recognized text.

[0144] In another optional embodiment of the present application, since the recognition result obtained by recognizing the target voice through the pre-trained speech recognition model includes the first recognized text and the basic score corresponding to the first recognized text, the score of the first recognized text can also be obtained by combining the first likelihood value and the basic score. Here, in order to facilitate the description below, the basic score of the first recognized text obtained by the speech recognition model is set as the first basic score.

[0145] Here, assuming that the first likelihood value is s, and the first basic score of the first recognized text is s asr , then the score S of the first recognized text can be obtained by the following formula (4):

[0146] S=k*s+(1-k)*s asr (4);

[0147] Wherein, k and (1-k) are the weight coefficients of the first likelihood value and the first basic score, respectively.

[0148] That is, obtaining the first basic score of the first recognized text of the target voice; wherein the first basic score is obtained based on the recognition result of the target voice, and the recognition result of the target voice includes the first recognized text and the first basic score of the first recognized text; using a pre-constructed second text scoring model to perform text difference-based scoring processing on the first recognized text to obtain a second basic score of the first recognized text; and obtaining the first basic score based on the first basic score, the second basic score and the first likelihood value, and combining the weight coefficients of the first basic score, the second basic score and the first likelihood value.

[0149] In another optional embodiment of the present application, the at least according to the first likelihood value, obtaining the score of the first recognized text, comprises:

[0150] The second text scoring model is pre-constructed, and text difference-based scoring processing is performed on the first recognized text to obtain a second basic score of the first recognized text.

[0151] On the basis of the second basic score and the first likelihood value, a weight coefficient of the second basic score and the first likelihood value is combined to obtain a score of the first recognized text.

[0152] That is, after the first likelihood value of the first recognized text is obtained according to the audio distribution feature corresponding to the first recognized text and the standard audio feature of the target voice, the existing technology is combined to score the recognized text from the text angle, and the score with the text scoring dimension and the text-audio scoring dimension is obtained.

[0153] For example, assuming that the first likelihood value is s, and the second basic score is s', the score S of the first recognized text can be obtained by the following formula (5):

[0154] S = k * s + (1-k) * s' (5);

[0155] Wherein, k and (1-k) are weight coefficients of the first likelihood value and the second basic score, respectively.

[0156] In the embodiments of the present application, the second text scoring model can be obtained by the following method:

[0157] A sample standard text of a sample voice and a sample candidate text obtained by performing voice recognition on the sample voice are obtained, a sample basic score of the sample candidate text is obtained according to the text difference between the sample standard text and the sample candidate text, and the sample basic score and the sample standard text are used as training samples to train the second text scoring model.

[0158] In another optional embodiment of the present application, the score of the first recognized text can also be constructed by combining the first likelihood value, the first basic score and the second basic score, so as to realize the scoring of the first recognized text from multiple different dimensions.

[0159] Specifically, assuming that the first likelihood value is s, the first basic score is s asr , and the second basic score is s', the score S of the first recognized text can be obtained by the following formula (6):

[0160] S = k1 * s + k2 * s asr +(1-k1-k2) * s' (6);

[0161] Wherein, k1, k2, (1-k1-k2) are weight coefficients of the first likelihood value, the first base score, and the second base score respectively.

[0162] In summary, the scoring method of the speech recognition text provided in the present application obtains the standard audio features of the target speech based on the speech coding process of the speech recognition process of the target speech from the perspective of speech recognition, providing a reference standard for scoring the recognized text; from the perspective of scoring the recognized text, the audio distribution features corresponding to the first recognized text of the target speech are obtained based on the first recognized text of the target speech; and the first recognized text is scored in combination with the audio distribution features and the standard audio features. In the process of scoring the recognized text, the correlation between the speech recognition text and the audio is considered, and the scoring of the speech recognition text is realized.

[0163] Example apparatus

[0164] Correspondingly, the present application also provides a scoring device for speech recognition text, please refer to FIG. 3 , FIG. 3 The scoring device for speech recognition text provided in another embodiment of the present application is shown in the structural schematic diagram.

[0165] As FIG. 3 shown, the scoring device for speech recognition text comprises:

[0166] The first unit 301 is configured to obtain the standard audio features of the target speech and the first recognized text corresponding to the target speech; and the standard audio features of the target speech are obtained based on the coding of the target speech.

[0167] The second unit 302 is configured to determine the audio distribution features corresponding to the first recognized text of the target speech according to the first recognized text of the target speech.

[0168] The third unit 303 is configured to obtain the score of the first recognized text according to at least the audio distribution features corresponding to the first recognized text and the standard audio features of the target speech.

[0169] In an optional embodiment of the present application, the obtaining of the standard audio features of the target speech and the first recognized text corresponding to the target speech comprises:

[0170] obtaining the coding result of the target speech in the process of speech recognition processing of the target speech, taking the coding result of the target speech as the standard audio features of the target speech;

[0171] and obtaining the recognition result of the target speech, wherein the first recognized text is included in the recognition result of the target speech.

[0172] In an optional implementation of the present application, in the process of obtaining the speech recognition result of the target speech, the encoding result of the target speech is taken as the standard audio feature of the target speech; and the first recognition text is included in the recognition result of the target speech, which includes:

[0173] obtaining the spectral feature of the target speech;

[0174] inputting the spectral feature of the target speech into a pre-trained speech recognition model, performing feature encoding processing on the spectral feature through an encoder of the speech recognition model to obtain the standard audio feature of the target speech;

[0175] and performing decoding processing on the standard audio feature through a decoder of the speech recognition model to obtain a plurality of recognition texts of the target speech;

[0176] taking any one of the plurality of recognition texts as the first recognition text.

[0177] In an optional implementation of the present application, the audio distribution feature corresponding to the first recognition text of the target speech is determined according to the first recognition text, which includes:

[0178] extracting the text feature of the first recognition text;

[0179] mapping the text feature of the first recognition text using a pre-trained first text scoring model to obtain the audio distribution feature corresponding to the first recognition text.

[0180] In an optional implementation of the present application, the text feature of the first recognition text is extracted, which includes:

[0181] performing speech frame level text feature encoding processing on the first recognition text corresponding to each speech frame of the target speech to obtain the speech frame level text feature of the first recognition text.

[0182] In an optional implementation of the present application, the pre-trained first text scoring model is trained in the following manner:

[0183] obtaining a sample speech with a standard recognition text and a sample audio feature of the sample speech;

[0184] performing speech recognition processing on the sample speech using a pre-trained speech recognition model to obtain a plurality of candidate recognition texts of the sample speech;

[0185] extracting a standard text feature of the standard recognized text and a candidate text feature of each candidate recognized text;

[0186] mapping the standard text feature and the candidate text features respectively by using a pre-constructed first text scoring model to obtain a standard audio distribution feature corresponding to the standard recognized text and a candidate audio distribution feature corresponding to each candidate recognized text;

[0187] determining a standard likelihood value of the standard recognized text audio distribution feature according to the standard audio distribution feature and the sample audio feature, and determining a candidate likelihood value of each candidate recognized text audio distribution feature according to the candidate audio distribution feature and the sample audio feature;

[0188] optimizing the pre-constructed first text scoring model based on the standard likelihood value and the candidate likelihood values.

[0189] In an optional embodiment of the present application, the obtaining of the score of the first recognized text according to at least the audio distribution feature corresponding to the first recognized text and the standard audio feature of the target voice includes:

[0190] determining a first likelihood value of the first recognized text audio distribution feature according to the audio feature distribution corresponding to the first recognized text and the standard audio feature of the target voice;

[0191] obtaining the score of the first recognized text according to at least the first likelihood value.

[0192] In an optional embodiment of the present application, the obtaining of the score of the first recognized text according to at least the first likelihood value includes:

[0193] obtaining a first basic score of the first recognized text of the target voice; wherein the first basic score is obtained based on a recognition result of the target voice, and the recognition result of the target voice includes the first recognized text and the first basic score of the first recognized text;

[0194] obtaining the score of the first recognized text based on the first basic score and the first likelihood value, and combining a weight coefficient of the first basic score and the first likelihood value.

[0195] In an optional embodiment of the present application, the obtaining of the score of the first recognized text according to at least the first likelihood value includes:

[0196] performing text difference-based scoring processing on the first recognized text by using a pre-constructed second text scoring model to obtain a second basic score of the first recognized text.

[0197] On the basis of the second basic score and the first likelihood value, a score of the first recognized text is obtained by combining weight coefficients of the second basic score and the first likelihood value.

[0198] In an optional embodiment of the present application, the obtaining of the score of the first recognized text according to the first likelihood value comprises:

[0199] obtaining a first basic score of the first recognized text of the target speech; wherein the first basic score is obtained on the basis of a recognition result of the target speech, and the recognition result of the target speech comprises the first recognized text and the first basic score of the first recognized text;

[0200] performing text difference-based score processing on the first recognized text by using a pre-constructed second text scoring model to obtain a second basic score of the first recognized text;

[0201] On the basis of the first basic score, the second basic score and the first likelihood value, a score of the first basic score is obtained by combining weight coefficients of the first basic score, the second basic score and the first likelihood value.

[0202] In an optional embodiment of the present application, the second text scoring model is obtained by training in the following manner:

[0203] obtaining a sample standard text of a sample speech and a sample candidate text obtained by performing speech recognition on the sample speech;

[0204] obtaining a sample basic score of the sample candidate text according to a text difference between the sample standard text and the sample candidate text;

[0205] training the second text scoring model by taking the sample basic score and the sample standard text as training samples.

[0206] The speech recognition text scoring device provided in the embodiment belongs to the same application concept as the speech recognition text scoring method provided in the above-mentioned embodiments of the present application, can execute the speech recognition text scoring method provided in any of the above-mentioned embodiments of the present application, and has the corresponding function modules and beneficial effects of executing the speech recognition text scoring method. Technical details not described in detail in the embodiment can be referred to the specific processing content of the speech recognition text scoring method provided in the above-mentioned embodiments of the present application, which will not be described here again.

[0207] Example electronic device

[0208] Another embodiment of the present application further provides an electronic device, which is described as follows.FIG. 4 , FIG. 4 An electronic device structure diagram is provided for another embodiment of the present application.

[0209] As shown in FIG. 4 , the electronic device includes:

[0210] a memory 200 and a processor 210;

[0211] The memory 200 is connected with the processor 210, and is configured to store a program.

[0212] The processor 210 is configured to realize the scoring method of the voice recognition text disclosed in any of the above embodiments by running the program stored in the memory 200.

[0213] Specifically, the electronic device can further include a bus, a communication interface 220, an input device 230 and an output device 240.

[0214] The processor 210, the memory 200, the communication interface 220, the input device 230 and the output device 240 are connected with each other through the bus. Among them:

[0215] The bus can include a path for transmitting information between various components of the computer system.

[0216] The processor 210 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or can be an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of programs of the present application. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a ready-to-use programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0217] The processor 210 can include a main processor, and can also include a baseband chip, a modem, etc.

[0218] The memory 200 stores programs for executing the technical solutions of the present application, and can also store operating systems and other key services. Specifically, the program can include program code, and the program code includes computer operation instructions. More specifically, the memory 200 can include read-only memory (ROM), other types of static storage devices that can store static information and instructions, random access memory (RAM), other types of dynamic storage devices that can store information and instructions, disk storage, flash, etc.

[0219] The input device 230 can include a device that receives data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, a gravity sensor, or the like.

[0220] The output device 240 can include a device that allows information to be output to a user, such as a display screen, a printer, a speaker, or the like.

[0221] The communication interface 220 can include a device using any transceiver to communicate with other devices or communication networks, such as an Ethernet, a radio access network (RAN), a wireless local area network (WLAN), or the like.

[0222] The processor 210 executes programs stored in the memory 200 and calls other devices, which can be used to implement each step of any of the scoring methods of speech recognition text provided by the embodiments described above.

[0223] Example computer program product and storage medium

[0224] In addition to the above methods and devices, the embodiments of the present application can also be a computer program product, which includes computer program instructions that, when executed by a processor, cause the processor to perform the steps of the scoring method of speech recognition text according to various embodiments of the present application described in the above "Exemplary Methods" section of the specification.

[0225] The computer program product can be written in any combination of one or more programming languages to perform the operations of the embodiments of the present application, including an object-oriented programming language, such as Java, C++, and the like, and a conventional procedural programming language, such as the "C" language or a similar programming language. The program code can be executed entirely on a user computing device, partially on a user device, as a separate software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0226] In addition, the embodiments of the present application can also be a storage medium having a computer program stored thereon, which is executed by a processor to perform the steps of the scoring method of speech recognition text according to various embodiments of the present application described in the above "Exemplary Methods" section of the specification, which can implement the following steps:

[0227] Step S201, obtaining a standard audio feature of a target speech and a first recognition text corresponding to the target speech; the standard audio feature of the target speech is obtained based on encoding of the target speech;

[0228] In step S202, an audio distribution feature corresponding to the first recognized text of the target speech is determined according to the first recognized text of the target speech.

[0229] In step S203, a score of the first recognized text is obtained according to at least the audio distribution feature corresponding to the first recognized text and a standard audio feature of the target speech.

[0230] For each method embodiment described above, in order to simply describe, a series of actions are described, but those skilled in the art should know that the application is not limited to the order of actions described, because according to the application, some steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the application.

[0231] It should be noted that each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same and similar parts between each embodiment can be referred to. For the device embodiment, since it is basically similar to the method embodiment, it is described relatively simply, and the relevant parts can be referred to the part of the method embodiment.

[0232] The steps in the method of each embodiment of the application can be adjusted, combined and reduced in sequence according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0233] The modules and sub-modules in the device and terminal of each embodiment of the application can be combined, divided and reduced according to actual needs.

[0234] In several embodiments provided by the application, it should be understood that the disclosed terminal, device and method can be implemented by other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or sub-modules is only a logical function division, and there can be another division way in actual implementation, for example, a plurality of sub-modules or modules can be combined or integrated into another module, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0235] The modules or sub-modules described as separate components may or may not be physically separate, and the components of the modules or sub-modules may or may not be physical modules or sub-modules, i.e., may be located in one place or distributed over multiple network modules or sub-modules. Some or all of the modules or sub-modules can be selected as needed to achieve the purposes of the embodiments.

[0236] In addition, each functional module or sub-module in the various embodiments of the present application can be integrated into one processing module, or each module or sub-module can exist physically alone, or two or more modules or sub-modules can be integrated into one module. The integrated module or sub-module can be realized in the form of hardware or in the form of a software functional module or sub-module.

[0237] The skilled person will further appreciate that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described above in general terms generally in accordance with their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Skilled persons can use different methods to implement the described functions for each particular application, but such implementation should not be considered beyond the scope of the present application.

[0238] The steps of the methods or algorithms described in connection with the embodiments disclosed herein can be directly implemented in hardware, software units executed by a processor, or a combination of both. The software units can be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0239] Finally, it should be noted that, in this document, the terms such as first and second are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles or devices including a series of elements not only include those elements, but also include other elements not explicitly listed or inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article or device including the element.

[0240] The foregoing description of the disclosed embodiments enables a person skilled in the art to make or use the application. Modifications of these embodiments will occur to those skilled in the art and are within the scope of the application as defined by the appended claims, the general principles defined herein can be applied to other embodiments without departing from the spirit or essential characteristics of the application. Thus, the present application is not to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method of scoring speech recognition text, characterized by, The method comprises: obtaining standard audio features of a target speech, and a first recognized text corresponding to the target speech; the standard audio features of the target speech are obtained based on encoding of the target speech; determining audio distribution features corresponding to the first recognized text according to the first recognized text of the target speech, wherein the audio distribution features comprise Gaussian weights, mean values, and variances; obtaining a score of the first recognized text according to at least the audio distribution features corresponding to the first recognized text and the standard audio features of the target speech; wherein the score of the first recognized text is determined based on at least a first likelihood value between the audio distribution features and the standard audio features.

2. The method of claim 1, wherein, The method comprises: obtaining encoding results of the target speech in a speech recognition process of the target speech; and taking the encoding results of the target speech as the standard audio features of the target speech; and obtaining a recognition result of the target speech, wherein the first recognized text is included in the recognition result of the target speech.

3. The method of claim 2, wherein, The method comprises: obtaining encoding results of the target speech in a speech recognition process of the target speech; and taking the encoding results of the target speech as the standard audio features of the target speech; and obtaining a recognition result of the target speech, wherein the first recognized text is included in the recognition result of the target speech. The method comprises: obtaining spectral features of the target speech; inputting the spectral features of the target speech into a pre-trained speech recognition model, performing feature encoding processing on the spectral features by an encoder of the speech recognition model to obtain the standard audio features of the target speech; 4. The method of claim 1, wherein, and performing decoding processing on the standard audio features by a decoder of the speech recognition model to obtain a plurality of recognized texts of the target speech; taking any recognized text in the plurality of recognized texts as the first recognized text. The method comprises:

5. The method of claim 4, wherein, extracting text features of the first recognized text; performing mapping processing on the text features of the first recognized text by using a pre-trained first text scoring model to obtain audio distribution features corresponding to the first recognized text.

6. The method of claim 4, wherein, The method comprises: performing speech frame level text feature encoding processing on the first recognized text to obtain speech frame level text features of the first recognized text, corresponding to each speech frame of the target speech. The pre-trained first text scoring model is trained by the following method: obtaining sample speech with a standard recognized text, and sample audio features of the sample speech; performing speech recognition processing on the sample speech by using a pre-trained speech recognition model to obtain a plurality of candidate recognized texts of the sample speech; extracting standard text features of the standard recognized text, and candidate text features of each candidate recognized text; The first text scoring model is pre-constructed, and the standard text features and the candidate text features are respectively mapped by using the first text scoring model to obtain standard audio distribution features corresponding to the standard recognized text and candidate audio distribution features corresponding to the candidate recognized text. According to the standard audio distribution features and the sample audio features, a standard likelihood value of the standard recognized text audio distribution features is determined, and according to the candidate audio distribution features and the sample audio features, candidate likelihood values of the candidate recognized text audio distribution features are determined. The first text scoring model is pre-constructed, and the standard text features and the candidate text features are respectively mapped by using the first text scoring model to obtain standard audio distribution features corresponding to the standard recognized text and candidate audio distribution features corresponding to the candidate recognized text.

7. The method of claim 1, wherein, The first text scoring model is pre-constructed, and the standard text features and the candidate text features are respectively mapped by using the first text scoring model to obtain standard audio distribution features corresponding to the standard recognized text and candidate audio distribution features corresponding to the candidate recognized text. The first text scoring model is pre-constructed, and the standard text features and the candidate text features are respectively mapped by using the first text scoring model to obtain standard audio distribution features corresponding to the standard recognized text and candidate audio distribution features corresponding to the candidate recognized text. The first text scoring model is pre-constructed, and the standard text features and the candidate text features are respectively mapped by using the first text scoring model to obtain standard audio distribution features corresponding to the standard recognized text and candidate audio distribution features corresponding to the candidate recognized text.

8. The method of claim 1, wherein, The first text scoring model is pre-constructed, and the standard text features and the candidate text features are respectively mapped by using the first text scoring model to obtain standard audio distribution features corresponding to the standard recognized text and candidate audio distribution features corresponding to the candidate recognized text. The second text scoring model is trained in the following manner: Sample standard text of sample voice and sample candidate text obtained by performing voice recognition on the sample voice are obtained.

9. The method of claim 1, wherein, Sample base scores of the sample candidate text are obtained according to text differences between the sample standard text and the sample candidate text. The sample base scores and the sample standard text are used as training samples to train the second text scoring model. The second text scoring model is trained in the following manner: ​ 10. The method according to claim 8 or 9, characterized in that, ​ ​ ​ ​ 11. A speech recognition text scoring apparatus characterized by comprising: ​ The first unit is configured to obtain a standard audio feature of a target speech and a first recognized text corresponding to the target speech; The standard audio feature of the target speech is obtained based on encoding of the target speech; The second unit is configured to determine an audio distribution feature corresponding to the first recognized text according to the first recognized text of the target speech, wherein the audio distribution feature comprises a Gaussian weight, a mean value and a variance; The third unit is configured to obtain a score of the first recognized text according to at least the audio distribution feature corresponding to the first recognized text and the standard audio feature of the target speech, wherein the score of the first recognized text is determined based on at least a first likelihood value between the audio distribution feature and the standard audio feature.

12. An electronic device, comprising: The method comprises: a processor; a memory for storing instructions executable by the processor; the processor is configured to execute the method of scoring a recognized text of a speech according to any one of claims 1-10 by running the instructions in the memory.

13. A computer storage medium, characterized in that The storage medium stores a computer program, and the computer program is executed by the processor to perform the method of scoring a recognized text of a speech according to any one of claims 1-10.

Images