Voice recognition method, device and electronic equipment

Abstract

The present application discloses a speech recognition method, device and electronic device, the method comprising: obtaining a speech signal to be recognized; determining a candidate text recognition result of the speech signal based on a continuous time series classification model, the candidate text recognition result comprising at least one candidate character sequence corresponding to the speech signal, the candidate character sequence comprising at least one character, the character belonging to one of an entity character and a blank placeholder; determining the placeholder distribution characteristics of the blank placeholders associated with each entity character in the candidate character sequence; determining the probability of occurrence of the placeholder distribution characteristics corresponding to each entity character in the candidate character sequence based on the possible distribution characteristics of the blank placeholders associated with different entity characters and the probability of occurrence of the possible distribution characteristics; determining the text recognition result based on at least one candidate character sequence and the probability of occurrence of the placeholder distribution characteristics corresponding to each entity character in the candidate character sequence. The scheme of the present application can improve the accuracy of speech recognition.

Description

Technical Field

[0001] The present application relates to the field of speech processing technology, and more specifically, to a speech recognition method, device and electronic device. Background Art

[0002] Traditional acoustic model training for speech recognition requires knowing the corresponding label for each frame of data in order to conduct effective training. Before training the data, speech alignment preprocessing is required. The speech alignment process itself requires repeated iterations to ensure more accurate alignment, which is a relatively time-consuming task.

[0003] Compared with traditional acoustic model training, acoustic model training using connectionist temporal classification (CTC) as the loss function is a completely end-to-end acoustic model training that does not require pre-alignment of data. Only one input sequence and one output sequence are needed to complete the training of the CTC model, which has led to an increasing number of applications for speech recognition based on CTC. However, currently, language recognition based on CTC is prone to missed detection, which may affect the accuracy of speech recognition. Summary of the invention

[0004] The present application provides a speech recognition method, device and electronic device.

[0005] A speech recognition method includes:

[0006] Obtaining a speech signal to be recognized;

[0007] Determine a candidate text recognition result of the speech signal based on a continuous time series classification model, wherein the candidate text recognition result includes at least one candidate character sequence corresponding to the speech signal, wherein the candidate character sequence includes at least one character, and the character is one of an entity character and a blank placeholder;

[0008] Determining a placeholder distribution feature of a blank placeholder associated with each entity character in the candidate character sequence;

[0009] Determine the occurrence probability of the placeholder distribution feature corresponding to each entity character in the candidate character sequence based on the possible distribution features of the blank placeholders associated with different entity characters and the occurrence probability of the possible distribution features;

[0010] A text recognition result is determined based on the at least one candidate character sequence and the occurrence probability of the placeholder distribution feature corresponding to each entity character in the candidate character sequence.

[0011] In a possible implementation, the possible distribution features of the configured different entity characters associated with the blank placeholders and the occurrence probabilities of the possible distribution features are obtained by:

[0012] obtaining a plurality of speech signal samples;

[0013] Based on the trained continuous time series classification model, determining at least one character sequence sample corresponding to the speech signal sample, the character sequence sample includes at least one character sample, and the character sample belongs to one of an entity character and a blank placeholder, and the trained continuous time series classification model is obtained by training based on the multiple speech signal samples;

[0014] Determining distribution characteristics of blank placeholders associated with each entity character in the character sequence sample;

[0015] Based on the distribution characteristics of the blank placeholders associated with each entity character in the character sequence sample, possible distribution characteristics of the blank placeholders associated with different entity characters and the occurrence probabilities of the possible distribution characteristics are determined.

[0016] In another possible implementation, determining the text recognition result based on the at least one candidate character sequence and the occurrence probability of the placeholder distribution feature corresponding to each entity character in the candidate character sequence includes:

[0017] Determining a first sequence occurrence probability of the candidate character sequence based on the occurrence probability of the placeholder distribution feature corresponding to each entity character in the candidate character sequence;

[0018] In combination with the first sequence occurrence probability of the candidate character sequence, a text recognition result corresponding to the at least one candidate character sequence is determined.

[0019] In yet another possible implementation, the candidate text recognition result further includes: a second sequence occurrence probability of the candidate character sequence;

[0020] Determining a text recognition result corresponding to the at least one candidate character sequence in combination with the first sequence occurrence probability of the candidate character sequence includes:

[0021] The text recognition result corresponding to the at least one candidate character sequence is determined by combining the first sequence occurrence probability and the second sequence occurrence probability of the candidate character sequence.

[0022] In another possible implementation, combining the first sequence occurrence probability and the second sequence occurrence probability of the candidate character sequence to determine the text recognition result corresponding to the at least one candidate character sequence includes:

[0023] In combination with the first sequence occurrence probability of the candidate character sequence, the second sequence occurrence probability of the candidate character sequence is corrected to obtain a third sequence occurrence probability of the candidate character sequence;

[0024] Based on the third sequence occurrence probability of the candidate character sequence, a text recognition result corresponding to the at least one candidate character sequence is determined.

[0025] In yet another possible implementation, determining a text recognition result corresponding to the at least one candidate character sequence based on the third sequence occurrence probability of the candidate character sequence includes:

[0026] sorting the at least one candidate character sequence according to the third sequence occurrence probability of the candidate character sequence to obtain a text recognition result, wherein the text recognition result includes the sorting of the at least one candidate character sequence;

[0027] or,

[0028] The candidate character sequence with the highest occurrence probability of the third sequence in the at least one candidate character sequence is determined as the text recognition result.

[0029] In yet another possible implementation, the blank placeholder associated with the entity character includes: a blank placeholder that is located after the entity character, is adjacent to the entity character, and appears continuously.

[0030] In yet another possible implementation, determining the candidate text recognition result of the speech signal based on the continuous time series classification model includes:

[0031] Extracting audio features of the speech signal;

[0032] Determining a vector encoding of the speech signal based on the audio feature;

[0033] Based on the continuous time series classification model and the vector encoding, a candidate text recognition result of the speech signal is determined.

[0034] A speech recognition device includes:

[0035] A speech acquisition unit, used for acquiring a speech signal to be recognized;

[0036] A speech recognition unit, configured to determine a candidate text recognition result of the speech signal based on a continuous time series classification model, wherein the candidate text recognition result includes at least one candidate character sequence corresponding to the speech signal, wherein the candidate character sequence includes at least one character, and the character is one of an entity character and a blank placeholder;

[0037] a distribution determination unit, configured to determine a placeholder distribution feature of a blank placeholder associated with each entity character in the candidate character sequence;

[0038] A probability determination unit, configured to determine the occurrence probability of the placeholder distribution feature corresponding to each entity character in the candidate character sequence based on the possible distribution features of the blank placeholders associated with different entity characters and the occurrence probability of the possible distribution features;

[0039] The result determination unit is used to determine the text recognition result based on the at least one candidate character sequence and the occurrence probability of the placeholder distribution feature corresponding to each entity character in the candidate character sequence.

[0040] Among them, an electronic device includes at least a processor and a memory;

[0041] Wherein, the processor is used to execute the speech recognition method as described in any one of the above;

[0042] The memory is used to store programs required by the processor to perform operations.

[0043] It can be known from the above scheme that in the present application, after at least one candidate character sequence corresponding to the speech signal is identified based on the continuous time series classification model, the placeholder distribution characteristics of the blank placeholders associated with each entity character in the candidate character sequence will be determined, and based on the possible distribution characteristics of the blank placeholders associated with different entity characters and the probability of occurrence of the possible distribution characteristics, the probability of occurrence of the placeholder distribution characteristics corresponding to each entity character in the candidate character sequence is determined. The probability of occurrence of the placeholder distribution characteristics corresponding to the entity characters in the candidate character sequence can be reflected by the probability of occurrence of the placeholder distribution characteristics corresponding to the entity characters in the candidate character sequence. The lower the probability of occurrence of the placeholder occurrence characteristics, the higher the possibility of error in the placeholder distribution characteristics corresponding to the entity characters. Therefore, combining the at least one candidate character sequence identified by the CTC model and the probability of occurrence of the placeholder distribution characteristics corresponding to each entity character in each candidate character sequence can be beneficial to correcting the candidate text recognition results, thereby obtaining more accurate text recognition results, and thus improving the accuracy of speech recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1A flow chart of a speech recognition method provided in an embodiment of the present application;

[0046] Figure 2 A schematic diagram of an implementation flow of determining possible distribution characteristics and occurrence probabilities of blank placeholders associated with entity characters provided in an embodiment of the present application;

[0047] Figure 3 A further flow chart of a speech recognition method provided in an embodiment of the present application;

[0048] Figure 4 A further flow chart of a speech recognition method provided in an embodiment of the present application;

[0049] Figure 5 A schematic diagram of a framework of an implementation principle of an embodiment of the present application;

[0050] Figure 6 A schematic diagram of the structure of a speech recognition device provided in an embodiment of the present application;

[0051] Figure 7 A schematic diagram of the composition architecture of an electronic device provided in an embodiment of the present application.

[0052] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims and the above drawings are used to distinguish similar parts, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated herein. DETAILED DESCRIPTION

[0053] The solution of the present application is applicable to any scenario where the text corresponding to a speech signal needs to be recognized. The solution of the present application can improve the accuracy of the text results recognized from the speech signal.

[0054] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0055] like Figure 1 As shown, it shows a flow chart of an embodiment of the speech recognition method provided by the embodiment of the present application. The method of this embodiment can be applied to any electronic device that requires speech recognition. The method of this embodiment may include:

[0056] S101, obtaining a speech signal to be recognized.

[0057] The speech signal is a speech signal that needs to be subjected to text recognition.

[0058] S102, determining a candidate text recognition result of the speech signal based on a continuous time series classification model.

[0059] Among them, by recognizing the speech signal based on the continuous time series classification model, a text recognition result can be obtained. In order to distinguish it from the text recognition result finally determined later, the text recognition result determined based on the continuous time series classification model is made into a candidate text recognition result.

[0060] It is understandable that at least one text result can be obtained from the candidate text recognition results of the speech signal obtained based on the CTC model, and each text result is a candidate character sequence. That is, the candidate text recognition result includes at least one candidate character sequence corresponding to the speech signal.

[0061] It is understandable that a new placeholder is introduced in the Connectionist temporal classification (CTC) algorithm to output the alignment result. This placeholder is called blank placeholder. Correspondingly, in the process of CTC model recognizing speech signals, if the valid output of a unit cannot be determined, a blank placeholder will be output. When the output of a unit can be determined, a valid symbol will be output. The valid symbol refers to the entity character other than the blank placeholder blank.

[0062] It can be seen that each candidate character sequence obtained by the CTC model includes at least one character, and each character is one of an entity character and a blank placeholder.

[0063] For example, in CTC-based speech recognition, φ can be used to represent a blank placeholder, and the candidate text recognition results obtained by the CTC model can include the following two candidate character sequences:

[0064] φφd dφeφeφp p

[0065] Candidate character sequence 1:

[0066] Candidate character sequence 2: φφd dφe e eφp p.

[0067] Among them, in the above candidate character sequence, all characters except φ are entity characters, for example, characters "d", "e" and "p" are all entity characters.

[0068] Of course, this is just an example, and there may be other possible ways to represent blank placeholders. Moreover, for different speech signals, the number of candidate character sequences and the specific content form in the text recognition results obtained by the CTC model will be different.

[0069] In this embodiment, the continuous time series classification model can be trained using multiple speech signal samples, for example, trained in a supervised manner based on multiple speech signal samples annotated with text labels, wherein the text label of the speech signal sample is the text actually corresponding to the speech signal sample.

[0070] Of course, there are many possibilities for the specific implementation of training a continuous time series classification model based on multiple speech signal samples, and this application does not impose any restrictions on this.

[0071] S103: Determine a placeholder distribution feature of a blank placeholder associated with each entity character in the candidate character sequence.

[0072] The blank placeholder associated with the entity character in the candidate character sequence may include: the blank placeholder that is adjacent to and appears continuously with the entity character in the candidate character sequence.

[0073] In an optional manner, taking into account the pronunciation of the entity characters in the speech signal and the length of the pause after the entity characters are uttered during the speech recognition process of the CTT model, the situation mainly affects the blank placeholder after the entity characters. Therefore, in the present application, the blank placeholder associated with the entity characters can be: a blank placeholder that is located after the entity character, adjacent to the entity character and appears continuously.

[0074] The placeholder distribution characteristics of the blank placeholders associated with the entity characters reflect the specific situation of the blank placeholders associated with the entity characters, for example, the number of blank placeholders associated with the entity placeholders.

[0075] For example, the placeholder distribution feature of the blank placeholders associated with the entity character may be the number of blank placeholders that appear immediately and continuously after the entity character.

[0076] Example: Still taking the candidate character sequence 1 in step S102 as an example, in the candidate character sequence 1, there is no blank placeholder after the first entity character "d" from left to right. Therefore, the blank placeholder distribution feature corresponding to the first entity character "d" is: the number of blank placeholders immediately and continuously after "d" is zero. Similarly, the blank placeholder distribution feature of the second entity character "d" is: there is 1 blank placeholder symbol immediately after "d".

[0077] S104, determining the occurrence probability of the placeholder distribution feature corresponding to each entity character in the candidate character sequence based on the possible distribution features of the blank placeholders associated with different entity characters and the occurrence probability of the possible distribution features.

[0078] Among them, the possible distribution characteristics of blank placeholders associated with entity characters are also the possible distribution characteristics of blank placeholders associated with entity characters, specifically referring to the possible placeholder distribution characteristics of blank placeholders associated with entity characters, and the meaning of the placeholder distribution characteristics is the same as described above.

[0079] For example, each possible distribution feature of blank placeholders associated with an entity character may represent the number of blank placeholders that appear after the entity character and are immediately adjacent to the entity character.

[0080] There may be multiple possible distribution characteristics of blank placeholders associated with each entity character.

[0081] The occurrence probability of the possible distribution feature of the blank placeholder associated with the entity character refers to the probability that the blank placeholder associated with the entity character is the possible distribution feature. Different possible distribution features of the blank placeholder associated with the entity character have different corresponding occurrence probabilities.

[0082] For example, for the entity character "s", assuming that there are three situations in which blank placeholders appear immediately and continuously after the entity character "s": the first is that there is no blank placeholder; the second is that there are two consecutive blank placeholders; the third is that there are three consecutive blank placeholders. Among them, the probability of the first situation is 10%, the probability of the second situation is 70%, and the probability of the third situation is 20%. Then it can be configured that the probability of 0 blank placeholders appearing immediately and continuously after "s" is 10%; the probability of 2 blank placeholders appearing immediately and continuously after "s" is 70%; the probability of 3 blank placeholders appearing immediately and continuously after "s" is 20%.

[0083] Among them, the possible distribution characteristics of blank placeholders associated with different entity characters and the corresponding occurrence probabilities of the possible distribution characteristics can be comprehensively determined by combining the character sequence texts corresponding to multiple speech signal samples and the character sequence corresponding to the speech signal samples recognized by the CTC model, without any restriction here.

[0084] It is understandable that, by combining the possible distribution characteristics and occurrence probabilities of blank placeholders associated with different configured entity characters, the occurrence probability corresponding to the blank placeholder distribution characteristics corresponding to a certain entity character in the candidate character sequence can be queried and determined.

[0085] For example, taking the possible distribution characteristics of the three possible blank placeholders of the entity character "s" above as an example, assuming that there is an entity character "s" in the candidate character sequence, and it is determined that the blank placeholder distribution characteristic of the entity character "s" in the candidate character sequence is: "s" is associated with 3 consecutive blank placeholders, then it can be determined that the probability of occurrence of this blank placeholder distribution characteristic is 20%.

[0086] S105, determining a text recognition result based on at least one candidate character sequence and the occurrence probability of the placeholder distribution feature corresponding to each entity character in the candidate character sequence.

[0087] It can be understood that the higher the probability of occurrence of the placeholder distribution feature of the blank placeholder associated with each entity character in the candidate character sequence, the more accurate the distribution feature of the blank placeholder associated with the entity character predicted by the CTC model.

[0088] It can be seen that the probability of occurrence of the placeholder distribution features corresponding to each entity character in the candidate character sequence can reflect the accuracy of the candidate character sequence predicted by the CTC model. On this basis, combined with the probability of occurrence of the placeholder distribution features corresponding to each entity character in the candidate character sequence, it is beneficial to screen or sort at least one candidate character sequence obtained by the CTC model, and realize the correction of the candidate text recognition results obtained by CTC, which can also reduce the accuracy of the text recognition results affected by missed detection and other reasons.

[0089] The text recognition result determined in step S105 may be: a ranking of the at least one candidate character sequence, and the ranking may reflect the accuracy of the candidate character sequence predicted by the CTC.

[0090] The determined text recognition result may also be one or more candidate character sequences with higher accuracy selected from the at least one candidate character sequence.

[0091] For example, in a possible implementation, the probability of occurrence of the first sequence of the candidate character sequence can be determined based on the probability of occurrence of the placeholder distribution features corresponding to each entity character in the candidate character sequence. Accordingly, the text recognition result corresponding to at least one candidate character sequence can be determined in combination with the probability of occurrence of the first sequence of the candidate character sequence. For example, the ranking of at least one candidate character sequence is determined based on the probability of occurrence of the first sequence, or the first at least one candidate character sequence with a higher probability of occurrence of the first sequence is selected from at least one candidate character sequence.

[0092] From the above content, it can be seen that after the present application identifies at least one candidate character sequence corresponding to the speech signal based on the continuous time series classification model, it will determine the placeholder distribution characteristics of the blank placeholders associated with each entity character in the candidate character sequence, and determine the probability of occurrence of the placeholder distribution characteristics corresponding to each entity character in the candidate character sequence based on the possible distribution characteristics of the blank placeholders associated with different entity characters and the probability of occurrence of the possible distribution characteristics. The probability of occurrence of the placeholder distribution characteristics corresponding to the entity characters in the candidate character sequence can be reflected by the probability of occurrence of the placeholder distribution characteristics corresponding to the entity characters in the candidate character sequence. The lower the probability of occurrence of the placeholder occurrence characteristics, the higher the possibility of error in the placeholder distribution characteristics corresponding to the entity characters. Therefore, combining the at least one candidate character sequence identified by the CTC model and the probability of occurrence of the placeholder distribution characteristics corresponding to each entity character in each candidate character sequence can be beneficial to correcting the candidate text recognition results, thereby obtaining more accurate text recognition results, and thus improving the accuracy of speech recognition.

[0093] It is understandable that the possible distribution characteristics of the blank placeholders associated with different entity characters pre-configured in the present application and the occurrence probability of the possible distribution characteristics can be obtained by statistics in a variety of ways.

[0094] In order to make the occurrence probability of possible distribution characteristics of blank placeholders associated with entity characters more accurate, the present application determines the possible distribution characteristics and corresponding occurrence probabilities of blank placeholders associated with various entity placeholders based on the CTC model and the speech signal samples used to train the CTC model before performing speech recognition based on the trained CTC model.

[0095] like Figure 2 As shown, it shows a schematic diagram of an implementation flow of determining possible distribution characteristics and occurrence probabilities of blank placeholders associated with entity characters provided in an embodiment of the present application. This embodiment may include the following steps:

[0096] S201, obtaining a plurality of speech signal samples.

[0097] The speech signal sample is a speech signal used to train the continuous time series classification model. For the sake of distinction, the speech signal used as a training sample is referred to as a speech signal sample.

[0098] S202: Determine at least one character sequence sample corresponding to the speech signal sample based on the trained continuous time series classification model.

[0099] The character sequence sample includes at least one character sample, and the character sample is one of an entity character and a blank placeholder.

[0100] It is understandable that for each speech signal sample, at least one character sequence can be determined based on the continuous time series classification model. The character sequence identified by the continuous time series classification model for the speech signal sample is called a character sequence sample, and the characters in the character sequence sample are called character samples.

[0101] Among them, the process of determining the character sequence samples of the speech signal is similar to the previous process of determining the candidate character sequence of the speech signal. For details, please refer to the relevant introduction of other embodiments of the present application, which will not be repeated here.

[0102] For example, the speech features of the speech signal sample may be determined first, and then the speech features may be input into the continuous time series classification model to obtain at least one character sequence sample output by the continuous time series classification model for the speech signal sample.

[0103] The continuous time series classification model is obtained by training based on multiple speech signal samples. The specific training process can refer to the above related introduction.

[0104] In order to improve the accuracy of the subsequent determination of various possible distribution features of the blank placeholders corresponding to the entity characters, the continuous time series classification model here is the same as the continuous time series classification model used for text recognition of the speech signal to be recognized in this application.

[0105] It can be understood that in practical applications, the continuous time series classification model in step S202 can be pre-trained. Of course, it is also possible to first train the continuous time series classification model based on multiple speech signal samples in step S202, and then determine the character sequence samples corresponding to each speech signal sample based on the trained continuous time series classification model.

[0106] S203: Determine the distribution characteristics of the blank placeholders associated with each entity character in the character sequence sample.

[0107] In this embodiment, the blank placeholder associated with the entity character may have the same meaning as in the previous embodiment. For example, the blank placeholder associated with the entity character in the character sequence may be a blank placeholder that is located after the entity character, adjacent to the entity character, and appears continuously in the character sequence sample.

[0108] Correspondingly, the distribution characteristics of the blank placeholders associated with the entity characters have the same meaning as the previous blank placeholder distribution characteristics. They are referred to as distribution characteristics here only to distinguish the distribution characteristics of the blank placeholders associated with the statistical entity characters from the placeholder distribution characteristics of the blank placeholders involved in the previous recognized speech signal.

[0109] For example, the distribution feature of the blank placeholders associated with the entity characters is the number of the blank placeholders associated with the entity characters.

[0110] S204, based on the distribution characteristics of the blank placeholders associated with each entity character in the character sequence sample, determine possible distribution characteristics of the blank placeholders associated with different entity characters and the occurrence probability of the possible distribution characteristics.

[0111] For example, for each entity character, various distribution characteristics of the blank placeholder associated with the entity character in each character sequence sample corresponding to multiple speech signal samples and the probability corresponding to each distribution characteristic can be counted, so as to obtain the possible distribution characteristics of the blank placeholder associated with the entity character and its occurrence probability.

[0112] For example, assuming that by analyzing the character sequence samples corresponding to multiple speech signal samples, it can be obtained that the distribution characteristics of the blank placeholders associated with the entity character "a" may include two types: "a" is immediately followed by one continuous blank placeholder; and "a" is immediately followed by two continuous blank placeholders. Based on this, the first occurrence number of "a" immediately followed by one continuous blank placeholder and the second occurrence number of "a" immediately followed by two continuous blank placeholders can be counted, and then, the probability of "a" being immediately followed by one continuous blank placeholder and the probability of "a" being immediately followed by two continuous blank placeholders can be determined by combining the first occurrence number and the second occurrence number.

[0113] It is understandable that in practical applications, the possible distribution characteristics and occurrence probability of the blank placeholders associated with each entity character can be represented by a bar graph or a Gaussian distribution. Of course, there may be other implementation methods, which are not limited thereto.

[0114] It should be noted that Figure 2 This is only one implementation method for determining the possible distribution characteristics and occurrence probability of blank placeholders associated with entity characters. However, determining the possible distribution characteristics and occurrence probability of blank placeholders associated with entity characters by other methods is also applicable to the present application and is not limited to this.

[0115] It is understandable that there are many possible specific ways to determine the text recognition result by combining the occurrence probability of the placeholder distribution feature corresponding to each entity character in the candidate character sequence.

[0116] For ease of understanding, a possible implementation method is described as follows: first determining the first sequence occurrence probability of the candidate character sequence, and then determining the text recognition result based on the first sequence occurrence probability. Figure 3 As shown, it shows another flow chart of the speech recognition method provided by the embodiment of the present application. The method of this embodiment may include:

[0117] S301: Obtain a speech signal to be recognized.

[0118] The speech signal is a speech signal that needs to be subjected to text recognition.

[0119] S302: for each speech signal, determine a candidate text recognition result of the speech signal based on the CTC model.

[0120] The candidate text recognition result includes at least one candidate character sequence and the predicted sequence occurrence probability of the candidate character sequence. The predicted sequence occurrence probability of the candidate character sequence is the possible probability that the text corresponding to the speech signal predicted by the CTC model is the candidate character sequence.

[0121] In the present application, in order to distinguish from the first sequence occurrence probability of the candidate character sequence determined based on the occurrence probability of the placeholder distribution feature in the previous embodiment, the predicted sequence occurrence probability of the candidate character sequence may also be referred to as the second sequence occurrence probability.

[0122] Each candidate character sequence includes at least one character, and each character is one of an entity character and a blank placeholder.

[0123] S303: for each candidate character sequence, determine a placeholder distribution feature of a blank placeholder associated with each entity character in the candidate character sequence.

[0124] S304, for each candidate character sequence, based on the possible distribution features of blank placeholders associated with different entity characters and the occurrence probabilities of the possible distribution features, determine the occurrence probability of the placeholder distribution features corresponding to each entity character in the candidate character sequence.

[0125] For the above steps S301 to S304, reference may be made to the relevant introduction of the previous embodiment, which will not be repeated here.

[0126] S305 , for each candidate character sequence, determining a first sequence occurrence probability of the candidate character sequence based on the occurrence probability of the placeholder distribution feature corresponding to each entity character in the candidate character sequence.

[0127] Among them, the probability of occurrence of the first sequence of the candidate character sequence can be related to the probability of occurrence of the placeholder distribution features corresponding to each entity character in the candidate character sequence. The higher the probability of occurrence of the placeholder distribution features corresponding to each entity character in the candidate character sequence, the higher the probability of occurrence of the first sequence of the candidate character sequence.

[0128] For example, the product or sum of the occurrence probabilities of the placeholder distribution features corresponding to each entity character in the candidate character sequence can be used as the first sequence occurrence probability of the candidate character sequence. Of course, this is just a simple example, and in practice, other factors can be considered to comprehensively determine the first sequence occurrence probability, and there is no limitation to this.

[0129] It can be understood that the higher the probability of occurrence of the first sequence of the candidate character sequence, the higher the accuracy of the candidate character sequence predicted by the CTC model, and the higher the possibility that the candidate character sequence is the text sequence of the speech signal to be recognized.

[0130] S306, determining a text recognition result corresponding to the at least one candidate character sequence by combining the first sequence occurrence probability of the candidate character sequence and the second sequence occurrence probability of the candidate character sequence in the candidate text recognition result.

[0131] For example, the order of the at least one candidate character sequence may be determined according to the first sequence occurrence probability and the second sequence occurrence probability, and a text recognition result including the order may be obtained.

[0132] For another example, one or more candidate character sequences may be selected from at least one candidate character sequence as the text recognition result of the speech signal according to the first sequence occurrence probability and the second sequence occurrence probability of the candidate character sequence.

[0133] It can be understood that the first sequence occurrence probability represents the probability of occurrence of the distribution characteristics of the blank placeholders associated with the entity characters obtained by statistics, and determines the possibility that the candidate character sequence is the character sequence text that truly corresponds to the speech signal to be recognized; and the second sequence occurrence probability is the possibility that the candidate character sequence predicted by the CTC model is the character sequence text that truly corresponds to the speech signal. Therefore, combining the first sequence occurrence probability and the second sequence occurrence probability of the candidate character sequence can more accurately represent the possibility that the candidate character sequence is the character sequence text that truly corresponds to the speech signal.

[0134] It can be seen from this that by combining the first sequence occurrence probability and the second sequence occurrence probability of the candidate character sequence, the candidate text recognition result can be corrected more accurately, thereby obtaining a more accurate text recognition result.

[0135] In a possible implementation, the first sequence occurrence probability of the candidate character sequence may be combined to correct the second sequence occurrence probability of the candidate character sequence to obtain a third sequence occurrence probability of the candidate character sequence.

[0136] For example, a first weight for the first sequence occurrence probability and a second weight for the second sequence occurrence probability can be set in advance. On this basis, for each candidate character sequence, a weighted sum of the first sequence occurrence probability and the second sequence occurrence probability of the candidate character sequence can be calculated, and the weighted sum can be used as the third sequence occurrence probability.

[0137] For another example, the first sequence occurrence probability of the candidate character sequence can be called the set weight coefficient to obtain the weighted first sequence occurrence probability; then the weighted first sequence occurrence probability is added to the second sequence occurrence probability of the candidate character sequence to obtain the third sequence occurrence probability of the candidate character sequence.

[0138] For another example, the product of the second sequence occurrence probability of the candidate character sequence and the first sequence occurrence probability thereof may be determined as the third sequence occurrence probability of the candidate character sequence.

[0139] Of course, there are other possible implementations for determining the third sequence occurrence probability by combining the first sequence occurrence probability and the second sequence occurrence probability of the candidate character sequence, which are not limited thereto.

[0140] Accordingly, a text recognition result corresponding to at least one candidate character sequence may be determined based on the third sequence occurrence probability of the candidate character sequence.

[0141] For example, at least one candidate character sequence may be sorted according to the probability of occurrence of the third sequence of the candidate character sequence to obtain a text recognition result. The text recognition result includes the sorting of the at least one candidate character sequence;

[0142] For another example, the candidate character sequence with the highest third sequence occurrence probability in the at least one candidate character sequence is determined as the text recognition result. Alternatively, the first at least one candidate character sequence with the highest third sequence occurrence probability is selected as the text recognition result.

[0143] It is understandable that in order to improve the accuracy of the CTC model in recognizing speech signals, after obtaining the speech signal to be recognized, the present application can also first extract the audio features of the speech signal and encode it, and then use the CTC model to determine the candidate text recognition result based on the encoding vector of the speech signal. The following is an explanation in conjunction with the flow chart.

[0144] like Figure 4 As shown, it shows another flow chart of the speech recognition method provided by the embodiment of the present application. The method of this embodiment may include:

[0145] S401: Obtain a speech signal to be recognized.

[0146] The speech signal is a speech signal that needs to be subjected to text recognition.

[0147] It is understandable that after the voice signal is collected by an audio collection device such as a microphone of an electronic device, the voice signal can be sampled to obtain a voice signal to be recognized.

[0148] S402: For each speech signal, extract the audio features of the speech signal.

[0149] There are many possible ways to extract the audio features of the speech signal, and this application does not limit this. For example, the Fbank feature of the speech signal can be extracted using the Fbank extraction algorithm. Alternatively, the audio features of the speech signal can be extracted using the Fbank algorithm plus the pitch algorithm.

[0150] S403: For each speech signal, determine a vector code of the speech signal based on the audio feature of the speech signal.

[0151] For example, the vector encoding of the speech signal can be determined based on a recurrent neural network (RNN) model, a long short-term memory model recurrent neural network (LSTM), an attention model, or a transformer model, without any restriction.

[0152] S404: for each speech signal, determine a candidate text recognition result of the speech signal based on the CTC model and the vector encoding of the speech signal.

[0153] For example, the vector encoding of the speech signal can be input into the CTC model to obtain the candidate text recognition result output by the CTC model.

[0154] The candidate text recognition result includes at least one candidate character sequence and a second sequence occurrence probability of the candidate character sequence. Each candidate character sequence includes at least one character, and each character is one of an entity character and a blank placeholder.

[0155] S405: for each candidate character sequence, determine a placeholder distribution feature of a blank placeholder associated with each entity character in the candidate character sequence.

[0156] S406, for each candidate character sequence, based on the possible distribution features of blank placeholders associated with different entity characters and the occurrence probabilities of the possible distribution features, determine the occurrence probability of the placeholder distribution features corresponding to each entity character in the candidate character sequence.

[0157] S407 , for each candidate character sequence, determining a first sequence occurrence probability of the candidate character sequence based on the occurrence probability of the placeholder distribution feature corresponding to each entity character in the candidate character sequence.

[0158] S408, combining the first sequence occurrence probability of the candidate character sequence, correcting the second sequence occurrence probability of the candidate character sequence, and obtaining a third sequence occurrence probability of the candidate character sequence.

[0159] S409: Determine the ranking of the at least one candidate character sequence based on the third sequence occurrence probability of each candidate character sequence.

[0160] It can be understood that steps S407 to S409 are described by taking one possible implementation method of determining the text recognition result as an example, but the other situations mentioned above are also applicable to this embodiment and will not be described in detail here.

[0161] It can be understood that in the speech recognition stage, based on the vector encoding of the speech signal and the use of the CTC model to determine the candidate text recognition model, the present application can also first determine the vector encoding of the speech signal sample when determining the possible distribution characteristics and occurrence probability of the blank placeholders associated with the entity characters, and then use the trained CTC model to determine the character sequence sample of the speech signal sample.

[0162] For easier understanding, see Figure 5 It shows a schematic diagram of the framework of an implementation principle of the solution of the present application.

[0163] Depend on Figure 5 It can be seen that:

[0164] During the training phase:

[0165] After the multiple speech signal samples are respectively subjected to feature extraction and vector encoding, the vector encodings of the multiple speech signal samples can be respectively input into the CTC model to be trained, so as to finally train the CTC model.

[0166] Based on the trained CTC model, at least one character sequence sample of each speech signal sample can be further determined. Accordingly, by counting the distribution characteristics of the blank placeholders associated with each entity character in each character sequence sample, a distribution model of the blank placeholders associated with each entity character can be obtained. The distribution model includes the occurrence probabilities of various possible distribution characteristics of the blank placeholders associated with the entity characters.

[0167] During the speech recognition phase:

[0168] After feature extraction and vector encoding, the speech signal to be recognized can be input into the trained CTC model. Then, the sequence occurrence probability of each candidate character sequence output by the CTC model can be corrected based on the occurrence probability of various possible distribution features associated with blank placeholders of each entity character in the distribution model. In this way, the text recognition result can be obtained based on the corrected sequence occurrence probability corresponding to each candidate character sequence.

[0169] Corresponding to the speech recognition method of the present application, the present application also provides a speech recognition device.

[0170] like Figure 6As shown, it shows a schematic diagram of the composition structure of a speech recognition device provided in an embodiment of the present application. The device of this embodiment may include:

[0171] The speech acquisition unit 601 is used to acquire a speech signal to be recognized;

[0172] A speech recognition unit 602 is used to determine a candidate text recognition result of the speech signal based on a continuous time series classification model, wherein the candidate text recognition result includes at least one candidate character sequence corresponding to the speech signal, wherein the candidate character sequence includes at least one character, and the character is one of an entity character and a blank placeholder;

[0173] A distribution determination unit 603, configured to determine a placeholder distribution feature of a blank placeholder associated with each entity character in the candidate character sequence;

[0174] A probability determination unit 604 is used to determine the occurrence probability of the placeholder distribution feature corresponding to each entity character in the candidate character sequence based on the possible distribution features of the blank placeholders associated with different entity characters and the occurrence probability of the possible distribution features;

[0175] The result determination unit 605 is used to determine the text recognition result based on the at least one candidate character sequence and the occurrence probability of the placeholder distribution feature corresponding to each entity character in the candidate character sequence.

[0176] In a possible implementation, the present application further includes:

[0177] A sample obtaining unit, used for obtaining a plurality of speech signal samples;

[0178] A sample recognition unit, configured to determine at least one character sequence sample corresponding to the speech signal sample based on a trained continuous time series classification model, wherein the character sequence sample includes at least one character sample, and the character sample belongs to one of an entity character and a blank placeholder, and the trained continuous time series classification model is obtained by training based on the plurality of speech signal samples;

[0179] A sample feature determination unit, used to determine the distribution features of the blank placeholders associated with each entity character in the character sequence sample;

[0180] The distribution probability determination unit is used to determine possible distribution features of blank placeholders associated with different entity characters and the occurrence probability of the possible distribution features based on the distribution features of blank placeholders associated with each entity character in the character sequence sample.

[0181] In a possible implementation, the result determination unit includes:

[0182] A probability determination unit, configured to determine a first sequence occurrence probability of the candidate character sequence based on the occurrence probability of the placeholder distribution feature corresponding to each entity character in the candidate character sequence;

[0183] The text determination unit is used to determine a text recognition result corresponding to the at least one candidate character sequence in combination with the first sequence occurrence probability of the candidate character sequence.

[0184] In yet another possible implementation, the candidate text recognition result obtained by the speech recognition unit further includes: a second sequence occurrence probability of the candidate character sequence;

[0185] The text determination unit is specifically configured to determine a text recognition result corresponding to the at least one candidate character sequence by combining the first sequence occurrence probability and the second sequence occurrence probability of the candidate character sequence.

[0186] In yet another possible implementation, the text determination unit includes:

[0187] A probability correction subunit, used to correct the second sequence occurrence probability of the candidate character sequence in combination with the first sequence occurrence probability of the candidate character sequence, so as to obtain the third sequence occurrence probability of the candidate character sequence;

[0188] The text determination subunit is used to determine a text recognition result corresponding to the at least one candidate character sequence based on the third sequence occurrence probability of the candidate character sequence.

[0189] In yet another possible implementation, the text determination subunit includes:

[0190] a first text determination subunit, configured to sort the at least one candidate character sequence according to a third sequence occurrence probability of the candidate character sequence, to obtain a text recognition result, wherein the text recognition result includes the sorting of the at least one candidate character sequence;

[0191] or,

[0192] The second text determination subunit is used to determine the candidate character sequence with the highest occurrence probability of the third sequence in the at least one candidate character sequence as the text recognition result.

[0193] In yet another possible implementation, the blank placeholder associated with the entity character includes: a blank placeholder that is located after the entity character, is adjacent to the entity character, and appears continuously.

[0194] In yet another possible implementation, the speech recognition unit includes:

[0195] A feature extraction unit, used to extract audio features of the speech signal;

[0196] A vector coding unit, configured to determine a vector coding of the speech signal based on the audio feature;

[0197] A model recognition unit is used to determine a candidate text recognition result of the speech signal based on the continuous time series classification model and the vector encoding.

[0198] In another aspect, the present application also provides an electronic device, such as Figure 7 As shown, it shows a schematic diagram of a composition structure of the electronic device, the electronic device can be any type of electronic device, and the electronic device at least includes a memory 701 and a processor 702;

[0199] The processor 702 is used to execute the speech recognition method in any one of the above embodiments.

[0200] The memory 701 is used to store programs required by the processor to perform operations.

[0201] It is understandable that the electronic device may further include a display unit 703 and an input unit 704 .

[0202] Of course, the electronic device may also have Figure 7 There may be more or fewer components, without limitation.

[0203] On the other hand, the present application also provides a computer-readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by a processor to implement the speech recognition method described in any one of the above embodiments.

[0204] The present application also proposes a computer program, which includes computer instructions stored in a computer-readable storage medium. When the computer program is run on an electronic device, it is used to execute the speech recognition method in any of the above embodiments.

[0205] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referred to each other. At the same time, the features recorded in each embodiment in this specification can be replaced or combined with each other, so that professionals in the field can implement or use this application. For device embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0206] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0207] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A speech recognition method, comprising: Obtaining a speech signal to be recognized; Determine a candidate text recognition result of the speech signal based on a continuous time series classification model, wherein the candidate text recognition result includes at least one candidate character sequence corresponding to the speech signal, wherein the candidate character sequence includes at least one character, and the character is one of an entity character and a blank placeholder; Determining a placeholder distribution feature of a blank placeholder associated with each entity character in the candidate character sequence; The blank placeholders associated with the entity character in the candidate character sequence include: blank placeholders that are adjacent to and appear continuously with the entity character in the candidate character sequence; Determine the occurrence probability of the placeholder distribution feature corresponding to each entity character in the candidate character sequence based on the possible distribution features of the blank placeholders associated with different entity characters and the occurrence probability of the possible distribution features; A text recognition result is determined based on the at least one candidate character sequence and the occurrence probability of the placeholder distribution feature corresponding to each entity character in the candidate character sequence.

2. According to the method of claim 1, the possible distribution characteristics of the configured different entity characters associated with the blank placeholders and the occurrence probability of the possible distribution characteristics are obtained by: obtaining a plurality of speech signal samples; Based on the trained continuous time series classification model, determining at least one character sequence sample corresponding to the speech signal sample, the character sequence sample includes at least one character sample, and the character sample belongs to one of an entity character and a blank placeholder, and the trained continuous time series classification model is obtained by training based on the multiple speech signal samples; Determining distribution characteristics of blank placeholders associated with each entity character in the character sequence sample; Based on the distribution characteristics of the blank placeholders associated with each entity character in the character sequence sample, possible distribution characteristics of the blank placeholders associated with different entity characters and the occurrence probabilities of the possible distribution characteristics are determined.

3. The method according to claim 1, wherein determining the text recognition result based on the at least one candidate character sequence and the occurrence probability of the placeholder distribution feature corresponding to each entity character in the candidate character sequence comprises: Determining a first sequence occurrence probability of the candidate character sequence based on the occurrence probability of the placeholder distribution feature corresponding to each entity character in the candidate character sequence; In combination with the first sequence occurrence probability of the candidate character sequence, a text recognition result corresponding to the at least one candidate character sequence is determined.

4. According to the method of claim 3, the candidate text recognition result further includes: A probability of occurrence of a second sequence of the candidate character sequence; Determining a text recognition result corresponding to the at least one candidate character sequence in combination with the first sequence occurrence probability of the candidate character sequence includes: The text recognition result corresponding to the at least one candidate character sequence is determined by combining the first sequence occurrence probability and the second sequence occurrence probability of the candidate character sequence.

5. The method according to claim 4, wherein the combining the first sequence occurrence probability and the second sequence occurrence probability of the candidate character sequence to determine the text recognition result corresponding to the at least one candidate character sequence comprises: In combination with the first sequence occurrence probability of the candidate character sequence, the second sequence occurrence probability of the candidate character sequence is corrected to obtain a third sequence occurrence probability of the candidate character sequence; Based on the third sequence occurrence probability of the candidate character sequence, a text recognition result corresponding to the at least one candidate character sequence is determined.

6. The method according to claim 5, determining a text recognition result corresponding to the at least one candidate character sequence based on the third sequence occurrence probability of the candidate character sequence, comprising: sorting the at least one candidate character sequence according to the third sequence occurrence probability of the candidate character sequence to obtain a text recognition result, wherein the text recognition result includes the sorting of the at least one candidate character sequence; or, The candidate character sequence with the highest occurrence probability of the third sequence in the at least one candidate character sequence is determined as the text recognition result.

7. The method according to claim 1, wherein the blank placeholder associated with the entity character comprises: A blank placeholder that is located after the entity character, adjacent to the entity character, and appears continuously.

8. The method according to claim 1, wherein determining the candidate text recognition result of the speech signal based on the continuous time series classification model comprises: Extracting audio features of the speech signal; Determining a vector encoding of the speech signal based on the audio feature; Based on the continuous time series classification model and the vector encoding, a candidate text recognition result of the speech signal is determined.

9. A speech recognition device, comprising: A speech acquisition unit, used for acquiring a speech signal to be recognized; A speech recognition unit, configured to determine a candidate text recognition result of the speech signal based on a continuous time series classification model, wherein the candidate text recognition result includes at least one candidate character sequence corresponding to the speech signal, wherein the candidate character sequence includes at least one character, and the character is one of an entity character and a blank placeholder; a distribution determination unit, configured to determine a placeholder distribution feature of a blank placeholder associated with each entity character in the candidate character sequence; The blank placeholders associated with the entity character in the candidate character sequence include: blank placeholders that are adjacent to and appear continuously with the entity character in the candidate character sequence; A probability determination unit, configured to determine the occurrence probability of the placeholder distribution feature corresponding to each entity character in the candidate character sequence based on the possible distribution features of the blank placeholders associated with different entity characters and the occurrence probability of the possible distribution features; The result determination unit is used to determine the text recognition result based on the at least one candidate character sequence and the occurrence probability of the placeholder distribution feature corresponding to each entity character in the candidate character sequence.

10. An electronic device comprising at least a processor and a memory; in, The processor is used to execute the speech recognition method according to any one of claims 1 to 8 above; The memory is used to store programs required by the processor to perform operations.

Images