Speech recognition method and device based on phoneme error correction, medium and equipment

Abstract

The invention discloses a speech recognition method and device based on phoneme error correction, a medium and equipment, and the method comprises the steps: carrying out the sound feature extraction of a to-be-recognized speech signal, and obtaining an audio signal feature; performing phoneme recognition based on the audio signal features to obtain a phoneme sequence; performing speaker phoneme error correction processing on the phoneme sequence to obtain a phoneme recognition sequence; and performing text mapping processing on the phoneme recognition sequence to obtain a speech recognition result. According to the method disclosed by the invention, the influence of wrong pronunciation on speech recognition is solved by correcting the phonemes, so that the recognition result is closer to real text information, and the speech recognition accuracy of the speaker is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to a speech recognition method, device, medium and equipment based on phoneme error correction. Background Art

[0002] In ASR (Automatic Speech Recognition, language recognition technology), for speech recognition problems, the existing technology converts the input audio signal into text information through various means, such as TDNN network, GMM-HMM model, etc., but due to the pronunciation habits of the speaker, the pronunciation of some texts is different from the normal pronunciation, resulting in pronunciation errors, and then the text corresponding to the speaker's voice cannot be accurately recognized, resulting in poor speech recognition effect.

[0003] Therefore, how to improve the accuracy of speaker speech recognition is a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of the above problems, the present invention is proposed to provide a method, device, medium and equipment for speech recognition based on phoneme error correction that overcome the above problems or at least partially solve the above problems.

[0005] Other features and advantages of the present invention will become apparent from the following detailed description, or may be learned in part by practice of the present invention.

[0006] According to a first aspect of an embodiment of the present invention, a speech recognition method based on phoneme error correction is provided, and the speech recognition method based on phoneme error correction includes:

[0007] Performing sound feature extraction processing on the speech signal to be recognized to obtain audio signal features;

[0008] Perform phoneme recognition based on the audio signal features to obtain a phoneme sequence;

[0009] Performing speaker phoneme error correction processing on the phoneme sequence to obtain a phoneme recognition sequence;

[0010] The phoneme recognition sequence is subjected to text mapping processing to obtain a speech recognition result.

[0011] According to a second aspect of an embodiment of the present invention, there is provided a speech recognition device based on phoneme error correction, the device comprising:

[0012] An audio feature extraction module is used to extract the sound features of the speech signal to be recognized to obtain the audio signal features;

[0013] A phoneme recognition module, used to perform phoneme recognition based on the audio signal features to obtain a phoneme sequence;

[0014] A phoneme error correction module, used for performing speaker phoneme error correction processing on the phoneme sequence to obtain a phoneme recognition sequence;

[0015] The text mapping module is used to perform text mapping processing on the phoneme recognition sequence to obtain a speech recognition result.

[0016] According to a third aspect of an embodiment of the present invention, a computer-readable storage medium is provided, in which computer program instructions are stored. The computer program instructions are loaded and executed by a processor to implement the operations performed by any of the methods described above.

[0017] According to a fourth aspect of an embodiment of the present invention, there is provided an electronic device, comprising a processor and a memory, wherein the memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, it implements instructions of any of the methods described above.

[0018] The technical solution provided in the embodiments of the present invention has at least the following technical effects or advantages:

[0019] The embodiments of the present invention provide a speech recognition method, device, medium and equipment based on phoneme error correction. The method performs speaker phoneme error correction on the phoneme sequence to obtain a phoneme recognition sequence; the phoneme recognition sequence is subjected to text mapping processing to obtain a speech recognition result. By correcting phonemes, the influence of incorrect pronunciation on speech recognition is resolved, making the recognition result closer to the real text information, and effectively improving the speaker's speech recognition accuracy.

[0020] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A flowchart of a speech recognition method based on phoneme error correction provided by an embodiment of the present invention;

[0023] Figure 2 This is a reference architecture diagram of the phoneme recognition model;

[0024] Figure 3 A schematic diagram of the principle structure of a speech recognition device based on phoneme error correction provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings.

[0026] Various structural schematic diagrams according to embodiments of the present disclosure are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may further design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0027] In the context of the present disclosure, when a layer / element is referred to as being "on" another layer / element, the layer / element may be directly on the other layer / element, or there may be an intervening layer / element therebetween. Additionally, if a layer / element is "on" another layer / element in one orientation, the layer / element may be "below" the other layer / element when the orientation is reversed. In the context of the present disclosure, similar or identical components may be denoted by the same or similar reference numerals.

[0028] In order to better understand the above-mentioned technical scheme, the above-mentioned technical scheme will be described in detail below in combination with specific implementation methods. It should be understood that the embodiments of the present disclosure and the specific features in the embodiments are detailed descriptions of the technical scheme of the present invention, rather than limitations on the technical scheme of the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0029] Figure 1 is a flow chart of a speech recognition method based on phoneme error correction provided by an embodiment of the present invention, such as Figure 1 As shown, the speech recognition method based on phoneme error correction includes the following steps:

[0030] S1, extracting and processing the sound features of the speech signal to be recognized to obtain audio signal features;

[0031] In the embodiment of the present invention, the speech signal to be recognized is a digital speech signal that needs to be recognized, and is a digital speech signal corresponding to the speech audio of a single speaker, which can be obtained by performing analog-to-digital conversion or digital acquisition on the speech audio of the single speaker. In the embodiment of the present invention, the speech signal to be recognized is preprocessed to obtain audio signal features, for example, by framing operation, so that each frame of audio signal data can finally correspond to a phoneme, so as to be applied in the subsequent phoneme recognition process.

[0032] Specifically, the embodiment of the present invention performs sound feature extraction processing on the speech signal to be recognized to obtain the audio signal feature, including: framing the speech signal to be recognized to obtain at least one frame of audio signal data; pre-emphasizing each frame of the audio signal data and adding a Hamming window; performing energy extraction on the audio signal data after pre-emphasis and adding a Hamming window to obtain a frame energy feature value; using a filter to process the frame energy feature value to obtain a frame audio signal feature, and summarizing all the frame audio signal features to obtain the audio signal feature.

[0033] The method for extracting the audio signal features in the embodiment of the present invention may be to extract MFCC (Mel frequency cepstral coefficient) features or Fbank (FilterBank filter bank) features; for example, if the method for extracting the audio signal features is to extract Fbank features, the embodiment of the present invention obtains at least one frame of audio signal data by performing frame processing on the speech signal to be recognized, and pre-emphasizes and adds a Hamming window to each frame of the audio signal data, thereby dividing the continuous speech signal into multiple short time periods (frames), and each frame is processed by a window function to reduce energy leakage during spectrum analysis, and at the same time, the energy of the high-frequency part of the signal is increased by cooperating with the pre-emphasis processing to increase the probability of speech recognition of favorable features; then, based on energy extraction of the audio signal data after pre-emphasis and adding a Hamming window, a frame energy feature value is obtained. , the frame energy characteristic value may be a power spectrum or an energy spectrum; in order to simulate the auditory characteristics of the human ear, the embodiment of the present invention may utilize a FilterBank filter to process the frame energy characteristic value to obtain a frame audio signal characteristic (Fbank characteristic), the filter may also be other filters, such as a Mel filter, etc., each frame of audio signal data corresponds to a frame audio signal characteristic, and the expression form of the frame audio signal characteristic may be represented by a vector; further, all the frame audio signal characteristics are summarized to obtain the audio signal characteristics, and the embodiment of the present invention does not limit the form of the audio signal characteristics, which may be a frame audio signal feature set or a matrix form, for example: all the frame audio signal characteristics are sequentially combined into a matrix form according to the frame order of the corresponding audio signal data to obtain the audio signal characteristics.

[0034] In some embodiments of the present invention, if the method for extracting the audio signal features is to extract MFCC features, the filter at this time can be a Mel filter. After using the filter to process the frame energy feature value, the output of the filter is logarithmic and a discrete cosine transform (DCT) is performed to obtain a set of coefficients, which are the audio signal features (MFCC features).

[0035] S2. Perform phoneme recognition based on the audio signal features to obtain a phoneme sequence;

[0036] In order to recognize the speech signal to be recognized as text in the embodiment of the present invention, the corresponding phonemes need to be recognized. Therefore, phoneme recognition is performed based on the audio signal features to obtain a phoneme sequence.

[0037] In an embodiment of the present invention, phoneme recognition is performed based on the audio signal features to obtain a phoneme sequence, including: performing phoneme recognition analysis on each framed audio signal feature in the audio signal features to obtain a phoneme probability feature value of each phoneme in a preset phoneme set corresponding to each framed audio signal feature; performing phoneme combination decoding based on the phoneme probability feature value to screen and combine the phonemes in the preset phoneme set to obtain an original phoneme sequence; and performing smoothing filtering on the original phoneme sequence to obtain the phoneme sequence.

[0038] Furthermore, in the embodiment of the present invention, the phoneme combination decoding is performed based on the phoneme probability feature value to screen and combine the phonemes in the preset phoneme set to obtain the original phoneme sequence, including:

[0039] The phonemes in the preset phoneme set are screened based on the phoneme probability feature values ​​to obtain the screened phonemes corresponding to the framed audio signal features of each frame; and the screened phonemes corresponding to all the framed voice signal features are combined into a sequence according to the frame order of the audio signal data corresponding to the framed voice signal features to obtain the original sequence of phonemes.

[0040] Specifically, the embodiment of the present invention can use the phoneme recognition model to perform phoneme recognition analysis on the framed audio signal features in the audio signal features to obtain the phoneme probability feature value of each phoneme in the preset phoneme set corresponding to each framed audio signal feature.

[0041] Optionally, in the embodiment of the present invention, the phoneme to which the maximum phoneme probability feature value in a preset phoneme set corresponds to the framed audio signal feature is determined as the screening phoneme corresponding to the framed audio signal feature.

[0042] The phoneme recognition model is obtained by training with a pre-collected data set and is arranged in the storage-computing integrated chip. In an embodiment of the present invention, the phoneme recognition model may be, for example, a TDNN network structure, in which the hidden layer is composed of a one-dimensional convolution kernel. This structure enables the phoneme recognition model to process input signals with time delays, thereby extracting features from time series data. The output I / Q signal of the TDNN network structure can be represented by an excitation function using input I / Q signals and signals with different multi-tap delays. This structure helps the model to learn and train.

[0043] It should be explained that the preset phoneme set is related to the language category of phoneme recognition. For example, if phoneme recognition is to recognize Chinese phonemes, then the preset phoneme set has a total of 218 phonemes, including the 217 phonemes contained in Chinese and silent phonemes.

[0044] Specifically, the embodiment of the present invention can use the phoneme recognition model to perform phoneme recognition analysis on each sub-frame audio signal feature in the audio signal feature, see Figure 2 As shown in FIG. 1 , a reference architecture diagram of a phoneme recognition model is shown. The phoneme recognition model includes multiple layers, wherein:

[0045] In the input layer, the model processes the same number of framed speech signal features each time until all framed speech signal features are processed. If the number is insufficient, silent frame features will be used to make up for it.

[0046] The layer layer also includes the fully connected layer, the normalization layer and the activation layer. The framed speech signal features input by the input layer will be sent to multiple layers. The features are extracted by considering the framed speech signal features of the current frame and the framed speech signal features of the surrounding frames, so that the dynamic changes in time can be captured.

[0047] In the network structure of the TDNN, each layer will perform a frame extraction operation, so that the network structure of the TDNN selectively focuses on the framed speech signal features of certain specific frames and ignores others. Through this frame extraction operation, the network structure of the TDNN can focus on the framed speech signal features with more information, ignore redundant parts, and capture long-term and short-term time dependencies, thereby improving the accuracy of phoneme recognition. The frame extraction operation can be achieved by setting each layer to apply a different time step (time stride), so that the network structure of the TDNN can capture features on different time scales.

[0048] The network structure of the TDNN outputs a recognized phoneme recognition matrix based on the extracted audio signal features after passing through the fully connected layer, the normalization layer and the activation layer. In an embodiment of the present invention, the phoneme recognition model recognizes Chinese phonemes, and the output dimension of the linear layer (fully connected layer) of the network structure of the TDNN is 218.

[0049] For example: the phoneme recognition network inputs five framed speech signal features, 1, 2, 3, 4, and 5, and the input arrangement order of the framed speech signal features is 1, 2, 3, 4, and 5. Then the phoneme recognition network will output a 5*X phoneme recognition matrix (X is the number of phonemes in the preset phoneme set, and each row in the phoneme recognition matrix corresponds to a phoneme in the preset phoneme set. For example, if the phoneme recognition network recognizes Chinese phonemes, then the preset X is 218). Then all the elements in the first column of the phoneme recognition matrix are the phoneme recognition analysis of the framed speech signal feature 1, and the phoneme probability feature values ​​of each phoneme in the preset phoneme set for the framed speech signal feature 1 are obtained (the same applies to the 2nd, 3rd, 4th, and 5th columns), and the phoneme corresponding to the row where the largest element in the first column is located is confirmed as the screening phoneme corresponding to the framed speech signal feature 1.

[0050] Optionally, in other embodiments of the present invention, the phoneme combination decoding based on the phoneme probability feature value is performed to screen and combine the phonemes in the preset phoneme set to obtain the original phoneme sequence, including: selecting a phoneme in the preset phoneme set as the possible phoneme corresponding to the framed speech signal feature; combining the possible phonemes corresponding to all the framed speech signal features into a sequence in the form of a signal according to the frame order of the audio signal data corresponding to the framed speech signal feature to obtain a possible phoneme sequence; based on the phoneme probability feature value corresponding to each possible phoneme, using a preset dynamic programming algorithm (such as the Viterbi algorithm) to perform maximum likelihood estimation decoding to find the possible phoneme sequence with the highest probability, and use it as the original phoneme sequence.

[0051] Furthermore, in order to make the phoneme sequence more concise and avoid redundant phonemes caused by noise, in the embodiment of the present invention, the original phoneme sequence is smoothed and filtered to obtain the phoneme sequence.

[0052] Specifically, in the embodiment of the present invention, the smoothing filtering of the original phoneme sequence to obtain the phoneme sequence includes:

[0053] Deleting repeated non-silent phonemes that appear continuously in the original phoneme sequence to obtain a filtered phoneme sequence;

[0054] Transient phoneme deletion is performed on the first phoneme sequence to obtain the phoneme sequence.

[0055] It should be noted that short-lived phonemes are non-silent phonemes whose consecutive repetition times in the original phoneme sequence are less than a preset threshold. For example, the original phoneme sequence is ['a','a','a','b','b','c','c','c','d','d','e','e','e','e','f','f','f','g','g'], and the filtered phoneme sequence is ['a','b','c','d','e','f','g'], and the preset threshold is 3 times. If the phoneme 'd' is repeated twice in a row, which is less than 3 times, it is a short-lived phoneme and needs to be deleted. Therefore, the phoneme sequence is ['a','b','c','e','f','g'].

[0056] It should be noted that, in some other embodiments of the present invention, the original phoneme sequence may be directly used as the phoneme sequence.

[0057] S3, performing speaker phoneme error correction processing on the phoneme sequence to obtain a phoneme recognition sequence;

[0058] In the embodiment of the present invention, in order to avoid mispronounced phonemes in the phoneme sequence, it is necessary to perform speaker phoneme error correction processing on the phoneme sequence to obtain a phoneme recognition sequence.

[0059] Specifically, the embodiment of the present invention performs speaker phoneme error correction processing on the phoneme sequence to obtain a phoneme recognition sequence including:

[0060] Perform voiceprint feature extraction on the audio signal feature to obtain a speaker recognition voiceprint; perform voiceprint matching processing on the speaker recognition voiceprint and a recorded voiceprint in a preset recorded voiceprint set to determine a voiceprint matching result; when the voiceprint matching result is a successful match, perform phoneme error correction processing on the phoneme sequence to obtain the phoneme recognition sequence; when the voiceprint matching result is an unsuccessful match, use the phoneme sequence as the phoneme recognition sequence.

[0061] Optionally, an embodiment of the present invention performs voiceprint feature extraction on the audio signal features to obtain a speaker recognition voiceprint, including: inputting the audio signal features into a voiceprint recognition model for linear transformation, performing pooling processing on the phoneme sequence after the linear transformation to obtain a pooling processing result; performing length compression on the pooling processing result; performing feature vector extraction on the pooling processing result after length compression, performing mean-variance normalization on all extracted features, and integrating them into a feature vector to obtain the speaker recognition voiceprint.

[0062] In an embodiment of the present invention, the speaker recognition voiceprint is a high-dimensional vector, and the speaker recognition voiceprint represents the characteristics of the speech signal to be recognized, for example, a 128-dimensional or 256-dimensional real number vector, and the real number vector is used to distinguish different speakers. Based on actual application requirements, the present invention can first determine the vector dimension of the speaker recognition voiceprint output by the required voiceprint recognition model, for example, the vector dimension of the speaker recognition voiceprint is determined to be 128 dimensions or 256 dimensions.

[0063] The embodiment of the present invention inputs the audio signal feature into the voiceprint recognition model for linear transformation to reduce the pooling parameters; the pooling processing result after the linear transformation is pooled to obtain the pooling processing result. The pooling processing can adopt attention statistical pooling (Attentive Statistic Pooling), statistical pooling (Statistics pooling), etc. The pooling processing can reduce the spatial dimension of the data while retaining important feature information.

[0064] Since the sequence length or vector dimension of the pooling processing result after the pooling processing may not meet the requirements, the length of the pooling processing result is compressed according to the determined vector dimension to obtain a sequence of dimensions of a specified size. The embodiment of the present invention then extracts feature vectors from the pooling processing result after length compression, and performs mean-variance normalization on all extracted features to ensure that the extracted features are on the same scale. All the extracted features are then integrated into a feature vector based on time series. For example, the context information can be captured through an attention layer to achieve integration of time series features to obtain the speaker recognition voiceprint.

[0065] The speaker recognition voiceprint is a voiceprint feature vector extracted from the phoneme sequence, and the recorded voiceprint set is a set of the recorded voiceprints. In the embodiment of the present invention, the speaker recognition voiceprint is matched with the recorded voiceprints in the preset recorded voiceprint set, for example, the speaker recognition voiceprint is calculated with the recorded voiceprint, and the matching result is determined according to their Euclidean distance or PLDA (Probabilistic Linear Discriminant Analysis), and whether the matching is successful is determined according to the matching result, and a voiceprint matching result is generated. Taking the Euclidean distance as an example, the Euclidean distance between the speaker recognition voiceprint and each recorded voiceprint is calculated. When there is a recorded voiceprint with a Euclidean distance less than a preset distance threshold, the voiceprint matching result is a successful match (if there is no recorded voiceprint with a Euclidean distance less than the preset distance threshold, the voiceprint matching result is a failed match), and the recorded voiceprint with the smallest Euclidean distance is the recorded voiceprint that successfully matches the speaker recognition voiceprint.

[0066] It should be noted that the recorded voiceprint is the voiceprint of a certain speaker, different recorded voiceprints represent voiceprints of different speakers, the recorded voiceprint corresponds to the speaker one-to-one, and the recorded voiceprint is of the same type as the speaker recognition voiceprint.

[0067] When the voiceprint matching result is a successful match, a phoneme error correction process is performed on the phoneme sequence to obtain the phoneme recognition sequence.

[0068] Specifically, in an embodiment of the present invention, when the voiceprint matching result is a successful match, the recorded voiceprint in the recorded voiceprint set that successfully matches the speaker recognition voiceprint is determined as the target recorded voiceprint; the speaker phoneme correction table corresponding to the target recorded voiceprint is obtained to obtain a target phoneme correction table, wherein the target phoneme correction table includes at least one error-prone phoneme and an error correction phoneme corresponding to each error-prone phoneme; the error-prone phonemes existing in the phoneme sequence are replaced with the error correction phonemes corresponding to them in the target phoneme correction table to obtain the phoneme recognition sequence.

[0069] It should be explained that the recorded voiceprint set is a collection of voiceprints of speakers who are prone to mispronunciation. In order to facilitate phoneme error correction for these speakers, each recorded voiceprint has a corresponding speaker phoneme error correction table (including phonemes that are prone to mispronunciation by the speaker (i.e., mispronunciation phonemes) and their corresponding correctly pronounced phonemes (i.e., error correction phonemes)). For example: recorded voiceprint A is the voiceprint of speaker A, and recorded voiceprint A corresponds to speaker phoneme error correction table A. Speaker phoneme error correction table A contains phonemes that are prone to mispronunciation by speaker A (i.e., mispronunciation phonemes) and their corresponding correctly pronounced phonemes (i.e., error correction phonemes).

[0070] The target phoneme error correction table is shown in Table 1 below:

[0071] Table 1 Target phoneme error correction table

[0072] Error-prone phonemes Error Correction Phonemes ua1 uai1 ua3 uai3 ua5 u5

[0073] The embodiment of the present invention confirms that the target in the voiceprint comparison module has entered the speaker phoneme correction table corresponding to the voiceprint, and obtains the target phoneme correction table based on the speaker phoneme correction table, performs error correction processing on the phoneme sequence, and replaces the error-prone phonemes in the phoneme sequence with the corresponding error correction phonemes in the target phoneme correction table to obtain the phoneme recognition sequence.

[0074] It should be noted that in other embodiments of the present invention, there may be a situation where the voiceprint matching result is an unsuccessful match. In this case, there is no need to perform a phoneme correction operation. The phoneme sequence is used as the phoneme recognition sequence, and the phoneme recognition sequence is mapped into text to achieve speech recognition.

[0075] In an embodiment of the present invention, the voiceprint recognition model extracts the voiceprint feature vector from the phoneme sequence, and the output speaker recognition voiceprint is a voiceprint matrix, and its linear layer (fully connected layer) output dimension is 128 or 256; the voiceprint recognition model is, for example, an ECAPA-TDNN model, a GMM-UBM model, an i-vector model, etc., which has a variety of output conditions and can directly output speaker classification results, as well as the function of extracting the voiceprint feature vector from the phoneme sequence. It can be selected according to actual application requirements, and the present invention is not limited to this.

[0076] And when the voiceprint recognition model is, for example, an ECAPA-TDNN model, the embodiment of the present invention can also adopt the network structure of TDNN, including a fully connected layer, a normalization layer, an activation layer, etc., and its structure can refer to the aforementioned content and will not be repeated here.

[0077] Since the structures of the phoneme recognition model and the voiceprint recognition model are similar, and the core architectures of the phoneme recognition model and the voiceprint recognition model structures both include convolutional layers and fully connected layers, etc., which are suitable for the use scenarios of the storage-computing integrated chip, based on this, the embodiments of the present invention can respectively deploy the phoneme error correction model and the voiceprint recognition model on the storage-computing integrated chip, thereby accelerating the running speed of the model.

[0078] In an embodiment of the present invention, the phoneme recognition model and the voiceprint recognition model can be respectively set in different functional areas on the same storage and computing integrated chip, each functional area is a memristor array, and each functional area performs corresponding functions respectively; in other embodiments of the present invention, the phoneme recognition model and the voiceprint recognition model can also be respectively set in two storage and computing integrated chips packaged together, and the chip with storage and computing integrated architecture is used as the underlying computing device, which can use parallel acceleration to improve the computing efficiency of the convolutional layer and the fully connected layer in the model, accelerate the calculation, and thus speed up the efficiency of speech recognition.

[0079] S4. Perform text mapping processing on the phoneme recognition sequence to obtain a speech recognition result.

[0080] In the embodiment of the present invention, the phoneme recognition sequence needs to be subjected to text mapping processing, so as to map the phonemes into text and obtain a speech recognition result.

[0081] In detail, an embodiment of the present invention performs text mapping processing on the phoneme recognition sequence to obtain a speech recognition result, including: performing text mapping processing on the phoneme recognition sequence to obtain multiple text sequences and probability values ​​corresponding to each of the text sequences; selecting the text sequence with the largest probability value as the speech recognition result.

[0082] Specifically, the embodiment of the present invention can perform text mapping processing on the phoneme recognition sequence through a language model, input the phoneme recognition sequence into the language model, and the language model maps the phonemes in the phoneme recognition sequence to text, generates a possible text sequence, and then calculates the probability of each text sequence, and selects the text sequence with the largest probability value as the speech recognition result. The language model is, for example, an n-gram model or a deep learning model. Taking the language model as an n-gram model as an example, the language model is a probability model obtained by counting the frequencies of each word in a preset training text, and its principle is to infer the nth word through the first n-1 words, and the principle is shown in the following reference:

[0083] Define a sequence s=(w1,…,w n ), then its probability value is:

[0084]

[0085] Among them, p is the probability value. Given the first k-1 words w1, w2, ..., w k-1 In the case of k |w1 k-1 ) is the kth word w k The conditional probability of occurrence; p(w1 n ) is p(w1......w n ), representing the sequence w1 to w n The joint probability of simultaneous occurrence; k and n are both positive integers; it should be noted that, in order to improve the calculation efficiency, the embodiment of the present invention can only consider w k The first two words of w k-1 ,w k-2 The probability value, that is, p(w k |w1 k-1 )=p(w k |w k-1 w k-2 ).

[0086] Taking the corrected phoneme recognition sequence: u3 y an2 m o2 x ing2 as an example, the calculation process is shown in the following reference:

[0087] There are four text sequences generated by the speech model, namely: language model, language magic, Swift Ink Punishment, etc., p(language model)>p(language magic)>p(Swift Ink Punishment), select the text sequence with the highest probability value p, then the text sequence "language model" is the speech recognition result.

[0088] Compared with the prior art, the speech recognition method based on phoneme error correction described in the embodiment of the present invention performs speaker phoneme error correction processing on the phoneme sequence to obtain a phoneme recognition sequence; and performs text mapping processing on the phoneme recognition sequence to obtain a speech recognition result. By correcting phonemes, the influence of incorrect pronunciation on speech recognition is solved, making the recognition result closer to the real text information, and effectively improving the speaker's speech recognition accuracy.

[0089] Based on the above embodiment, as the above Figure 1 The present invention provides an embodiment of a speech recognition device based on phoneme error correction, and the device embodiment is similar to Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices, see Figure 3 As shown, the speech recognition device based on phoneme error correction includes:

[0090] The audio feature extraction module 100 is used to extract the sound features of the speech signal to be recognized to obtain the audio signal features;

[0091] A phoneme recognition module 200, configured to perform phoneme recognition based on the audio signal features to obtain a phoneme sequence;

[0092] A phoneme error correction module 300 is used to perform speaker phoneme error correction processing on the phoneme sequence to obtain a phoneme recognition sequence;

[0093] The text mapping module 400 is used to perform text mapping processing on the phoneme recognition sequence to obtain a speech recognition result.

[0094] The speech recognition device based on phoneme error correction described in the embodiment of the present invention can execute the speech recognition method based on phoneme error correction provided in the above embodiment. The speech recognition device based on phoneme error correction has the corresponding functional steps and beneficial effects of the speech recognition method based on phoneme error correction described in the above embodiment. Please refer to the embodiment of the speech recognition method based on phoneme error correction for details. The embodiment of the present invention will not be repeated here.

[0095] The embodiment of the present invention also provides an electronic device, which may include a processor and a memory, wherein the processor and the memory may be connected via a bus or other means. The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components and other chips, or a combination of the above-mentioned various types of chips. The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the speech recognition method based on phoneme error correction in the embodiment of the present invention. The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, the speech recognition method based on phoneme error correction in the above method embodiment is realized.

[0096] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. The one or more modules are stored in the memory, and when executed by the processor, the speech recognition method based on phoneme error correction in the above method embodiment is executed. The specific details of the above electronic device can be understood by corresponding to the corresponding descriptions and effects in the above method embodiment, and will not be repeated here. Those skilled in the art can understand that all or part of the processes in the above embodiment method can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and the program, when executed, may include the processes of the embodiments of the above methods. Among them, the storage medium can be a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memory.

[0097] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0098] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those expressly recited in each claim. Rather, as reflected in the claims, inventive aspects lie in less than all of the features of the individual embodiments previously disclosed. Therefore, the claims that follow the detailed description are hereby expressly incorporated into the detailed description, with each claim itself serving as a separate embodiment of the present invention.

[0099] It should be noted that the above-mentioned embodiments illustrate rather than limit the invention and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims.

Claims

1. A speech recognition method based on phoneme error correction, characterized in that: The method comprises: Performing sound feature extraction processing on the speech signal to be recognized to obtain audio signal features; Perform phoneme recognition based on the audio signal features to obtain a phoneme sequence; Performing speaker phoneme error correction processing on the phoneme sequence to obtain a phoneme recognition sequence; The phoneme recognition sequence is subjected to text mapping processing to obtain a speech recognition result.

2. The speech recognition method based on phoneme error correction according to claim 1, characterized in that: The step of performing speaker phoneme error correction processing on the phoneme sequence to obtain a phoneme recognition sequence includes: Extracting voiceprint features from the audio signal features to obtain a speaker recognition voiceprint; Performing voiceprint matching processing on the speaker recognition voiceprint and the recorded voiceprint in the preset recorded voiceprint set to determine the voiceprint matching result; When the voiceprint matching result is a successful match, performing phoneme error correction processing on the phoneme sequence to obtain the phoneme recognition sequence; When the voiceprint matching result is an unsuccessful match, the phoneme sequence is used as the phoneme recognition sequence.

3. The speech recognition method based on phoneme error correction according to claim 2, characterized in that: The extracting voiceprint features from the audio signal features to obtain the speaker recognition voiceprint comprises: Inputting the audio signal feature into a voiceprint recognition model for linear transformation, and performing pooling processing on the phoneme sequence after the linear transformation to obtain a pooling processing result; Performing length compression on the pooling processing result; The pooling processing result after length compression is subjected to feature vector extraction, all extracted features are subjected to mean-variance normalization, and then integrated into a feature vector based on time series to obtain the speaker recognition voiceprint.

4. The speech recognition method based on phoneme error correction according to claim 2, characterized in that: The performing phoneme error correction processing on the phoneme sequence to obtain the phoneme recognition sequence comprises: Determine the recorded voiceprint in the recorded voiceprint set that successfully matches the speaker recognition voiceprint as the target recorded voiceprint; Acquire a speaker phoneme error correction table corresponding to a target recorded voiceprint to obtain a target phoneme error correction table, wherein the target phoneme error correction table includes at least one error-prone phoneme and an error correction phoneme corresponding to each error-prone phoneme; The error-prone phonemes in the phoneme sequence are replaced with the corresponding error-correcting phonemes in the target phoneme error-correcting table to obtain the phoneme recognition sequence.

5. The speech recognition method based on phoneme error correction according to claim 1, characterized in that: The performing text mapping processing on the phoneme recognition sequence to obtain a speech recognition result comprises: Performing text mapping processing on the phoneme recognition sequence to obtain a plurality of text sequences and probability values ​​corresponding to each of the text sequences; The text sequence with the largest probability value is selected as the speech recognition result.

6. The speech recognition method based on phoneme error correction according to claim 1, characterized in that: The sound feature extraction process of the speech signal to be recognized is performed to obtain the audio signal features, including: Performing frame processing on the speech signal to be recognized to obtain at least one frame of audio signal data; Pre-emphasize each frame of the audio signal data and add a Hamming window; Performing energy extraction on the audio signal data after pre-emphasis and adding a Hamming window to obtain a frame energy characteristic value; The frame energy characteristic values ​​are processed by using a filter to obtain frame audio signal characteristics, and all the frame audio signal characteristics are summarized to obtain the audio signal characteristics.

7. The speech recognition method based on phoneme error correction according to claim 1, characterized in that: The performing phoneme recognition based on the audio signal feature to obtain a phoneme sequence includes: Performing phoneme recognition analysis on each sub-frame audio signal feature in the audio signal feature to obtain a phoneme probability feature value of each phoneme in a preset phoneme set corresponding to each sub-frame audio signal feature; Performing phoneme combination decoding based on the phoneme probability feature value to screen and combine the phonemes in the preset phoneme set to obtain an original phoneme sequence; The original phoneme sequence is smoothed and filtered to obtain the phoneme sequence.

8. A speech recognition device based on phoneme error correction, applied to the method according to any one of claims 1 to 7, characterized in that: The device comprises: An audio feature extraction module is used to extract the sound features of the speech signal to be recognized to obtain the audio signal features; A phoneme recognition module, used to perform phoneme recognition based on the audio signal features to obtain a phoneme sequence; A phoneme error correction module, used for performing speaker phoneme error correction processing on the phoneme sequence to obtain a phoneme recognition sequence; The text mapping module is used to perform text mapping processing on the phoneme recognition sequence to obtain a speech recognition result.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which are loaded and executed by a processor to implement the operations performed by the method according to any one of claims 1 to 7.

10. An electronic device comprising a processor and a memory, characterized in that: The memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, the processor implements instructions of any one of the methods described in claims 1-7.

Images